Large language models like GPT-3 are known for their ability to generate coherent and informative natural language texts. However, there are limitations to encoding all necessary knowledge in these models, which can lead to memory distortion and hallucinations. To address these challenges, a system called LLM-AUGMENTER has been proposed. This system augments a black-box LLM with plug-and-play modules to ground its responses in external knowledge. It also uses iterative prompt revision to improve the factuality of its responses. The effectiveness of LLM-AUGMENTER has been validated in task-oriented dialog and question-answering scenarios. The system involves three main steps: retrieving evidence from external sources, prompting the LLM with the evidence, and iterating the prompt based on feedback. The performance of LLM-AUGMENTER has been evaluated using automatic metrics and human evaluations. Overall, LLM-AUGMENTER significantly reduces hallucinations without sacrificing the quality of the responses. The source code and models of the system are publicly available. Check out the paper and project for more information.