DIDACT: Training AI Models for Software Development
Did you know that software isn’t created all at once? It goes through a series of steps and improvements before it becomes good enough to be merged into a code repository. And software engineering is not a solo process, but a collaboration between human developers, code reviewers, and various tools. That’s where DIDACT (Dynamic Integrated Developer ACTivity) comes in.
DIDACT is a methodology for training large machine learning models specifically for software development. What makes it unique is that it uses the entire process of software development as training data, rather than just the finished code. By exposing the model to the real-life situations that developers face and the actions they take, the model learns about the dynamics of software development and becomes more aligned with developers’ workflow.
To build and train the DIDACT model, we leveraged Google’s extensive software development data. We collected data on various developer activities such as editing, debugging, repair, and code review. With this dataset, we created three different DIDACT tools: Comment Resolution, Build Repair, and Tip Prediction. These tools were integrated into different stages of the development workflow and received positive feedback from thousands of internal developers.
What’s even more exciting is that DIDACT is just the beginning. It opens up the possibility of developing general-purpose developer-assistance agents. The trained model can be used in surprising ways by prompting it with different developer activities and combining multiple predictions to create longer activity trajectories. This paves the way for agents that can assist developers throughout the software development process.
DIDACT takes advantage of the vast amount of data Google has accumulated over the years. Every operation related to code is logged, providing a detailed record of the software engineering process. From code edits to comments and variable renames, every step is recorded and can be replayed. Google’s code lives in a single repository called a monorepo, and developers work on their code changes in a local workspace. When they are ready, they create a changelist in Google’s code-review system and address reviewer comments in an ongoing dialog. Throughout this process, developers also interact with various software engineering tools like compilers and testing frameworks.
The DIDACT model utilizes these interactions to assist developers in their tasks. It defines a set of tasks related to individual developer activities, such as repairing a broken build or addressing a code-review comment. These tasks are described using a formalism that takes a code file, annotations specific to the activity, and produces an action to address the task. This formalism, known as DevScript, allows us to capture a wide range of tasks in a concise and efficient manner.
By combining code snippets and annotations, the DIDACT model can predict development actions like edits or comments. It excels in tasks like code clean-up and history-augmented code completion. With the help of history, the model can make better predictions about what the developer should do next, resulting in more accurate and efficient assistance.
In conclusion, DIDACT is an innovative methodology for training AI models for software development. By using the entire process of software development as training data, it improves the alignment between AI models and developers’ workflow. The promising results obtained from the implementation of DIDACT tools demonstrate its potential to assist professional software developers and serve as a basis for developing agents that can assist throughout the software development process. With its ability to leverage extensive data and its multimodal nature, DIDACT opens up new possibilities in the field of AI-assisted software development.