OpenAI is currently developing a research program to evaluate the economic effects of code generation models. They are also inviting external researchers to collaborate on this project. With the rapid growth of large language models (LLMs) trained on code, it has become crucial to study their impact on individuals, companies, and society as a whole. Codex, an LLM developed by OpenAI, has shown promising results by generating correct code 28.8% of the time on evaluation problems.
This milestone has significant implications for the future of coding and the industries that rely on it. In order to shed light on these implications, OpenAI presents a research agenda aimed at assessing the economic factors related to Codex. By examining the broader applicability of code generation models to software development and the potential impact of future LLMs, they aim to generate evidence and establish methodologies for researching the economic effects of AI models.
OpenAI suggests that academic and policy research should focus on studying code generation models and other LLMs. This research will provide crucial evidence to inform decisions in three key areas: deployment policy, AI system design, and public policy. To guide the research, OpenAI identifies six priority outcome areas: productivity, employment, skill development, inter-firm competition, consumer prices, and economic inequality. For each area, they briefly review existing literature on the impacts of AI and propose important research questions that can be explored using Codex.
To encourage further research, OpenAI is calling for expressions of interest from external researchers. They aim to collaborate with these researchers and their customers to better understand and measure the economic impacts of code generation models and other LLMs.
For more information, you can visit the OpenAI blog post on economic impacts [insert hyperlink].