In a recent paper, researchers explored how populations of deep reinforcement learning (deep RL) agents can learn microeconomic behaviors like production, consumption, and trading. The findings showed that artificial agents are capable of making economically rational decisions and responding to changes in supply and demand. The population of agents learns to adapt prices based on the resources available in their vicinity, with some agents even engaging in the transportation of goods to maximize profit. This research expands the field of multi-agent reinforcement learning and introduces new challenges for agents to tackle.
The goal of multi-agent reinforcement learning research is to develop agents that possess the full range and complexity of human social intelligence. However, the research conducted so far has overlooked important areas where human intelligence excels and where human beings spend a significant amount of time and effort. Economics is one such domain that has been lacking in the research. Therefore, the aim of this work is to create simulated environments based on trading and negotiation themes that can be used by researchers in multi-agent reinforcement learning.
Agent-based models are used in economics to simulate economic behaviors. These models often incorporate preconceived notions of how agents should act in economic scenarios. In this study, the researchers present a simulated world where agents learn economic behaviors from scratch, similar to what is taught in Microeconomics 101. The agents not only make decisions about production, consumption, and prices, but also need to navigate a physical environment, find resources, and trade with others. The advancements in deep RL techniques now allow agents to learn these behaviors independently, without the need for a programmer to input domain-specific knowledge.
The environment developed for this research is called “Fruit Market.” It is a multiplayer environment where agents produce and consume two types of fruit: apples and bananas. Each agent excels in producing one type of fruit but prefers the other. The goal is for the agents to learn to barter and exchange goods, resulting in mutual benefit.
The experiments conducted in this study demonstrate that current deep RL agents can learn to trade, and their behaviors align with microeconomic theory when faced with shifts in supply and demand. Moreover, the researchers provide scenarios that are challenging for analytical models but straightforward for deep RL agents. For instance, in scenarios where different types of fruit grow in distinct areas, agents learn to specialize in transporting fruit between regions to take advantage of price differences.
Agent-based computational economics also employs similar simulations for economic research. This work shows that deep RL techniques can adapt and act within these environments based on their own experiences, without any pre-programmed economic knowledge. This highlights the progress made in multi-agent RL and deep RL within the reinforcement learning community and showcases the potential of these techniques for advancing simulated economics research.
As part of the path towards artificial general intelligence (AGI), multi-agent reinforcement learning research needs to cover all critical domains of social intelligence. However, traditional economic phenomena like trade, bargaining, specialization, consumption, and production have been overlooked until now. This paper addresses this gap and provides a platform for further research. To facilitate future studies in this field, the Fruit Market environment will be included in the upcoming release of the Melting Pot suite of environments.