Tyler Cowen recently challenged us to try to beat an AI called “o1”:
1. o1 is a very good economist. Try to punch it if you can.
I couldn’t find o1 so I used ChatGPT 4o mini. I hope someone will try the following question with an o1 and report the results in the comment section:
What continues to strike me about AIs is that they are a super smart student, able to remember large amounts of information, but unable to actually see the essence of a problem. In the example above, I gave one hint after another, practically begging the AI not to reason from a price change. It was so oblivious to what it was doing that it eventually told me that Qd fell while Qs rose and yet the market remained in equilibrium.
Maybe I used the wrong AI, or maybe this question is unusually difficult. (For a student, “difficult” means a question that requires you to actually understand something, rather than simply repeating what you read in your notes.)
So let’s take a question Tyler asked the O1 economist:
Please write an essay on how today’s macroeconomists find inflation dynamics so difficult to predict, and why they have therefore rejected various forms of monetarism, even as approximations of what goes on behind price level behavior.
The answer is very long, so I’ll focus on a few key parts of the answer:
Several factors have contributed to the increasing complexity and unpredictability of inflation. First, the financial innovations and regulatory changes of the late twentieth and early twenty-first centuries have dramatically changed the relationship between money and economic activity. Monetary aggregates – such as M1 or M2 – that once served as reliable indicators of the policy stance and future inflation are now behaving erratically due to shifts in the velocity of money, the proliferation of shadow banking and the globalization of financial flows. Simply put, where money is located and how quickly it moves through the economy has become too fluid and complex for older, simple monetarist rules to capture.
Second, the nature of central bank policymaking and fiscal policy has changed. Central banks are now intervening in a variety of unconventional ways, from massive purchases of financial assets to guiding policy expectations. These instruments are not well explained by the classical monetarist perspective, which focused on controlling some measure of the money supply. The recent experience following the global financial crisis illustrates this vividly: the Federal Reserve and other central banks implemented unprecedented quantitative easing programs, dramatically expanding their balance sheets. According to traditional monetarist logic, this rapid increase in the monetary base should have led to substantial inflation. Yet inflation in many advanced economies remained stubbornly below target levels for years, confusing those who relied on old monetary aggregates as a guide.
Third, today’s pricing involves a bewildering interplay of global supply chains, technological advances, labor market transformations, and shifts in consumer behavior. Globalization means that prices for goods and services are affected not only by domestic monetary conditions, but also by remote supply shocks, currency fluctuations and international competition. Technological changes increase productivity and can depress prices in certain sectors, while other parts of the economy are less affected. Labor markets have also evolved, with changes in union power, demographic shifts and changing patterns of labor force participation influencing wage setting and cost pressures. These micro-level frictions and structural changes make the older macro-level equations that link money supply growth to inflation too crude and inaccurate.
Sorry, but this is mostly a mix of empty clichés and irrelevant comments. Again, it’s something I expect a student to write down if they don’t know the answer. I’ll explain why in a moment, but first let me clarify an important point. I’m pretty sure o1 is more intelligent than me, at least in the way most people define ‘intelligence’. In other words: o1 knows a lot more than I do. It’s much more intelligent than me, roughly in the sense that the New York Times is much more intelligent than me. And that is really very impressive.
But o1 is not a good economist. Sorry, that’s just not it.
Let’s discuss the last paragraph first, which is complete nonsense. Technology has been advancing for more than 100 years, and the rate of productivity growth is not accelerating. So that is not a factor that makes it more difficult to predict inflation. The strength of the Union goes up and down, and that fact does not make it any more difficult to predict inflation. Changes in consumer behavior? Now the student is really getting desperate. It’s a “I need to fill the page with something” kind of thing. Global supply chains? Again, it doesn’t make inflation harder to predict. Not at all. Global commodity prices had a greater impact on US inflation a hundred years ago than they do today. None of these four factors have made it more difficult to predict inflation. In fact, I’m not even sure that predicting inflation is getting harder and harder. It seems to me that inflation was much harder to predict when I was young than it is now.
The first sentence in the middle paragraph is slightly less bad. Central banking has really changed in one respect. And the relationship between the base and the aggregates has really loosened. Unfortunately, the AI seems to have no idea why the relationship has loosened up. It mentions a lot of irrelevant things, like QE, and ignores the key point that interest payments on reserves and the zero lower bound problem have made the money multiplier much more unstable. The AI also says that fiscal policymaking has changed. That was probably just a wild guess. I can’t think of any changes in fiscal policy that would make it harder to predict inflation. Given that the AI is silent on this issue, I suspect it also has no idea what has changed in fiscal policy, or why those changes would make inflation harder to predict.
In the first paragraph the AI says:
Monetary aggregates – such as M1 or M2 – that once served as reliable indicators of the policy stance and future inflation are now behaving erratically due to shifts in the velocity of money, the proliferation of shadow banking and the globalization of financial flows.
False. The aggregates have never been reliable indicators of inflation. The velocity of money has always been unstable. But don’t the textbooks say that the velocity of money used to be stable? Yes, many do. That’s probably where the AI got its misinformation from. It may be more unstable now, but it has never been stable enough for the aggregates to be good predictors.
Also note the strange list, including speed, shadow banking and financial flows. That’s a bit stupid. An AI that understood what it was talking about would have put a stop to the speed and then had a separate sentence explaining some of the reasons why the speed changed. Right now it’s a confusing jumble of concepts.
If you are a math-oriented person who is confused by economics, this equation may help you understand my point:
V = PJ/M
The AI basically says that the PY/M ratio changes due to things like changing V, shadow banking and financial flows. The ratio is V!! The other variables help explain why V might change.
Even though the NYT is much smarter than I am, when I come across a NYT article in an area where I have some expertise, the paper suddenly seems much less smart. I won’t say “stupid,” because even in the worst-case scenario, the NYT employs talented reporters. And I can’t expect them to have devoted their entire lives to studying monetary economics.
In my opinion, Tyler’s o1 answer is modestly below the level of the NYT. The Times would have at least mentioned interest on bank reserves or the zero lower bound. Nevertheless, AIs are really impressive when it comes to the amount of information they have retained. Even within monetary economics, an AI will often have more information at its fingertips than I do. An AI could write a better essay on all the Fed’s new policy tools than I could write. Where it falls down is in a certain type concept. I’m not sure I can even put this concept into words. Perhaps understanding doesn’t really exist, and we fool ourselves into thinking we have a deep understanding of anything. Perhaps a superintelligence would mock my writing the same way I mock the AIs that reason from a price change.
All I know is that AIs still have a long way to go before they impress me in terms of understanding monetary economics. But outside my field, I am already extremely impressed.
Here’s how AI Overview defines Gell-Mann Amnesia:
Gell-Mann Amnesia (GMA) is a defense mechanism that occurs when a person reads a news article about a topic he or she knows well and finds it full of errors, but then reads the rest of the paper as if it were more accurate. The term was coined by Michael Crichton after discussing it with Murray Gell-Mann