Corporations do not share our priorities. They are hive organisms constructed out of teeming workers who join or leave the collective: those who participate within it subordinate their goals to that of the collective...In short, we are living in the aftermath of an alien invasion.
I used to find it odd that these hypothetical AIs were supposed to be smart enough to solve problems that no human could, yet they were incapable of doing something most every adult has done: taking a step back and asking whether their current course of action is really a good idea. Then I realized that we are already surrounded by machines that demonstrate a complete lack of insight, we just call them corporations.
Capitalism and artificial intelligence are both powerful optimization systems, but their relationship is more than metaphorical. New research shows that they share a deeper mathematical relationship. Markets are a type of neural network that perform gradient descent by backpropagation. More specifically, supply chains and competitive markets learn through backpropagation.
Capitalism and AI have another property in common: they have the wrong objective function. From economic theory, it’s clear markets fail when it comes to issues such as providing public goods, inequality, long-term planning, managing tail risks, and accounting for externalities. Sometimes policies are able to correct these externalities through taxation, subsidies, or other legal restrictions. However, these policies frequently take decades to implement rather than months. If capitalism had better objectives, many of these problems would not arise in the first place.
The AI Objectives Institute therefore proposes the creation of a new institution focused on aligning capitalism as the first case of powerful AI. For artificial intelligence, markets and humanity, our objective is to create better objectives.