Profits are an imperfect measure of human welfare. Maximising profits can lead to increasing income and wealth which in turn can increase welfare. However, there are at least two reasons why profit maximisation and welfare maximisation differ. (1) externalities (2) endogenous preferences.

1. Externalities: The classic case is pollution: industry creates products which are desired by some. However, this might affect people negatively which are not part of the market interaction.

2. Endogenous preferences: consider two levels of preferences: what we want (first order) and what we want to want (second order). Algorithms might be affecting already first order preferences by changing content view and production. For example it seems plausible that a person doomscrolling or writing angry responses on Facebook might have slid into that behaviour partially due to the design of the platform. Plausibly, even second order preferences might be affected, although this is even harder to prove.

This project has two aims: First, validate hypotheses about negative effects of profit motivated corporations. Second, extrapolate how these effects might play out under transformative AI. We will develop or adapt a stylised formal model if the time allows.

The project might be structured around how firms become less constrained by (a) their workers, (b) governments and (c) consumers/voters.

(a) Currently workers wish to constrain the management of corporations. For example, Google’s “Don’t be evil” principle might have been a tool to retain top talent who wanted to work for a company that aligned with their values. The less workers are necessary, the less these human preferences affect companies.

(b) AI empowers corporations against governments. First, one factor that is hard to relocate is workers. If workers are automated, threatening governments to re-locate might be more effective. Second, corporations in industries without strong network effects are typically most profitable at an intermediate size, due to congestion effects between human workers. AI might enable larger firms which have more leverage over governments. Third, and this is more uncertain, automation could go faster in corporations than governments, following past technologies. This, again, would increase the power of corporations relative to governments - for example it could make lobbying easier and change regulation to their favour.

(c) Corporations might also affect consumer preferences more effectively with advertisements or by influencing their information environment (social media, newspapers, television channels, …). This in turn might limit how effectively those consumers will control the government.

The goal of this project is to check if this list is comprehensive, and sufficient to lead to a catastrophic scenario.