Dec 27, 2024

The pit in my stomach with AI progress

Each time I watch AI complete a task in minutes that once took me hours, I feel a looming sense that something big is coming. Today, AI deploys web apps from a single prompt and controls my computer. In 5-10 years? It might run our entire company.

The Road to a Fully AI-Run Company

A powerful assistant

Today, you’re most likely using AI to write emails, do market research and debug code. ChatGPT can browse the internet, reading through thousands of webpages. It also writes and runs lines of code to complete tasks for you. For example, to plot graphs it might use Matplotlib, a popular plotting library in Python.

As computer vision becomes more accurate, future AI tools won't just be confined to generating text, images and audio — they'll be able to do anything a human could do on a computer. For example, Claude’s Sonnet 3.5 has computer use capabilities and this demo showcases AI agents.

AI tools will break into the physical space as well with advances in robotics from companies like Figure and Tesla.

Companies are beginning to embrace the efficiency boost of enabling their employees to use AI tools for their tasks, adopting custom AI-powered tools that can generate financial reports, answer customer service queries, or perform checks on code before deployment.

Right now, it feels like a powerful software upgrade. Tasks that used to take hours are now completed within minutes with far higher quality. Companies and governments are beginning to invest in products that promise to leverage AI to help them achieve their goals. You might even be tasked with building workflows and adopting tools so you can more heavily rely on this powerful assistant.

AI-run departments

Take a typical customer service team. Where once hundreds of employees responded to thousands of queries, now chatbots and automated workflows handle the bulk of queries. Humans step in only for the trickiest cases — perhaps a tenth of the volume. As AI gets better at learning from user feedback through more examples and more powerful models, even fewer humans are needed until only a small group of supervisors remain to manage exceptions and spot-check responses.

This pattern wouldn’t just stop at customer service. HR, marketing, accounting — even R&D — would adopt specialised AI to complete their tasks, each overseen by one or two human “AI managers” who know how to spot mistakes and keep workflows in sync.

As these AI-powered teams grow more capable, managers themselves start leaning on AI to schedule tasks, make data-driven decisions, and even propose strategic plans. Work from AI that once required ten people to manage can now be done by two or three equipped with real-time analytics tools.

The work of each department, which used to be completed by thousands of employees, is now whittled down to a handful of humans supervising AI.

An automated C-Suite

Costs drop, speed increases, and product quality improves as the company fights to remain competitive with other companies that are rapidly adopting AI.

The C-suite, charged with consolidating each department’s data to steer strategy, finds that AI can deliver real-time insights, often more quickly and accurately than humans. The executives will increasingly rely on AI to synthesise complex reports, brainstorm ideas and evaluate strategic priorities. Tasks that once demanded days of work from a seasoned team can now be completed by a single person with robust AI tools.

Ultimately, the only humans left might be those who occasionally spot-check the AI’s outputs — but even that can be automated away. At this point, the company effectively runs itself around the clock, reshaping the very notion of what it means to be a “company”.

Power concentrated in the hands of a few

If a single person, or a small group owns or controls the AI that runs an entire corporation, it’s effectively like having an army of brilliant, tireless employees in every department.

In an ideal world, AI-driven corporations might produce goods and services cheaply, reduce human drudgery, and operate with near-perfect efficiency — potentially boosting everyone’s living standards.

However, this degree of concentrated power also means that mistakes or malicious actions can scale rapidly, especially with fewer people in the loop to keep the corporation accountable. If these “hyper-efficient” AI systems pursue objectives misaligned with public welfare or are used by bad actors, the consequences can be devastating.

Even relatively basic AI algorithms have amplified hate speech and contributed to real-world violence, as seen in Myanmar with Facebook’s engagement algorithms inciting violence, which led to the genocide of over 700,000 Rohingya people. If more advanced AI becomes self-improving and gains control over major decisions, it could lead to greater consequences and outpace our ability to impose safeguards.

Where does that leave us?

The promise of AI is too sweet to resist. Once a few companies adopt AI-driven strategies to slash costs and outmanoeuvre competitors, others will feel immense pressure to follow suit. Governments will do the same to compete with other nations for global economic or military advantage.

In such a race, safety considerations are often sidelined — much like the nuclear arms race, where being “first” overshadowed long-term caution. Scientists working on the early stages of the Manhattan Project sensed the destructive power they were unleashing, yet they pressed on, fearful of losing their place at the helm of scientific discovery or risk letting a rival nation gain the upper hand. Humanity ultimately built a technology capable of eradicating all life on Earth and very nearly ended us all.

AI could become an even more transformative force, potentially bringing incredible breakthroughs like disease eradication, cheap and abundant goods, and solutions to complex global challenges. But there’s also the chance we’ll concentrate too much power in too few hands or that misaligned AI systems will cause havoc faster than we can respond.

Yet, we will keep pressing forward because of competitive dynamics, market forces, and our naive optimism that we will have the time to fix these problems as they crop up.

We don’t know if catastrophe will strike with the suddenness of a bomb or through a slow descent into social upheaval. We also can’t say how much time we’ll have to react once the warning signs appear — or even if those signs will come within a century, a decade, the next year or at all.

But one thing is certain – we will continue to build AI that reshapes our world at breakneck speed. This looming sense of AI’s massive potential will only grow. Our collective choices — from policymakers to everyday users — will decide whether it becomes a monumental success or a tragic failure.

💡 If you’d like to learn about the transformative potential of AI and its impact on humanity, BlueDot offers a free Intro to Transformative AI course.