This note is the twenty-eighth letter in the 104-days-of-summer-vacation series. You can also follow the full twitter thread here, and leave any thoughts and comments that might come up!

Dearest Reader,

People talk about how artificial intelligence is a tool. It seems to me that these people who claim so, are either ignorant of intelligence’s potential or are specifically referring to artificial intelligence in its current form today. I think artificial intelligence, in its general sense, cannot be a tool.

Let’s start in the nature of intelligence. Tools are designed to fulfill a purpose, a hammer hits a nail and a computer computes computations. Tools are mechanical in the sense that they follow defined steps to a predestined outcome. If the desired outcome changes, the tools do not adapt to those changes.

Weak AI in its current form, fits this description. It tends to be good at one thing but terrible at everything else, just like a conventional tool. But general intelligence, is not a series of steps, but a way to generate a series of steps. General intelligence is the ability to plan and execute steps in pursuit of any arbitrary goal.

If intelligence was a tool, it’s a hammer to which everything can be a nail. Intelligence is a rising tide that lifts all boats, people who are more intelligent tend to also be more healthy, more wealthy and just better at all sorts of things. While tools are for fulfilling a purpose, intelligence seems to be for fulfilling any purpose.

This is why I think none of us can afford to ignore artificial intelligence, whether we care about the technology or not. Already our newest tool AIs can make their own decisions in multi-step problems, these are the first tools in history to have agency.

This means massive upside for us, as I’ve said in super-obsidian, and massive risk as well, like I said in fear-irrelevance.

At night, I dream of what life might look like when computers go from being tools, to being agents. It looks likely that we might need answers to those questions in our lifetimes.

~ Shan