Season of singularity

It has been several decades since I first encountered scenarios of high-tech doom or transcendence. Science fiction and futurology have given us many: galactic societies of leisure and immortality, worlds dismantled and turned into giant computers, cryptic aliens or AI gods overseeing eschatological cycles.

Meanwhile, the real world has actually entered some version of an age of artificial intelligence. Right now, while I write this blog post in one window, in another I can see a dialogue I had with an AI – the new version of Microsoft’s Bing.

First I asked it to summarize a well-known essay about AI doom (“AGI Ruin”). The summary rapidly deviates from the actual contents of the essay – this quaint tendency among our new AIs is called “hallucination” – but the imaginary essay it does describe, is summarized crisply and coherently, and the lengthy description (including 26 bullet points) took less than a minute to generate.

Bing then suggested that I ask, “How can we solve these problems?” That was indeed what I had planned to ask it. The value of this plan was somewhat reduced by Bing’s deviation from the actual text, but I clicked anyway, and quickly obtained a Wikipedia-quality list of generic AI safety methods, along with links to more information.

After that we discussed Dadaist poetry, and it gave me an example of one poet’s work, “translated from German”, which again appears to be hallucinated; and I tested its ability to multiply two six-digit numbers (the answer was almost right).

I’m not aware that science fiction ever anticipated an age of AI in which the AI so frequently made up things, or got them wrong. But then, AIs were supposed to do everything via facts and logic, that was how they differed from human beings. Even within the field of AI, essentially no one anticipated that AI would arise by first creating “language models” that could generate imaginary texts of almost any kind, and then tuning them to manifest a conversational persona which tries to be helpful and accurate. One may anticipate that the tuning will continue to improve their power and accuracy, while reducing the unrequested excursions into fantasy.

But should one anticipate that these AIs, or something like them, will actually surpass human intelligence and control? I think yes. They aren’t yet smarter than the smartest humans, but they have already extended the speed advantage of computers, from just the domain of calculation, to more difficult tasks like essay writing and image generation. The quality of their output depends greatly on how they are prompted – we don’t actually know their full power. They are mysterious oracles that happened to emerge from the statistical modeling of human prose, which are now being subjected to fierce field-testing, study, and re-engineering.

They are already being used to provide a stream of consciousness for simple computational “agents”; programs that treat the AI’s words, not as comments, but as commands. In effect, they no longer just think, they also act. For now, their actions mostly consist of writing code and conducting web searches, but in time they will be given machine bodies to control too.

In itself, this is a revolutionary, science-fictional event: to have computer programs among us that can talk glibly about themselves and about almost any topic, and which will obligingly write poems, devise itineraries, discuss politics or philosophy or history… But how far will they go? Will they surpass us, conquer us, domesticate us, uplift us, ignore us?

I began by saying that I have long known what it means to envision “science fiction” happening in real life. I have also lived long enough to see it not happen, to experience instead this era’s version of mundane daily life. All of my experience, and all of humanity’s experience, says, don’t expect transhumanity or extinction, just expect a standard human life cycle, just another human generation among many.

And yet, there it is in the other browser window: the obliging computer oracle with its unknown potential, something that never existed before. Only a few months old, but already beginning to shake the world. How could this not be humanity’s destiny, finally arrived?