The servers are on the moon (2072)

I thought it was a bit weird of you to write a letter to yourself in 50 years’ time, given that all your stated beliefs on emerging technologies involved either the end of civilisation or the digitisation of human life. Still, I’m glad you did!

You were right about mind uploading being a big deal, though that was obvious in 2022, right? A lot has changed but, as far as I can tell, I’m still made of atoms rather than bits — inhabiting “meatspace” as you put it… (I haven’t heard that term in a long time)

As for the AI stuff, it’s good at art and science — and it basically powers everything — but it doesn’t decide anything, really. (I'm not quite sure how to distill the concept of agency within the context of Drexler's law of specialized software ascendancy into a neat, digestible summary; I have various mnemonic media on The Political Economy of Automatic Differentiators, but attaching them would kill the old-timey feel of this letter.) All that really matters is that, a few decades ago, a motley assortment of development schemes were made to avoid (vaguely-defined) combinations of ‘learned optimisation’ and ‘agency’. It essentially means that impressive AI systems never had enough autonomy and goal-directedness to destroy everything on their own; though we needed to more than just conduct a hodgepodge of evaluation tests.

2112^{11}

So in Cana of Galilee Jesus did his first miracle. There he showed his glory, and his followers believed in him. (John, 2:11)

We created AI systems that could conduct the lion's share of our research in silico, and research labs prioritized 'tool AI.' This gave us a clearer path to realizing Bostrom's vision of differential technological development. At least, that's how it seemed to me during the 13 years I spent working on these strategies.

I still think the researchers were doing the pro-social thing, but tools and agents aren't always so distinct (especially in the hands of fallible humans). By 2048, it was clear that AI systems were basically capable of turning water into wine: increasing computational capabilities bred better algorithmic improvements, of course, and those algorithmic improvements quickly brought about better comp…— you know the deal.

A generally intelligent artificial agent: there were a scary few years when it totally could’ve been made (even by a small lab). Everyone in the world knew the risks, but that wasn't exactly sufficient for mitigating them.

So, instead, all computational activity — from every broadband signal to every chip — got policed.

I thought there might be a combination of defensive technologies that would inhibit single agents from becoming dominant, but I couldn't tell you what that would be even now (21 years later). We'd had a shitload of AI-generated technological breakthroughs, including lots of scary biological stuff, as well as (more significantly) absurdly productive factories that started displacing almost all other economic activity. Unlike me, every smart person was focused on either (1) exploiting the power dynamics resulting from this economic environment, or (2) designing political institutions for uploaded minds.

After the recent and somewhat palatable use of oppressive surveillance, property confiscation, and censorship, the political consensus was to prevent the 'non-democractic' usage of all advanced AI tools. Forgive me if I sound ambivalent about all of this, since I am grateful that civilisation is still here, etcetera etcetera. The trouble with quashing all personal usage of advanced technology, in a world that is determined by said technology, is that it extinguishes any sense of individual agency. But I'm rather old fashioned now and the younger generation aren't as hung up on 'changing the world' as us narcissistic millenials. I moan about value lock-in and lost potential, but my kids either ignore me or tell me "you can create your utopia when you're uploaded".

The emulation population

As for mind uploading, it’s kind of just everyone’s afterlife. We all know we’ll end up creating lots of pure ems (i.e. people whose minds never ran on wetware), but Robin Hanson’s legacy was to scare a ton of people shitless about an intergalactic capitalist expansion of enslaved consciousnesses. Cue some spicy population-ethics debates among your friends on whether a Malthusian collapse is ‘necessarily’ a ‘moral catastrophe’, while the rest of the world sensibly decides to seize the means of emulation and socialise the crap out of that industry.

Anyway, the servers are on the moon. Like the literal moon. I find it kind of weird looking up and thinking about the 1.4 billion consciousnesses there, but it’s not like you can see any of the heatsink infrastructure yet. The ems are doing a fair bit of philosophising before they expand to other planetary systems, but they mostly play games as far as I can tell.

Em-life seems fine, something to look forward to, but it’s kind of abstract. I thought it’d be weird being trapped in a computer, but all the ems I know say it’s not like that, and you evolve new sensory experiences. I ask them what a normal day is like, and all of them seem bemused by the concept — I guess they don’t have ‘day’s? Or perhaps the notion of ‘normal’ is now alien to them, I don’t know. I’ll understand it more clearly when I’m there but we don’t have any approved way of reinhabiting a human body and so I’m savouring my time in this one while it lasts.


Down here we mainly focus on how we’re going to make Earth nicer in the short-term. A lot of our physical infrastructure has been replaced so that its upkeep and maintenance is automated, so now we, uh, design stuff? We’ll generate things like new sports equipment, stand-up comedy routines, and rapid transit carriages filled with art and nice armchairs. We discuss economics and politics and study the cosmos (as reported back by our probes). No sign of other life, yet!

The key result is that our cities are absolutely lovely. Everywhere is basically like the Netherlands that you know now, but with zero cars and far, far fewer sharp objects.

The fact that we have the prospect of eternal life has made us much more risk averse; we mourn every unexpected loss. Pre-emptive brain scans are taken, but — for quasi-spiritual reasons — everyone plans on being uploaded as they die. We don’t talk about it much.

All the best,
Finn

RSS feed | Site built with Franklin.jl