To Imagine AI, Imagine No AI

The appreciate an AI future, consider a future without them. That is, imagine that we somehow manage to forever prevent any AIs who might take control of civilization. But also assume that our descendants continue to evolve competitively, with technical and economic growth for millions or billions of years. Now ask: what exactly did you think would happen there?In my view, it is pretty obvious that our “human” descendants would change radically. Eventually for sure, but maybe even rather quickly. After all, we now differ greatly not only from our ancestors of ten million years ago, but even from our ancestors of a thousand years ago. Also, rates of change have been greatly accelerating, and new tech will let us make many deep changes to our bodies, brains, and environments. Our future civ should thus be far larger, more complex, and more varied. Faster rates of change and longer lifespans (likely including immortality) will let more different generations overlap in time, inducing more stronger conflicts between them. And in these conflicts, we expect later generations to be abler, richer, smarter, and nearer to the center of control and info. And thus to win. Some future changes will count as robustly-valuable innovations that accumulate and last a long time, much like the past innovations that we now embody and expect to long continue. Like eyes, consciousness, reason, laughter, love, law, markets, etc. But other changes may be less robust, varying more across space and time. Along those sorts of change dimensions, our descendants attitudes, habits, and values will “drift”, and without obvious limits. And some fiercely hate the prospect of such drift. To limit this drift, some hope that we will develop better “mind control” tech, together with strong civ-wide centralized governance. These would allow authorities to better control citizens, and a whole generation of parents to better control the next generation of kids. Such centralized powers seem to me both dangerous and distasteful. This mind control would at least partially displace past processes by which generations have chosen how to change their attitudes, styles, and values. Okay, now imagine a future date when becomes possible for some to live beyond Earth. Some may then reason:As an Earth person, I am partial toward Earth people, relative to space people. If we let people leave Earth, then space people will become different in some ways, grow more than Earth people, and eventually come to dominate. We should prevent this horror by preventing anyone from leaving Earth, at least until we find ways to centrally and absolutely mind-control all space people.This kind of partiality is infertile; genetic or memetic evolution does not in general select for it. Yes, this strategy might promote pro-Earth-people partiality, but it hurts all other partialities that would be helped by a much larger future economy. Now let’s consider AI. Humans now occupy a tiny corner of a vast mind space, and compared to our more human descendants, AIs would more expand out into this mind space, analogous to [...]