Ramez Naam, author of Nexus and Crux (two books I enjoyed and recommend), has recently put together a few guest posts for Charlie Stross (another author I love). The posts are The Singularity Is Further Than It Appears and Why AIs Won’t Ascend in the Blink of an Eye.
They’re both excellent posts, and I’d recommend reading them in full before continuing here.
I’d like to offer a slight rebuttal and explain why I think the singularity is still closer than it appears.
William Hertling has given a reply to Ramez Naam’s articles that I linked to this morning. I don’t know whether Naam will reply, but I have a short reply, and I think it applies to both of their thinking to some extent.
A lot of singulatarians assume that Moore’s Law will proceed indefinitely. As I’ve written before, I think this is a questionable assumption. Moore’s Law is not an indefinite proposition, but an S curve.
I mentioned the S curve in another post earlier this week, but the main idea is that it represents a situation where progress is rapid, for a while, but eventually it levels out. Whether we’re talking about growth in market share, in scientific knowledge, or any other aspect of rapid growth, that growth almost always reaches a point of depletion or saturation.
When you’re in the steep upslope of the S curve, it’s often very difficult to see when it will level off. Indeed, if it lasts for a long time, it’s very tempting to assume that it will never level off.
In the case of Moore’s Law, that leveling off is likely to happen when processor technology butts up against the laws of physics. When will that be? I don’t know, but silicon technology is expected to do it around 2020. Will quantum computing come to the rescue? Perhaps, but that’s a matter of faith in singularity thinking rather than one of any kind of certainty. And even quantum computing will eventually reach its limit.
The question then is, within the laws of physics, how much can the human brain be improved upon? Almost certainly it can be improved dramatically, but it is a very large assumption that it can be improved upon to the many orders of magnitude implied by singularity thinking.
Remove that assumption, the assumption of limitless improvement in processing power and capacity, and the idea of a runaway singularity becomes much more of a long shot.
h/t John Blackman