I was reading an article in SciAm this morning about the possibility of a robot uprising. Don’t laugh yet, this is a very real, if still quite remote possibility.
The main idea that was described was that AI could rise one day to self-awareness and to an ability to improve itself through self-replication beyond human abilities to control it.
Sure, that’s one possibility, and some people are actually arguing that if that’s the case, maybe it’s just the march of evolution and humankind is just destined to one day become obsolete and be replaced by something fitter, whether from good old evolution or by artificially creating its own replacement.
I would tend to agree but I do have an objection. There is a distinction in this kind of speculation that is not often pointed out: self-replication and evolution are not the same thing.