I worry about a feedback loop:
https://headtopics.com/us/ai-learns-to-write-computer-code-in-stunning-advance-32646647
We don't have good quantum computers yet---and whatever comes with them...however strange and oddly bodied.
Since A.I. is still early in its development---now is the perfect time to use it as follows:
Have A.I. write code so dense there is no hacking it....everything plugged in a 1 and a 0 at a time. No back doors allowed, as it were.
You standardize that. Hard-wire it.
Operating systems for equipment get this pure code and is then air-gapped to make sure.
The A.I. that writes such code is air-gapped.
The result, we get from the start what Bowman reduced HAL to...pure simple systems...
The A.I. is allowed to rebuild our wonky computer systems---but can't do a Forbin on us when it evolves later on---and we keep the
Dybbuk in his box.
And as something of a student of the left-handed path---I use that word "Dybbuk" cautiously...a good name for any arising Boltzmann Brain.
The ancients believed if you could find the true name of a thing---you had power over the thing.
Now, we have power word kill:
https://phys.org/news/2022-12-ai-words-cancer-cells.html
Be it Kurzweil's coding or Karswell's calligraphy---it all comes down to the proper casting of the runes:
“Do not call up that which you cannot put down.”
― H P Lovecraft, The Case of Charles Dexter Ward
