paperclip maximizer

30 Oct 2021 02:15 - 16 Dec 2022 12:22
Open in Logseq
    • An imaginary intelligence that exemplifies the consequences of the orthogonality thesis; in the sense that it is an intelligence with vast capabilities for rational action, but in the pursuit of an idiotic goal that is dangerous to human values. See the LessWrong page.
    • This seems entirely implausible to me. Part of this exercise is to investigate and defend that intuition and related doubts about the ironclad mathematical certainties that Rationalism produces so effortlessly.
    • My most popular blog post of all time (see AI risk ≡ capitalism) made the observation that the real human-hostile stupid-genius optimization engine is capitalism. Apparently what I thought of as a stunning insight is actually something of a cliche, but I'm still pretty convinced of its importance.
    • Paperclip Maximizer.full.jpg