Computer Power and Human Reason

30 Oct 2021 02:15 - 30 Jul 2023 03:10
Open in Logseq
    • book by Joseph Weizenbaum
    • This book was everywhere when I was at MIT in the late 1970s, but I don't know if I ever actually read it back then. I currently seem to be recapitulating some of its thinking, and wondering if I was on the wrong side of some conflict, due to follies of youth.
    • One thing I didn't get at first about Weizenbaum – he is not only an enemy of AI as some kind of horrible intrusion of the mechanical into realms where it does not belong – he is equally adamantly opposed to such seemingly innocent manifestations of technology like hacker culture and teaching children to program.
    • All manifestations of something bad. Not quite computers, or technology, more like the spirit underlying them, something like instrumental reason:
      • This book is only nominally about computers...the computer is used here merely as a vehicle for moving certain ideas that are much more important than computers...we have made the world too much into a computer, and that this remaking of the world in the image of the computer started long before there were any electronic that we have computers, it becomes somewhat easier to see this imaginative transformation we have worked on the world.
    • Various Weizenbaum/Minsky stories
      • From The Empathy Diaries
        • Programs, argued Weizenbaum, can be written to help machines make decisions. But only people have the capacity for moral choice. This is ultimately what makes us human. And human choice is the product of judgment, not calculation. It includes nonmathematical factors, such as emotions.
        • Then Weizenbaum made a leap that I hadn’t: It was therefore immoral to teach children to program. When he looked at the children Seymour [Papert] was teaching, Weizenbaum didn’t see the wonder of “powerful ideas.” He saw the spread of a dangerous way of thinking to the young and vulnerable. Programming served as a primer in instrumental reason, a form of rationality that focused on the most efficient means to achieve an end but did not reflect on the values of that end. The act of programming, said Weizenbaum, encouraged programmers to live in the closed world of the machine. Where Seymour saw computers as a privileged place to learn, Weizenbaum saw a place where you could forget other people and your emotional and moral responsibilities to them.