Yuval Noah

tendency to summon powers we cannot control stems not from individual psychology

never summon powers you cannot control.

The lesson to the apprentice—and to humanity—is clear: never summon powers you cannot control.

The main argument of this book is that humankind gains enormous power by building large networks of cooperation, but the way these networks are built predisposes us to use that power unwisely. Our problem, then, is a network problem.

unchecked AI advancement could culminate in a large-scale loss of life and the biosphere, and the marginalization or even extinction of humanity

In the next few decades, it will likely gain the ability even to create new life-forms, either by writing genetic code or by inventing an inorganic code animating inorganic entities.

never summon powers you cannot control.

Why are we so good at accumulating more information and power, but far less successful at acquiring wisdom?

The tendency to create powerful things with unintended consequences started not with the invention of the steam engine or AI but with the invention of religion. Prophets and theologians have summoned powerful spirits that were supposed to bring love and joy but occasionally ended up flooding the world with blood

By 2020, 95.6 percent of children worldwide lived beyond their fifteenth birthday,[11] and in Germany that figure was 99.5 percent.[12] This momentous achievement would not have been possible without collecting, analyzing, and sharing massive amounts of medical data about things like blood groups. In this case, then, the naive view of information proved to be correct.
However, the naive view of information sees only part of the picture, and the history of the modern age was not just about reducing child mortality. In recent generations humanity has experienced the greatest increase ever in both the amount and the speed of our information production.