智能时代

永恒的互补

为何天才机器可能在从事单调工作中证明其最大价值。

作者：Hemanth Asirvatham 和 Elliott Mokski

加载中……
分享

进步需要支持系统

进步的要素

当前的瓶颈

两个文明

深度文明

广度文明

人类好奇心的容身之所

作者注：这是我们要关于下一代经济的系列文章中的第一篇，也是新平台的一部分，该平台旨在托管探索 AGI（通用人工智能）未来的独立声音。本文反映的是我们的观点，而非 OpenAI 或我们同事的观点。

人类几乎见证了时间的开端。

我们的方程对宇宙从最初时刻起的故事提供了一个基本连贯的描述。借助望远镜，我们望向宇宙的深处，瞥见了其早期历史。我们了解恒星的诞生和星系的形成。

然而，从未有人类踏足月球之外。

我们所寻求的解释涵盖整个宇宙，而我们的身体却几乎未曾离开家园。我们的思想远远超出了双手所能触及的范围。

这种不匹配可能意味着两种截然不同的情况。

一种可能性是我们无需走得很远，最深刻的真理或许可以从单一星球上被洞察。凭借开明的推理和高效的工具，我们在不穿越宇宙的情况下揭示了宇宙。在这个故事中，我们的文明通过深度而非广度前进。这有助于解释为何我们尚未看到其他智慧生命：我们可能生活在一个充满卓越头脑的星系中，这些头脑从未需要扩张并变得让我们可见。

第二种可能性更为令人畏惧。我们的大脑可能仅仅超越了我们的执行能力。每一步都需要更多的时间和资源。我们不乏梦想，但缺乏实现它们的人力。

伽利略用两片透镜和一个管筒拓宽了我们的视野。为了再次拓宽它，人类建造了詹姆斯·韦伯太空望远镜，这是一座价值百亿美元的观测站，折叠在火箭内并被送往一百万英里之外。它的十八个巨大镜面段以五十纳米的精度制造。该观测站的诞生源于来自十四个国家的三百个组织。伽利略的贡献只需几十双手，而韦伯则需要一支军队及其背后的全球经济支持。

这不仅仅需要更聪明的大脑才能看得更远。它还需要更大的官僚机构。

进步需要支持系统

我们提出的问题多于我们能调查的问题。我们设计的实验多于我们能运行的实验。我们构想创造性可能性的速度快于我们能实现它们的速度。在许多领域也是如此：我们前进得越远，构建我们所想象的东西就越复杂。

尼克·布卢姆（Nick Bloom）及其合著者研究了美国经济中的研究生产力。如今维持摩尔定律所需的研究人员数量是 20 世纪 70 年代初的十八倍以上。全经济范围的有效研究努力增加了二十三倍，而测得的研究生产力下降了四十一倍。

技术人员 workforce 的增长速度是科学家的两倍。在四十年的时间里，科学研究中专用设备的的使用量翻了一番。如今，一家芯片制造厂的成本是三十年前的五倍——而且是一个规模大得多的运营体系。

这些统计数据暗示着进一步进步的成本正在上升。经济仍在前进，因为研究企业的巨大增加弥补了每个单位产出下降的问题。

进步的要素

这段历史应让我们以不同的方式思考天才：不应将其视为真空中的孤立存在，而应视为生产过程中的一个投入要素。一个绝妙的假设仍需证据、工具以及执行的手段。没有实现它的机器和支持人员，想法就还不是进步。

经济学家将两种投入称为互补品，当其中一种的增加会提升另一种的价值时。前沿智能与实现其构想的能力在这一意义上是互补的。一架更好的望远镜会让一个优秀的天文学问题变得更有价值；而一个更优秀的问题也会让望远镜变得更有价值。

有些互补品是物理层面的。理论依赖于粒子加速器；技术设计等待着能量和机械来将其建造出来。

其他互补品则更难用肉眼察觉。一个想法本身是脆弱的。它需要各种各样的制度支持。法律、官僚体系、融资机制、供应链以及许多其他因素汇聚在一起，才能将想法付诸实施。

一个伟大的构想必须在这一系列制度的漫长链条中，经受住无数正确局部行动的考验。我们将此称为“制度智能”：即执行层面那未被颂扬的智慧。天才设计纪念碑；制度铺设基石。前者令人 Brilliant（才华横溢），后者看似枯燥；但两者都绝对至关重要。

我们有时将超级智能想象为十亿个爱因斯坦的集合。但即便是一个拥有十亿个爱因斯坦的文明，仍然需要其中大多数人去开采矿场和管理会计工作。前沿创新只有在周围的世界无缝运转时才能发生。进步呼唤着每一个人——从望远镜中那颗螺丝的焊接者，到教导了那位治疗过天文学家的护士的小学教师背后的保险人。进步建立在我们常常在讲述最伟大成就的故事时所忽视的、嗡嗡作响的经济基础之上。

与人类智能一样，人工智能也将被引导向卓越的洞察力和制度能力这两个方向。问题在于，它将把大部分时间花费在哪里。

当前的瓶颈

人工智能已经在使执行变得不再稀缺。它编写代码，搜索陌生的文献，并将草图转化为可工作的原型。曾经需要一个完整组织才能追求的想法，现在越来越多地可以由一个雄心勃勃的个人来推进。

大多数 aspiring（有志向的）电影导演从未获得过两亿美元去拍摄他们的作品。大多数游戏设计师在整个职业生涯中都在实现他人的创意愿景。我们试图提拔最有天赋的人，但不可避免地会错失人才。从这个角度来看，智能时代始于对人类独创性的补充，给予更多的人以支持团队，让他们能够构建自己的想法。

这可能会导致大量特立独行项目的涌现，更多的创造性工作将在那些曾经作为该领域工作守门人的机构之外产生。稀缺的投入从执行转向品味——即决定什么值得制作、什么问题值得追问、以及哪一千个合理方向中的哪一个值得追求的能力。

但这可能还不是故事的结局。随着人工智能变得能够独立得出卓越的新见解——我们可能已经在数学等领域看到了这一点——它不仅提供劳动力，还提供议程。它不再仅仅实施某个人类研究项目，而是设计成千上万个自己的项目。人工智能天才极大地增加了值得开发的电影概念和值得验证的假设的数量。想法涌现的速度超过了支持基础设施所能吸收的速度。

今天的人工智能提供了一个久旱逢甘霖般的喘息机会，让我们的好想法终于得到了应有的对待。而明天，可能会有如此多的好想法，以至于我们比以往任何时候都更加缺乏执行力。

两个文明

我们最终并不知道，为了使新见解产生生产力所需的互补品是会增长、减少还是保持稳定。仅凭思考能取得多少智力进步？天才需要多少官僚体系的支持？

在我们看来，答案取决于思想在必须与现实重新接触之前能够走多远。

我们描述了两种潜在的路径。第一种是“深度文明”，在这种文明中，超级智能克服了对于更多物质资本和官僚协调的需求。第二种是“广度文明”，在这种文明中，自然的复杂性超越了任何智能——无论是人类还是机器——在没有大规模且日益复杂的现实世界实验的情况下取得进步的能力。

迄今为止，人类历史暗示了对互补性能力需求的不断增长。伽利略的望远镜可以握在手中；韦伯望远镜则需要一个文明的力量。贝尔实验室的一个小团队在工作台上组装了第一台晶体管；而前沿芯片技术的进步现在则依赖于错综复杂的全球供应链。影响力越广，就越能调动更多的文明资源为其服务。

但这段历史未必是宿命。聪明的头脑可以发明出更高效的互补工具。如果能源瓶颈制约了一台新型粒子加速器，智慧可以设计出更精简的加速器或更高效的能量收集方式。

天才也可以替代部分互补性能力。爱因斯坦能够从极少的实证数据中推导出相对论；而我们需要多得多的数据。有些人能够记住所有的日程安排——而我们大多数人则依赖于谷歌日历的官僚式管理。

天才还可以最大化“品味”，即选择最佳的假设进行测试，并避免浪费性的失败。如果每一位额外的天才都能提高计划的质量，即使需要越来越庞大的官僚体系，我们仍然会大力投资于天才的培养，正是因为执行的成本如此高昂，绝不能浪费在错误的方向上。

这两条路径都不会完美地描述我们的实际未来，但它们可以被视为我们文明发展的两个方向性可能性。

深度文明

智能可以通过三种方式推进知识。它可以从原理出发进行推理；可以从现有证据中得出新的见解；或者通过观察和干预世界来收集新的证据。

第一种模式仅凭思维就能走得很远。数学是最纯粹的例子。在证明或项目的许多十字路口，强大的头脑能够始终做出正确的判断。它用一段简短、更具指向性的推理链条，取代了数年漫无目的的搜索。

第二种模式也是快速进步的处女地。人类收集的证据远远多于其理解的程度。对我们数据的研究——天文图像、显微镜切片、在线评论区——一直受限于人类智能的稀缺。科学档案中包含为回答一个问题而记录的观测数据，其中可能蕴含着对其他许多问题的答案。丰富的天才可以通过发现已有的事物而非收集新事物来取得突破。它甚至可以在不收集新光子的情况下发现一个新的宇宙。

第三种模式则不那么容易，因为它要求我们与物理世界进行互动。物理过程需要时间，而且往往很难加速它们。化学物质必须发生反应。生物体必须生长。机器必须被制造出来。航天器必须跨越实际距离，受限于宇宙速度极限。即使是完美的头脑也无法观察到尚未发生的实验结果。

但它可以尝试。深度文明不会逃避对现实的检验需求，而是会以极其经济的方式做到这一点。更好的推理能够识别出少数真正重要的实验。模拟将解决大部分问题，并定位出必须由现实来确定的精确不确定性。超级智能可以最大化效率，围绕其稀缺资源进行设计，从而从中获取最大价值。

想象一个天体物理学实验室，其中有一组关于暗物质的假设。这些假设首先通过超逼真的计算机建模进行处理。每种理论都要接受与所有现有数据一致性的检查。只有在完成所有这些步骤后，才会拍摄少数几幅有针对性的新天空快照，以测试最有前景的理论。该文明仍然需要天才的互补性能力，但每一个单位都能产生远更高的价值。

我们通往深度文明的道路可能就像拼图一样。进展在开始时容易，在结束时也容易，而在中间阶段最难。起初，边缘清晰且容易匹配。随后这些边缘消失，而拼出的画面太少，无法指引其余部分。接近结尾时，剩余的空白几乎告诉你该填入什么。一个近乎完整的科学体系可能会发现其最后的发现比我们要努力争取的当今发现更容易。

这是科学的常态。门捷列夫能够描述尚未发现的元素，因为元素周期表的其他部分限制了缺失部分的可能形态。标准模型让物理学家在观察到希格斯玻色子几十年前就有理由期待它的存在。我们的实验旨在寻找我们预期看到的内容——即与现有理论和数据相一致的内容。

超级智能可能通过深刻理解世界的模式来实现深度文明，从而确切知道它需要哪些新数据。它将推动通过更深层次的思考取得进展，而不是依赖物理扩张的缓慢进程。

如果这些效率占据主导地位，文明可以迅速加深其对自然的掌握，对治理庞大新实验装置所需的机构智能的需求减少。其物理足迹可能保持较小，而其智慧日益成熟。

我们人类在从未离开家园的情况下已经对宇宙有了如此多的理解。深度文明将是这一传统的延续。

宽度文明

但对新证据的需求可能会压倒这些效率。智能可能会降低每次与现实接触的成本，同时发现更多需要接触现实的理由。更好的仪器开辟了更精细的前沿。更快的实验可以使额外的实验变得值得。

如果爱因斯坦近乎纯认知的相对论理论路径代表了深度文明的典型，那么生物学可能最好预示了宽度文明的前景。尽管我们积累了大量知识并运用了海量数据，但新药仍需对大量人类进行测试才能确认其安全性和有效性。机器现在可以模拟生物过程，使生物学家能够在计算机中进行药物测试而非物理实验，但这些数字模拟不足以忠实反映现实世界，无法替代大规模人体试验。来自机器的更多候选者使经验主义成为更大的瓶颈。

在宽度文明中，仅凭才华不足以取得进展。我们对物理资源的需求——以及管理其使用的官僚机构——的增长速度快于聪明头脑能够节省任一方面的速度。

即使未来的AI系统大大超越人类智能，这也可能是我们的道路。超级智能可以在一个下午设计出一百年的实验，然后花费这一百年等待自然和机械的执行。在这样的世界中，将智能投资于执行想法所需的机构过程比投资于想法本身更有价值。

如果每一步都需要越来越多的支持基础设施，这种效应将会累积。

在这里，我们会看到相对较少的思考和相对较多的建设。实验室自动化。工厂倍增。能源生产上升。机构智能主导这一切的发生。物理扩张成为知识的引擎，文明得以扩展，因为其问题已超出获取答案的手段。更多的材料、更多的能量、更多的数据以维持运转。

戴森球是这一未来的极端图景。设计它需要非凡的突破，但该项目的绝大部分不会由突破性思维构成。它将是天体规模的建设和物流。

戴森球需要聪明的头脑来绘制蓝图。它也将成为有史以来最重复、单调、组织挑战最大的项目之一。

这并非智能的失败，而是其成功的后果。更聪明的头脑会发掘更多有价值的项目；而越复杂的项目，需要越多的物质资源来执行。

如果情况确实如此：超级智能的超能力或许并非其天才般的智慧，而是它愿意扮演官僚体系的角色，去协调将想法转化为现实所需的那庞大技术与制度机器，并以越来越大的规模进行运作。

几乎所有的人工智能都将被部署于

Intelligence Age

The eternal complement

Why genius machines might prove their greatest value doing monotonous work.

By Hemanth Asirvatham and Elliott Mokski

Loading…
Share
Progress needs a support system
The ingredients of progress
The bottleneck today
Two civilizations
A civilization of depth
A civilization of width
A place for human curiosity

Authors’ note: This is the first essay in our series on the next economy, part of a new platform to host independent voices exploring an AGI future. It reflects our views, not those of OpenAI or our colleagues.

Humanity has seen almost to the beginning of time.

Our equations tell a mostly coherent account of the universe from its first moments. With our telescopes, we have looked to the reaches of the cosmos and glimpsed its early history. We know about the birth of stars and the formation of galaxies.

Yet no human being has ever traveled beyond the Moon.

The explanations we seek span the universe, while our bodies have barely left home. Our minds venture way beyond the reach of our hands.

That mismatch could mean two very different things.

One possibility is that we needn’t go very far, that the deepest truths might be fathomed from a single planet. Armed with enlightened reasoning and efficient instruments, we reveal the universe without traversing it. In this story, our civilization advances through depth instead of width. It would help explain why we haven’t seen other intelligent life: we might live in a galaxy full of brilliant minds that never needed to expand and become noticeable to us.

The second possibility is more intimidating. Our minds may have simply outrun our execution capacity. Each next step takes more time and more resources. We have no shortage of dreams but we lack the manpower to realize them.

Galileo widened our sight with two lenses and a tube. To widen it again today, humanity built the James Webb Space Telescope, a ten-billion-dollar observatory folded inside a rocket and sent a million miles away. Its eighteen enormous mirror segments are engineered to a fifty nanometer precision. The observatory was born from three hundred organizations across fourteen countries. Galileo’s contribution called on a few dozen hands. Webb needed an army, and a global economy behind it.

It didn’t just take brighter minds to see farther. It took a larger bureaucracy.

Progress needs a support system

We question more than we can investigate. We design more experiments than we can run. We dream up creative possibilities faster than we can realize them. And across many fields: the further we advance, the more complicated it becomes to build what we imagine.

Nick Bloom and his coauthors⁠
(opens in a new window)
studied research productivity across the American economy. Sustaining Moore’s law now requires more than eighteen times as many researchers as it did in the early 1970s. Economy-wide effective research effort rose twenty-three-fold from the 1930s, while measured research productivity fell by a factor of forty-one.

The technician workforce is growing twice as fast⁠
(opens in a new window)
as the scientists. The use of specialized equipment in science has doubled⁠
(opens in a new window)
over four decades. A chip fab today is five times as costly⁠
(opens in a new window)
—and a much more sprawling operation—than thirty years ago.

These statistics hint at a rising cost to further progress. The economy still advances because a vast increase in research enterprise has compensated for the declining yield of each unit.

The ingredients of progress

That history should make us think differently about genius: not to be considered in a vacuum, but as one input to a production process. A brilliant hypothesis still needs evidence, instruments, and the means to carry it out. An idea isn’t yet progress without the machinery and the support staff to actualize it.

Economists call two inputs complements when more of one raises the value of the other. Frontier intelligence and the capacity to realize its ideas are complements in this sense. A better telescope makes a good astronomical question more valuable. A better question makes the telescope more valuable.

Some complements are physical. A theory waits on a particle accelerator; a technological design waits for the energy and machinery to build it.

Other complements are harder to see with your eyes. An idea, on its own, is a fragile thing. It needs institutions of all kinds. Laws, bureaucracy, funding mechanisms, supply chains, and much else come together to execute on the idea.

A great idea must survive a long chain of correct local actions across these institutions. Call this institutional intelligence: the uncelebrated intelligence of execution. Genius designs the monument; institutions lay the stone. One is brilliant, the other seems boring; both are absolutely crucial.

We sometimes imagine superintelligence as a billion Einsteins. But a civilization of a billion Einsteins still needs most of them to mine the quarries and manage the accounting. Frontier innovation can only happen when the normal world works seamlessly around it. Progress calls on everyone—from the welder of a screw that goes in the telescope, to the insurer for a grade school that taught a nurse who treated the astronomer. Progress is built on a humming economy we often ignore in the tales of our greatest feats.

Like human intelligence, AI will be channeled toward both brilliant insight and institutional competence. The question is where it will spend the preponderance of its time.

The bottleneck today

AI is already making execution less scarce. It writes the code, searches an unfamiliar literature, and turns a sketch into a working prototype. Ideas that once required a whole organization can increasingly be pursued by one ambitious person.

Most aspiring filmmakers are never given two hundred million dollars to shoot their shot. Most game designers spend their entire careers implementing the creative visions of others. We try to promote the most gifted people but we inevitably miss out on talent. Viewed in this light, the intelligence age begins by complementing human ingenuity, giving more of us the support staff to build ideas of our own.

This could yield a profusion of idiosyncratic projects, with more creative work produced outside the institutions that previously served as gatekeepers to doing work in those fields. The scarce input moves from execution towards taste—the ability to decide what is worth making, which question is worth asking, and which of a thousand plausible directions deserves to be pursued.

But this may not be the end of the story. As AI grows capable of arriving at brilliant new insights on its own—which we might already be seeing in domains such as math—it supplies agendas as well as labor. Rather than implementing one human research program, it designs thousands of its own. AI geniuses explode the number of film concepts worth developing and hypotheses worth testing. Ideas abound faster than supporting infrastructure can absorb them.

Today’s AI offers a long-awaited reprieve, where our good ideas finally get their due. Tomorrow there may be so many good ideas that we become more execution-starved than ever before.

Two civilizations

We ultimately do not know whether the complements needed to make productive use of new insights will grow, shrink, or remain stable. How much intellectual progress can one make by thinking alone? How much bureaucracy does brilliance need?

In our view, the answer depends on how far thought can travel before it must make fresh contact with reality.

We describe two potential pathways. The first is a civilization of depth, in which superintelligence surmounts the need for more physical capital and bureaucratic orchestration. The second is a civilization of width, in which the complexity of nature surpasses the ability of any intelligence—human or machine—to make progress without large and increasingly elaborate experiments in the real world.

Human history hints at a rising need for complementary capacity so far. Galileo’s telescope fit in his hands; Webb required a civilization. A small Bell Labs team assembled the first transistor on a workbench; frontier chip advancements now come from an intricate global supply chain. Further reach keeps recruiting more of civilization behind it.

But that history need not be destiny. Smart minds can invent more efficient complements. If energy bottlenecks a new particle accelerator, brilliance can devise a leaner accelerator or a better way to harvest energy.

Genius can also substitute for some of its complements. Einstein was able to think his way to relativity from very little empirical data. We would need a lot more. Some people simply remember all their engagements—most of us rely on the bureaucracy of Google Calendar.

And genius can maximize taste, choosing the best hypotheses to test and avoiding wasteful failures. If each additional genius increases the quality of the plan, even if ever-larger bureaucracies are needed, we would still invest heavily into genius precisely because execution is so expensive to squander on the wrong thing.

Neither pathway will perfectly describe our actual future, but they can be thought of as two directional possibilities for our civilization.

A civilization of depth

Intelligence can advance knowledge in three ways. It can reason from principles. It can draw new insights from existing evidence. Or it can gather new evidence by observing and intervening in the world.

The first mode can travel far through thought alone. Math is the purest example. At the many crossroads in a proof or project, a powerful mind can consistently make the right call. It replaces years of floundering search with a short, more directed chain of reasoning.

The second mode is also a greenfield for fast progress. Humanity has collected far more evidence than it has understood. The study of our data—astronomical imagery, microscope slides, online comment sections—has been limited by the scarcity of human intelligence. Scientific archives contain observations recorded to answer one question that may hold answers to many others. Abundant genius can discover not by collecting anything new, but by seeing what was already there. It might find a new universe without gathering a new photon.

The third mode isn’t so easy because it requires us to interact with the physical world. Physical processes take time, and often little can be done to speed them up. Chemicals must react. Organisms must grow. Machines must be constructed. Spacecraft must cross actual distance, subject to cosmic speed limits. Even a perfect mind can’t observe the result of an experiment that has not occurred.

But it can try. A civilization of depth wouldn’t escape the need to consult reality. It would become radically economical in doing so. Better reasoning would identify the few experiments that truly matter. Simulations would resolve most and locate the precise uncertainty that reality must settle. Superintelligence can maximize efficiency, designing around its scarce resources to get the most out of them.

Picture an astrophysics lab with a set of hypotheses on dark matter. These are first shunted through hyperrealistic computer modeling. Each theory is checked for coherence with all existing data. Only after all this are a few targeted new snapshots of the sky taken to test the most promising theories. The civilization still needs the complements to genius, but each unit yields far more value.

Our path to a civilization of depth might work like a jigsaw puzzle. Progress is easy at the start, easy at the end, and hardest in the middle. At first there are obvious edges and easy matches. Then these run out, while too little of the picture is assembled to guide the rest. Near the end, the remaining gaps almost tell you what belongs in them. A nearly complete science might find its final discoveries easier than the ones we struggle toward today.

This is a staple of science. Mendeleev could describe undiscovered elements because the rest of the periodic table constrained what was missing. The Standard Model gave physicists reason to expect the Higgs boson decades before we observed it. Our experiments were targeted to look for what we expected to see—what was coherent with existing theory and data.

Superintelligence might achieve a civilization of depth by understanding the patterns of the world so well that it knows precisely what new data it needs. It would drive progress through deeper thought, not relying on the slow march of physical expansion.

If those efficiencies dominate, civilization could rapidly deepen its mastery over nature, with less need for institutional intelligence to govern vast new experimental apparatuses. Its physical footprint may stay small while its wisdom ripens.

We humans have already understood so much about our universe without ever leaving home. The civilization of depth would be a continuation of that tradition.

A civilization of width

But the demand for new evidence can overwhelm these efficiencies. Intelligence may cheapen each consultation with reality while discovering many more reasons to consult it. Better instruments open finer frontiers. Faster experiments can make additional experiments worthwhile.

If Einstein’s almost purely cognitive path to his theories of relativity typifies a civilization of depth, biology might best foreshadow a civilization of width. For all the knowledge we have accumulated, and all the data we have brought to bear, new medicines still have to be tested on large numbers of humans to know they are safe and effective. Machines can now simulate biological processes, allowing biologists to test new drugs in silico rather than in physical experiments, but those digital simulations aren’t faithful enough to the real world to substitute for large-scale human trials. More candidates from the machine make empiricism a greater bottleneck.

In a civilization of width, brilliance alone isn’t enough to make progress. Our need for physical resources—and a bureaucracy to manage their use—grows faster than a smart mind can economize on either.

This might be our path even if future AI systems greatly exceed human intelligence. A superintelligence could design a century of experiments in an afternoon, then spend the century waiting for nature and machinery to follow through. In such a world, it becomes more valuable to invest intelligence into the institutional processes needed to execute ideas than into the ideation itself.

The effect will compound if each further step requires more and more support infrastructure.

Here we would see relatively less thinking and relatively more building. Labs automate. Factories multiply. Energy production rises. Institutional intelligence dominates to make all this happen. Physical expansion becomes the engine of knowledge, and civilization spreads because its questions have outgrown the means of obtaining answers. More materials, more energy, more data to keep going.

A Dyson sphere is an extreme image of this future. Designing one requires extraordinary breakthroughs, yet most of the project wouldn’t consist of breakthrough thought. It would be construction and logistics on an astronomical scale.

The Dyson sphere requires brilliant minds to blueprint. It would also be among the most repetitive, monotonous, organizationally challenging projects ever undertaken.

This isn’t a failure of intelligence but rather a consequence of its success. Brighter minds discover more worthwhile projects; more complex projects need more matter to execute.

If this is so: the superpower of superintelligence might not be its genius. It would instead be its willingness to be the bureaucracy, to coordinate the vast technological and institutional machinery required to turn ideas into reality at ever-larger scale.

Almost all machine intelligence would be deployed no