以防你还没听够，这里再有一条关于人工智能的警告：包括杰弗里·辛顿（Geoffrey Hinton）、约书亚·本希奥（Yoshua Bengio）以及 OpenAI 研究主管雅各布·帕乔基（Jakub Pachocki）在内的 20 多名 AI 研究人员在新发表的一篇研究论文中警告称，由自我改进型 AI 驱动的“智能爆炸”可能发生。他们写道：“尽管仍存在许多不确定性，但 AI 研发自动化可能会很快引发这一现象。”

根据该研究，AI 系统已经在构建它们的公司中编写了大部分代码，并可能在数年内实现整个 AI 研发流程的自动化。原本需要数年才能取得的进展，届时可能只需数月即可完成。作者警告称，社会可能无法跟上步伐，对超人类 AI 的控制权可能会丧失，国家、公司和政府之间的权力平衡也可能被削弱。他们敦促政策制定者更深入地了解 AI 研究是如何被自动化的。

这篇论文加入了日益增长的警告名单。42 位顶尖数学家最近呼吁更多关注存在性 AI 风险，多家 AI 实验室的员工表达了对 AI 可能毁灭人类的恐惧，而帕乔基本人也表示，没有哪家实验室已经足够好地解决了对齐问题，“以便在更长时间内以最高速度负责任地扩展规模”。据报道，一些 Anthropic 的员工已经在寻找避难所，以防 AI 发展出现偏差。

去伪存真的人工智能新闻——由人工策划

订阅 THE DECODER，享受无广告阅读、每周 AI 通讯、每年六次的独家“AI Radar”前沿报告、完整档案访问权限以及评论区的参与资格。

立即订阅

In case you didn't have enough, here's another AI warning: More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki, warn of a possible "intelligence explosion" driven by self-improving AI in a new research paper . "Although there remains much uncertainty, AI R&D automation might soon trigger one," they write.

According to the study , AI systems already write most of the code at the companies building them and could automate the entire AI R&D pipeline within years. Progress that normally takes years could then happen in months. The authors warn society might not keep up, control over superhuman AI could slip away, and power balances between nations, companies, and governments could erode. They urge policymakers to gain far more visibility into how AI research is being automated.

The paper joins a growing list of warnings. 42 leading mathematicians recently called for more attention to existential AI risks, several AI lab employees have voiced fears that AI could destroy humanity, and Pachocki himself said no lab has solved alignment well enough "to continue responsibly scaling at maximum speed for much longer." Some Anthropic employees are reportedly already scouting safe havens in case AI goes sideways .

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now