This is a paper about ASI safety.
Abstract:
As artificial intelligence technology rapidly advances, it is likely to implement Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI) in the future. The highly intelligent ASI systems could be manipulated by malicious humans or independently evolve goals misaligned with human interests, potentially leading to severe harm or even human extinction. To mitigate the risks posed by ASI, it is imperative that we implement measures to ensure its safety and controllability. This paper analyzes the intellectual characteristics of ASI, and three conditions for ASI to cause catastrophes (harmful goals, concealed intentions, and strong power), and proposes a comprehensive safety solution. The solution includes three risk prevention strategies (AI alignment, AI monitoring, and power security) to eliminate the three conditions for AI to cause catastrophes. It also includes four power balancing strategies (decentralizing AI power, decentralizing human power, restricting AI development, and enhancing human intelligence) to ensure equilibrium between AI to AI, AI to human, and human to human, building a stable and safe society with human-AI coexistence. Based on these strategies, this paper proposes 11 major categories, encompassing a total of 47 specific safety measures. For each safety measure, detailed methods are designed, and an evaluation of its benefit, cost, and resistance to implementation is conducted, providing corresponding priorities. Furthermore, to ensure effective execution of these safety measures, a governance system is proposed, encompassing international, national, and societal governance, ensuring coordinated global efforts and effective implementation of these safety measures within nations and organizations, building safe and controllable AI systems which bring benefits to humanity rather than catastrophes.
Content:
Contact:
Chinese Version:
摘要：
随着AI（人工智能）技术的快速发展，未来很可能会发展出AGI（通用人工智能）以及ASI（超级人工智能）。高度智能的ASI可能被恶意人类利用，或者其自身发展出与人类不一致的目标，从而给人类带来巨大的伤害，极端情况下甚至可能导致人类灭绝。为了防范ASI带来的风险，我们需要采取措施确保ASI安全可控。本文通过分析ASI的智力特点和AI造成灾难的三个条件（有害目标、隐藏意图、强大实力），设计出一套全面的安全解决方案，包括三个风险防范策略（AI对齐、AI监控、实力安全）来消除AI造成灾难的三个条件，以及四个实力均衡策略（分散AI实力、分散人类实力、限制AI发展、提升人类智力）来确保AI与AI、AI与人、人与人之间的实力均衡，从而实现安全和稳定的人机共存的社会。基于上述策略，本文提出了11个大类，共47项具体安全措施，为每个安全措施设计了具体的方案，并对其实施的收益、成本、阻力进行评估，给出相应的优先级。此外，为了确保这些安全措施的有效落实，本文还设计了相应的治理体系，包括国际治理、国内治理和社会治理，确保AI安全措施能够在全球范围内协调一致，在各个国家、各个机构内部得到有效落实，从而确保AI安全可控，为人类带来福祉而不是灾难。
内容：