Anthropic's Proposal for a Pause in AI Development
Anthropic, the company behind the Claude chatbot, has proposed that top artificial intelligence companies develop a coordinated way to pause the development of advanced AI systems. This proposal comes as a warning that the technology is improving so quickly that there is a risk of humans losing control.
According to Anthropic, as cutting-edge AI gets increasingly faster at carrying out tasks, it would be beneficial for the world to have the option to slow or temporarily pause its development. The company's internal research institute plans to explore this issue in collaboration with others and take actions to help build systems for a credible slowdown or pause.
OpenAI's Alternative Approach
Anthropic's rival, OpenAI, has argued for a different approach in a report published on Wednesday. OpenAI stated that democratic governments, rather than private companies acting alone, should ultimately determine the rules, safeguards, and accountability mechanisms for AI development. The company believes that decisions about the pace of AI innovation should not be left to any one lab, company, or special interest group.
AI models are getting faster, with rapid increases in how quickly they can carry out software tasks like coding on their own. Based on current trends and given enough computing power, an AI system could be able to design and develop its own successor, known as recursive self-improvement.
Risks and Benefits of Self-Building AI
Self-building AI would be a major technological milestone that could bring benefits in science, healthcare, and other areas. However, it also might increase the risks of humans losing control over AI systems. Some tech industry figures have long warned of such a scenario.
A team of researchers at the University of Toronto has shown how AI tools could be used to create a new kind of AI worm that adapts its hacking strategy as it spreads from device to device and takes over a vast computing network. This highlights the need for a coordinated global mechanism to pause AI development, allowing advanced AI labs to verify that global rivals have actually stopped or slowed their work.
Collaboration and Governance
Anthropic's proposal emphasizes the need for collaboration between companies, government agencies, and academic researchers to develop countermeasures as AI-powered hacking tools supercharge the search for computer vulnerabilities. Nicolas Papernot, a researcher at the University of Toronto, agrees that there should be more collaboration to develop countermeasures and warns that anything connected to the internet is now at risk due to the low cost of mounting cyberattacks.
The proposed coordination would let advanced AI labs verify that global rivals have actually stopped or slowed their work, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. A coordinated global mechanism is needed because without it, a slowdown in AI development could let the least cautious players catch up and add to pressure on companies and governments as they make tough choices about AI safety.
Conclusion
Anthropic's proposal for a pause in AI development highlights the need for a coordinated global approach to mitigate the risks associated with rapid advancements in AI technology. As AI models continue to improve, it is essential to develop systems for a credible slowdown or pause to ensure that humans do not lose control over AI systems.
Source: SecurityWeek