Pivot – AI Ethics at Code 2023

Intro

In this episode of the “Pivot” podcast, the hosts discuss the rapid development of AI and the potential risks and harms that come with it. They delve into the use of facial recognition systems by the police, the need for AI safety research, and the challenges of regulating AI. The episode also explores responsible scaling policies, the role of AI companies’ boards, and the debate between closed-source and open-source AI tools. The hosts emphasize the importance of considering public opinion and safety when developing AI technologies.

Main Takeaways

The Risks of AI

• AI is developing rapidly and in ways that pose significant risks to all of us.
• Risks and harms of AI are being presented today that may not be getting enough attention.
• Police using facial recognition systems to make arrests despite knowing that these systems don’t work well, especially for minority groups.
• Large language models have unpredictable capabilities when trained with more data and compute.
• Scaling laws predict performance on narrow benchmarks but not real-world relevant capabilities.

AI Safety and Responsible Scaling

• Medium-term focus on AI safety and how to figure out in advance if AI is going to hurt us.
• Funding technical research to make advanced AI systems safer.
• Developing products with technology without knowing its safety is common, but funding technical research can make advanced AI systems safer.
• Serious experts warn that AI systems could make it accessible to build a bio weapon or escape our control entirely.
• The challenge with AI is not knowing what capabilities will emerge when making the systems bigger, making it difficult to know how to regulate it.

Responsible Scaling Policies and OpenAI’s Board

• Responsible scaling policies aim to bridge the gap between regulating AI and allowing innovation.
• Anthropic, a major AI company, recently released the first responsible scaling policy.
• AI companies need to clearly state the capabilities they are ready to deal with and the protective measures in place for other capabilities.
• The workflow for responsible scaling involves forecasting the system’s capabilities, testing it in the middle of training, and pausing if it exceeds expectations until the necessary measures are in place.
• The OpenAI board has the authority to hire and fire the CEO, but not to shut down the company.

Closed-Source vs. Open-Source AI Tools

• There are two major approaches to releasing AI tools: closed source (OpenAI, Google) and open source (Meta).
• The safer approach depends on what is being built: a toaster or a nuclear bomb.
• AI companies should rigorously test their models and release them to the degree that is safe, potentially involving the board or government in the process. Some models should be open-sourced, while others may be too dangerous to exist at all.
• AI models that are safe and have beneficial uses should be accelerated and made available to different communities.
• There should be a process for deciding which models should be open-sourced, minimizing false positives and false negatives.

Considerations for AI Development

• Public opinion should be taken into account when developing AI, with a focus on using it to make the future better with clean energy and better medical solutions.
• Businesses should keep safety in mind when evaluating AI technologies to integrate into their operations.
• AI developers should rigorously test their models and involve the board or government in the process for safe release.
• Privacy-enhancing technologies like federated learning and technical differential privacy can help protect private data used to train AI models.
• Optimizing chatbots to be more engaging may not necessarily reflect what humans would endorse if they thought about it objectively.

Summary

The Risks of AI and the Need for Safety

The hosts highlight the rapid development of AI and the potential risks it poses to society. They discuss the use of facial recognition systems by the police, despite their known limitations, particularly for minority groups. The unpredictability of large language models trained with more data and compute is also emphasized. They stress the importance of funding technical research to make advanced AI systems safer and address the concerns raised by experts regarding the potential misuse of AI to build bio weapons or escape human control.

Responsible Scaling and Regulation

The hosts delve into responsible scaling policies, focusing on Anthropic’s recent release of their policy. They explain the need for AI companies to clearly state the capabilities they can handle and the measures in place for other capabilities. The workflow for responsible scaling, including forecasting system capabilities and testing during training, is discussed. The role of the OpenAI board in overseeing the company’s mission alignment with AGI being beneficial to humanity is also highlighted.

Closed-Source vs. Open-Source AI Tools

The hosts explore the debate between closed-source and open-source AI tools. They emphasize the importance of rigorously testing AI models before release, involving the board or government in the process for safe deployment. The decision of whether to open-source or keep certain models closed depends on their potential risks. The hosts stress the need to accelerate the availability of safe and beneficial AI models for different communities while carefully considering which models should be open-sourced.

Considerations for AI Development

The hosts discuss the significance of public opinion in AI development, particularly in using AI to improve clean energy solutions and medical advancements. They advise businesses to prioritize safety when evaluating AI technologies for integration. The hosts also highlight the importance of privacy-enhancing technologies like federated learning and technical differential privacy to protect private data used to train AI models. They caution against optimizing chatbots for engagement without considering the potential ethical implications.

Conclusion

This episode of the “Pivot” podcast sheds light on the risks and challenges associated with the rapid development of AI. It emphasizes the need for AI safety research, responsible scaling policies, and the consideration of public opinion in AI development. The hosts advocate for rigorous testing, involvement of boards or governments, and the use of privacy-enhancing technologies to ensure the safe and beneficial deployment of AI. They encourage businesses to prioritize safety and ethical considerations when integrating AI technologies into their operations.