Scott analyzes OpenAI's new deliberative alignment approach and explores different possibilities for who should ultimately control AI systems as they become more powerful. Longer summary

Scott discusses OpenAI's new paper on deliberative alignment, which combines constitutional AI with chain of thought reasoning to create more thoughtful AI responses. He explains how the process works by having AI models reflect on moral questions using a specification document. The post then explores different possible approaches to AI chains of command, including prioritizing companies, governments, specifications, moral law, average citizens, or humanity's coherent extrapolated volition. Scott expresses concern that we're heading toward either corporate or government control of AI systems, while acknowledging there may be better alternatives. Shorter summary