Scott examines the debate over open-weights AI, explaining why he remains neutral despite risks of criminal misuse, arguing that waiting for inevitable incidents is more strategic than preemptively burning political capital fighting the strong pro-open-weights coalition. Longer summary

Scott discusses the debate over open-weights AI (where AI model weights are publicly available, similar to open-source software). He explains that while open weights AI offers freedom from corporate control, it also enables criminal misuse like hacking and bioterrorism. Despite this being a reasonable concern for AI safety advocates, Scott argues most remain neutral because waiting for the first incidents to occur (rather than preemptively fighting a political battle) is the more strategic approach. He distinguishes between existential risks from superintelligence (which require preemptive action) and criminal misuse risks (which will trigger government response after initial incidents). Scott concludes that the open-weights community deserves a chance to prove their 'good guy with an AI' defense theory can work, even though he's skeptical, because fighting them preemptively would waste political capital on a likely-losing battle. Shorter summary

Scott Alexander analyzes California's AI regulation bill SB1047, finding it reasonably well-designed despite misrepresentations, and ultimately supporting it as a compromise between safety and innovation. Longer summary

Scott Alexander examines California's proposed AI regulation bill SB1047, which aims to regulate large AI models. He explains that contrary to some misrepresentations, the bill is reasonably well-designed, applying only to very large models and focusing on preventing catastrophic harms like creating weapons of mass destruction or major cyberattacks. Scott addresses various objections to the bill, dismissing some as based on misunderstandings while acknowledging other more legitimate concerns. He ultimately supports the bill, seeing it as a good compromise between safety and innovation, while urging readers to pay attention to the conversation and be wary of misrepresentations. Shorter summary

Scott Alexander discusses recent events in prediction markets, including a costly prank on Manifold, debt ceiling forecasts, a study on long-term predictions, and a high-profile hyperinflation bet. Longer summary

This post covers several topics in the world of prediction markets and forecasting. It begins with the story of a Manifold user who lost $29,000 in a prank market, leading to changes in Manifold's policies. It then discusses recent predictions about the US debt ceiling, a study on long-term forecasting accuracy, and a high-profile bet about hyperinflation between Balaji Srinivasan and James Medlock. The post ends with updates on other prediction markets, including ones about open-source AI and the ACX book review contest. Shorter summary