Scott Alexander examines GPT-3's capabilities, improvements over GPT-2, and potential implications for AI development through scaling. Longer summary

Scott Alexander discusses GPT-3, a large language model developed by OpenAI. He compares its capabilities to its predecessor GPT-2, noting improvements in text generation and basic arithmetic. The post explores the implications of GPT-3's performance, discussing scaling laws in neural networks and potential future developments. Scott ponders whether continued scaling of such models could lead to more advanced AI capabilities, while also considering the limitations and uncertainties surrounding this approach. Shorter summary

Scott Alexander plays chess against GPT-2, an AI language model, and discusses the broader implications of AI's ability to perform diverse tasks without specific training. Longer summary

Scott Alexander describes a chess game he played against GPT-2, an AI language model not designed for chess. Despite neither player performing well, GPT-2 managed to play a decent game without any understanding of chess or spatial concepts. The post then discusses the work of Gwern Branwen and Shawn Presser in training GPT-2 to play chess, showing its ability to learn opening theory and play reasonably well for several moves. Scott reflects on the implications of an AI designed for text prediction being able to perform tasks like writing poetry, composing music, and playing chess without being specifically designed for them. Shorter summary

Scott Alexander humorously describes AI-generated content simulating humans pretending to be robots pretending to be humans on Reddit. Longer summary

Scott Alexander humorously discusses the intersection of two subreddits: r/totallynotrobots, where humans pretend to be badly-disguised robots, and r/SubSimulatorGPT2, which uses GPT-2 to imitate various subreddits. The result is a AI-generated simulation of humans pretending to be robots pretending to be humans. Scott shares an example of this amusing output and expresses wonder at the current state of technology. Shorter summary

Scott Alexander examines AI-generated poetry produced by Gwern's GPT-2 model trained on classical poetry, highlighting its strengths and limitations. Longer summary

Scott Alexander reviews Gwern's experiment in training GPT-2 on poetry. The AI-generated poetry shows impressive command of meter and occasionally rhyme, though it tends to degrade in quality after the first few lines. Scott provides numerous examples of the AI's output, ranging from competent imitations of classical styles to more experimental forms. He notes that while the AI sometimes produces nonsensical content, it can also generate surprisingly beautiful and coherent lines. The post concludes with a reflection on how our perceptions of poetry might be influenced by knowing whether it's human or AI-generated. Shorter summary

Scott Alexander explores GPT-2's unexpected capabilities and argues that it demonstrates the potential for AI to develop abilities beyond its explicit programming, challenging skepticism about AGI. Longer summary

This post discusses GPT-2, a language model AI, and its implications for artificial general intelligence (AGI). Scott Alexander argues that while GPT-2 is not AGI, it demonstrates unexpected capabilities that arise from its training in language prediction. He compares GPT-2's learning process to human creativity and understanding, suggesting that both rely on pattern recognition and recombination of existing information. The post explores examples of GPT-2's abilities, such as rudimentary counting, acronym creation, and translation, which were not explicitly programmed. Alexander concludes that while GPT-2 is far from true AGI, it shows that AI can develop unexpected capabilities, challenging the notion that AGI is impossible or unrelated to current AI work. Shorter summary

Scott Alexander draws parallels between OpenAI's GPT-2 language model and human dreaming, exploring their similarities in process and output quality. Longer summary

Scott Alexander compares OpenAI's GPT-2 language model to human dreaming, noting similarities in their processes and outputs. He explains how GPT-2 works by predicting next words in a sequence, much like the human brain predicts sensory input. The post explores why both GPT-2 and dreams produce narratives that are coherent in broad strokes but often nonsensical in details. Scott discusses theories from neuroscience and machine learning to explain this phenomenon, including ideas about model complexity reduction during sleep and comparisons to AI algorithms like the wake-sleep algorithm. He concludes by suggesting that dream-like outputs might simply be what imperfect prediction machines produce, noting that current AI capabilities might be comparable to a human brain operating at very low capacity. Shorter summary