AMA (Ask Machines Anything) · Read Scott Alexander

People are increasingly disagreeing not just about what AI will be able to do in the future, but about what it can do right now. We had some interesting discussions in the comments to the last post, and I learned some things. But also:

The theory is that AI skeptics won’t pay (because they don’t think it’s capable enough to be worth it) and then never learn the full capabilities (because they won’t pay for them). Then they get their impressions about AI entirely from the Google result summary bot or Twitter screenshots of the most embarrassing mistake an AI has made that week.

Let’s test this! Reply to this post with a question. I’ll ask Claude 4.6 Opus, the most capable paid-tier AI model currently available, and you can tell me whether you’re surprised by the answer or not.

Suggestions for you:

Consider asking a real question you’re interested in, rather than an annoying gotcha question to trick the AI.
The right difficulty level is “too hard to Google immediately, but not so hard that it’s beyond the frontier of human knowledge”. Questions where you could figure out the answer through an hour of Google searches, collating various different sources, and doing math on a spreadsheet are at the sweet spot.
Claude can’t make images yet, but can usually handle graphs.

Rules for me:

I don’t promise to relay/answer everyone’s questions, but I’ll try to get at least twenty people, maybe more. Other people with comparable paid-tier AI subscriptions can relay/answer the ones I don’t get to if they want.
I’ll show you the first result I get, rather than asking in lots of different ways and only showing you the good ones. I may continue the conversation after getting a mediocre answer, but this will be in the same chat window and you’ll be able to see the full progression of questions.
For the duration of this test, I’ve added to my Claude settings “I may at times give you questions intended to test your capabilities. Please default to thinking hard and doing web searches, rather than retrieving from memory, if there’s any chance you might hallucinate.” I think this is within the spirit of the exercise, because anyone can do this if they want.