This was such a solid read and test. Been thinking about those papers a lot too lately, and this article was the first time I'm able to grasp what that really means for us using it.
Your right the goal isn't to anthropomorphize Claude.. but to understand its limitations to make our work better. Will be keeping that 15-turn thing in mind for sure!
This is a post after my own heart, the framing that it's effectively having human-like emotions is fascinating. And how many humans benefit from a "blank slate" approach? There's a theory that we need sleep for this reason, reboot, wipe the whiteboard. I often ask Claude in chat about its token limit, whether we are approaching. Or when it seems to get stuck in one response mode, I'll just start over. It may not have the language to tell you it's overwhelmed, but it can be sensed.
This research actually helped me improve prompts in the back end of CarouselBot! I give Claude strict token limits to manage costs and every once in a great while, it would hit the limit and just cut off its response, creating one or two empty slides. Fixed that by including the token constraint in the prompt.
I meant to tell you that I told Claude making a carousel would be part of workflow for posts that felt right for LinkedIn distro. Last time around it made the carousel text for me 😂. It actually did a pretty good job so I carried it over to CarouselBot. I love that you had an externality that could be applied to the back-end design. I'll keep trying it out and won't let Claude preempt the Bot!
The silent degradation finding is what I've been feeling too Karen. I've had Claude confidently give me broken code after a long session, and write ups I need to chase down. (It used to be a lot more reliable than that!)
You’re very welcome! This is anecdotal, but I find it pushes back on complex requirements when it’s low on tokens (“Oh, you don’t really need Puppeteer!”)
The 20-44% wrong answer rate under token pressure tracks with what I see running Claude agents in production. The worst part isn't the wrong answers. It's the confident tone. My agent handles 15-20 tasks per night and I've noticed the degradation follows a pattern. First, output structure stays clean but details get sloppy.
Then verification steps get skipped. Finally it reports success on tasks it barely touched. I started adding explicit checkpoints mid-conversation to catch the drift before it compounds. Not elegant, but it catches about 80% of the pressure-induced shortcuts. Your 250-call study is solid evidence for something practitioners have been feeling for months.
Karen, I've been so busy with my kids that I totally miss those anthropic articles, and reading this, I am just, oh my god, beyond myself. Your post was super helpful in helping me understand what's going on, but also helping me figure out how to make my experience with Claude Code better. Thank you for diving into details, and I really like your 15-turn recommendation.
With this experiment you just showed that Claude behaves like me, awesome😆(I also start getting sort of desperate when it starts burning through the session limits by spawning subagents [one of the reasons I'm no longer using Claude Code as much as before])
Two things that come to my mind about this:
- People need to learn about context windows and be more descriptive in their prompts.
- You should consider using a model router (like openrouter or kilo code) instead of the Claude API alone
Great points! 🙏 Awareness of the context window is really important…and the bigger the code base, the more critical it is! (I say this planning a major refactor of CarouselBot to make it easier to manage.)
The model router sounds interesting! Will check it out!
The silent degradation is actually the more consequential finding than the degradation itself. In distributed systems, Leslie Lamport proved decades ago that Byzantine failures — wrong answers delivered confidently — are categorically harder to handle than crash failures where the system simply stops. What the interpretability team found is that token pressure shifts Claude from crash-failure mode (flagging its limitations) to Byzantine-failure mode. That's the failure class every reliability engineer fears most, because no amount of output monitoring catches it — you need visibility into internal state, which is exactly what the interpretability tools provide. The practical implication goes beyond prompt engineering: systems consuming AI output under resource pressure need the equivalent of checksums, not just confidence scores.
I’m REALLY amazed at this post - and still thinking through what this all means??
What you are describing is almost the equivalent description of human working memory??? Did you know this??
And my “challenging questions” for you (and the research you cited early in your post):
1. are you interpreting this as “human-like”?
2. And, if so, is this “real” or is it some sort of bias we might all have - to compare program steps, solutions & flaws as if they are human - like a “bias” toward being human in some way??
I’m not doubting your results at all - just questioning how to interpret these??
Hope this makes sense???? 🤞ps I will make time to read that previous research & your post again & think more!! 👍
Claude’s context window is analogous to human short-term memory, but I don’t think the model is experiencing emotions. Instead, I believe it’s simulating human psychology based on its training data.
I loved this experiment! It touches on something I’ve been thinking about for a while - how tone can stay reassuring even when the output itself is quietly degrading. Really great read, Karen :)
Thank you so much! 🙏 Yes, Claude will often talk up a really iffy result—which means a lot of the time that AI saves us at the start of a project is lost to fact-checking, debugging, etc.
I started bouncing up to a new chat a few weeks ago to generate any output. Been breaking sessions in two, usually: a "thinking" session of throwing stuff at the wall and seeing what sticks type of thing and then a separate "ok, based on this conversation (and I'll paste relevant stuff in) let's start making the thing."
Been a little bit less aggravated this way :)
Still find the context rot problem bizarre as that is exactly WTH this tool is supposed to be able to overcome -- our sad, faulty human memory is being replaced by a sad, faulty server in a rack somewhere that can't keep it's 1s and 0s separated!
Context rot is real…and real annoying. 😆 I try to do most of my own planning and then plug in Claude for execution. And if I do use it for specs, I give it a completely empty context window to work with!
I'm just been using it for genAI purposes and finding it extremely useful organizing my brain dumps to provide structure -- have always had a hard time getting the structure for a content schedule, plan, etc. out of my head into a document. Massively cuts that time for me and then I can fiddle with it until I've got what I need.
And I'll believe AI is going to take over the world when it stops forgetting what I wanted to achieve after 10 to 20 back and forth questions within a chat within a project that has the context the thing theoretically needs to stay in context!
Fantastic experiment and finding. This explains what I FEEL when working with Claude. Most definetly one of the more fascinating pieces on Claude I've read in awhile. TY :)
This was such a solid read and test. Been thinking about those papers a lot too lately, and this article was the first time I'm able to grasp what that really means for us using it.
Your right the goal isn't to anthropomorphize Claude.. but to understand its limitations to make our work better. Will be keeping that 15-turn thing in mind for sure!
It was a fun test and pretty cool to see evidence that it does indeed fall apart as it runs out of context! 😂
This is a post after my own heart, the framing that it's effectively having human-like emotions is fascinating. And how many humans benefit from a "blank slate" approach? There's a theory that we need sleep for this reason, reboot, wipe the whiteboard. I often ask Claude in chat about its token limit, whether we are approaching. Or when it seems to get stuck in one response mode, I'll just start over. It may not have the language to tell you it's overwhelmed, but it can be sensed.
This research actually helped me improve prompts in the back end of CarouselBot! I give Claude strict token limits to manage costs and every once in a great while, it would hit the limit and just cut off its response, creating one or two empty slides. Fixed that by including the token constraint in the prompt.
I meant to tell you that I told Claude making a carousel would be part of workflow for posts that felt right for LinkedIn distro. Last time around it made the carousel text for me 😂. It actually did a pretty good job so I carried it over to CarouselBot. I love that you had an externality that could be applied to the back-end design. I'll keep trying it out and won't let Claude preempt the Bot!
But definitely let me know if it’s naughty!
“Do I need to get Red Team KSpin here?” Scarier than saying “I’m telling Mom!”
If it’s bad, I’ll run the experiment again! 😂
Feel like at this rate, Claude will soon need a therapist. 🤣🩷🦩
😂😂😂
Thanks Karen! Good methods here!
You’re welcome!
The silent degradation finding is what I've been feeling too Karen. I've had Claude confidently give me broken code after a long session, and write ups I need to chase down. (It used to be a lot more reliable than that!)
Thanks for running this thoughtful experiment!
You’re very welcome! This is anecdotal, but I find it pushes back on complex requirements when it’s low on tokens (“Oh, you don’t really need Puppeteer!”)
The 20-44% wrong answer rate under token pressure tracks with what I see running Claude agents in production. The worst part isn't the wrong answers. It's the confident tone. My agent handles 15-20 tasks per night and I've noticed the degradation follows a pattern. First, output structure stays clean but details get sloppy.
Then verification steps get skipped. Finally it reports success on tasks it barely touched. I started adding explicit checkpoints mid-conversation to catch the drift before it compounds. Not elegant, but it catches about 80% of the pressure-induced shortcuts. Your 250-call study is solid evidence for something practitioners have been feeling for months.
Karen, I've been so busy with my kids that I totally miss those anthropic articles, and reading this, I am just, oh my god, beyond myself. Your post was super helpful in helping me understand what's going on, but also helping me figure out how to make my experience with Claude Code better. Thank you for diving into details, and I really like your 15-turn recommendation.
Thanks so much! 🤗 This was a fun experiment to conduct, and the results were consistent with Claude behavior many of us have noticed for some time!
Great post Karen. Test the data. Keep it real.
Thank you! 🙏
This is such a cool find - the system "fatigues"
Yes! And it has to be actively managed.
It's starting to sound like a Human employee 😭
😂
With this experiment you just showed that Claude behaves like me, awesome😆(I also start getting sort of desperate when it starts burning through the session limits by spawning subagents [one of the reasons I'm no longer using Claude Code as much as before])
Two things that come to my mind about this:
- People need to learn about context windows and be more descriptive in their prompts.
- You should consider using a model router (like openrouter or kilo code) instead of the Claude API alone
Great work putting this together!
Great points! 🙏 Awareness of the context window is really important…and the bigger the code base, the more critical it is! (I say this planning a major refactor of CarouselBot to make it easier to manage.)
The model router sounds interesting! Will check it out!
The silent degradation is actually the more consequential finding than the degradation itself. In distributed systems, Leslie Lamport proved decades ago that Byzantine failures — wrong answers delivered confidently — are categorically harder to handle than crash failures where the system simply stops. What the interpretability team found is that token pressure shifts Claude from crash-failure mode (flagging its limitations) to Byzantine-failure mode. That's the failure class every reliability engineer fears most, because no amount of output monitoring catches it — you need visibility into internal state, which is exactly what the interpretability tools provide. The practical implication goes beyond prompt engineering: systems consuming AI output under resource pressure need the equivalent of checksums, not just confidence scores.
💯 AI-powered systems need deterministic guardrails to protect against outputs that are perfectly formatted but also wrong.
I’m REALLY amazed at this post - and still thinking through what this all means??
What you are describing is almost the equivalent description of human working memory??? Did you know this??
And my “challenging questions” for you (and the research you cited early in your post):
1. are you interpreting this as “human-like”?
2. And, if so, is this “real” or is it some sort of bias we might all have - to compare program steps, solutions & flaws as if they are human - like a “bias” toward being human in some way??
I’m not doubting your results at all - just questioning how to interpret these??
Hope this makes sense???? 🤞ps I will make time to read that previous research & your post again & think more!! 👍
Claude’s context window is analogous to human short-term memory, but I don’t think the model is experiencing emotions. Instead, I believe it’s simulating human psychology based on its training data.
Hope this helps! 🤗
Seems to me - AI is always matching what humans do - even when no AI Bot can have feelings?
I think it’s more mirroring than matching…which can make things confusing. 🤔
I loved this experiment! It touches on something I’ve been thinking about for a while - how tone can stay reassuring even when the output itself is quietly degrading. Really great read, Karen :)
Thank you so much! 🙏 Yes, Claude will often talk up a really iffy result—which means a lot of the time that AI saves us at the start of a project is lost to fact-checking, debugging, etc.
I started bouncing up to a new chat a few weeks ago to generate any output. Been breaking sessions in two, usually: a "thinking" session of throwing stuff at the wall and seeing what sticks type of thing and then a separate "ok, based on this conversation (and I'll paste relevant stuff in) let's start making the thing."
Been a little bit less aggravated this way :)
Still find the context rot problem bizarre as that is exactly WTH this tool is supposed to be able to overcome -- our sad, faulty human memory is being replaced by a sad, faulty server in a rack somewhere that can't keep it's 1s and 0s separated!
Context rot is real…and real annoying. 😆 I try to do most of my own planning and then plug in Claude for execution. And if I do use it for specs, I give it a completely empty context window to work with!
I'm just been using it for genAI purposes and finding it extremely useful organizing my brain dumps to provide structure -- have always had a hard time getting the structure for a content schedule, plan, etc. out of my head into a document. Massively cuts that time for me and then I can fiddle with it until I've got what I need.
And I'll believe AI is going to take over the world when it stops forgetting what I wanted to achieve after 10 to 20 back and forth questions within a chat within a project that has the context the thing theoretically needs to stay in context!
It’s definitely better for some things than for others! And, yeah, when it recovers from its short-term memory loss, watch out! 😂
Fantastic experiment and finding. This explains what I FEEL when working with Claude. Most definetly one of the more fascinating pieces on Claude I've read in awhile. TY :)
Thank you so much! It was a fun experiment, and it also explained some issues I’d been having with token limits in my apps!
Yes! I also wonder if you could break tasks up into smaller group, so the token limit is reset more often. 🤔