Stella Lorenzo, one of the AMD AI group's senior directors, checked 6,852 Claude code sessions. She went through 17,800+ thinking blocks, 234,760 tool calls, and more than 18,000 prompts and found that Claude was kind of getting dumber.
A classic instance of AI degradation, primarily caused by AI agent memory failures.
To get to the optimal output, users were running more and more correction sessions. The underlying LLM running the agent can be highly intelligent and capable but still make mistakes because its memory now has dumps of diverse tasks that you made it do.
Apart from that, in some other tests, the SWE bench verified vendors saw that Opus 4.5, Opus, and Gemini 3.1 Pro scored 80.9%, 80.8%, and 80.6%, respectively. The 0.3% gaps are basically nothing, but if you check how much these models can remember something from past chat sessions, you'll often come across weird AI hallucinations.
What do we learn from here?
It's not the model's intelligence capability but rather how efficiently it can recall from its memory that's critical for cases of software, coding, and product building.
Wanna vibe code and come up with your own small context model iteration or agent? Read this NFT.EU AI model guide to see which of the current models pulls up best.
A million-token context window doesn't equate to durable memory
All the top models right now, even the open-source ones, are aiming for a million-token context, and that's nice, actually, as someone can continue the same chat without moving to a new one. Exported conversation KV caches don't recalculate early tokens and continue running, so that you don't have to recall where to pull from the previous conversation.
A 2023 study named “Lost in the Middle” found that LLM outputs always tend to associate key facts at the start or at the end of outputs but seldom in the middle. Think of this as a safe way to follow through on the prompt instructions, but unfortunately it detaches the abstractness of humanity from putting in info wherever (and whenever) relevant.
Is raw context capacity meaningless then?
Context density matters more than raw capacity in AI models
Not entirely, but context density (a measure of how much input in a single prompt is required for the model to come up with the best output) matters more.
Need an example? Well, imagine two scenarios:
In the first one, you were attaching a 100-page PDF with 5 different files. The context window is still 1 million, but density is about 10%. The output will likely have more fluff, and it'll cost you more tokens to shape it, followed by potentially increased latency.
In scenario B, imagine you've put in the same source materials, but context density is about 80%. The response will be much more fluff-free, factual, and actually useful.
This isn't a hard pass or something that’s absolute but measuring output accuracy is the least you can do. If less input gives you faster and more likely results, then indeed context might be your problem.
A bad prompting habit is always writing long prompts. This creates context clash and your model can start to fumble matching new sets of longer instructions with the old ones that you've given but tweaked a little. It can jam up tweaks, old tweaks in the new output due to dumb guessing.
As per Anthropic, agentic use cases call for 4x more tokens, 15x more if it's a multi-agent system (the reason why people are making memes of million-dollar monthly token inverse as flows). Using large-sized source materials, multiple image variations, and poorly segregated memory can negatively impact your desired output.
You might argue: large context windows don't automatically translate to memory loss.
And you're not absolutely wrong with that opinion. Thing is…the more source information input, the more the agent will keep overanalysing it and your input prompt to deliver you the best output (especially if it's an Opus- or GPT SOL level LLM). Hence, it's always better to break down your tasks into 3-4 steps and, if needed, start a fresh chat to keep everything coherent.
Storage systems of memory first AI architectures
Persistent memory for AI Agents is basically a record kept for the module to follow through on your style of daily work and prompting.
Every user has a definite style or preference of output. The models simply store those preferences as memory in terms of small snippets of events, types of facts usually required, improvement and iteration prompts, and output language preference. Such simply adds to the personalisation of the output.
This is why separating agentic memory or creating multi-small agents, each for specific jobs, is good. If you work with agentic AI memory architecture daily, the four levels simply described are:
Working memory, which is the current plan or goal and any open questions the memory asks you back before producing the final output. Keep it as short and crisp at the start but as contextual as possible.
Episodic memory, which simply means the model's capability to store parts of dated conversations, tool calls, any failures, and the type of results that you love.
Semantic memory, which is how the agent ties up your earlier facts with your current requirements and creates a coherent event. Informational validation is still compulsory without fully depending on the model's capability.
Procedural memory, which simply means approved instruction sets and skills.md files that you prefer to use the most, distributed as per different time periods, keeping them intact and stored as per time periods. Version updates are very critical so that one chat LLM can be used by multiple users. Hence, nowadays the project folder feature.
Vector retrieval and knowledge graph usability
Similar meaning passages are filtered, re-ordered, and narrowed down using fine-tuning and re-ranking, which improves precision.
A knowledge graph basically creates links between synonymous elements for the model to churn out context whenever words and phrases are replaced. Mem0 and Honcho extract these as the main elements.
On LoCoMo, Mem0 gained about 26% over OpenAI memory on a full-context basis in memory-updating scores. It had about 95% less latency at 95% throughput, meaning 90%+ less token costs.
The main goal while working with AI should always be reducing the final cost. This means aiming for fewer reasoning steps, segregating the workload, and having useful memory so that agents don't need to go through source elements again and again. A consistent prompting style and preferred output style also help reduce discovery costs.
Context engineering is different from prompt engineering
Context engineering is different from prompt engineering
The latter only depends on the type of words used and their utility, but context engineering sends the info to the model. For each task, it pulls up live system state, any stored memory, be it the user's identity, tool calls, and set rules and regulations.
There are different kinds of assembly procedures, among which TERC(I) is a popular one.
T for task decides the result state, format, and metrics for checking success.
E for environment identifies the system, the user, their account, browser profiles, and elements that the user has given the model access to.
R for relevant facts. Models are always to be prompted to validate information by subject, element, and information recency.
C for constraints applies organisational policy restrictions, negative prompts, whatnot, not to do, basically hard limits on the model's memory.
A suggestion for anyone who's looking to write is to cut out noise from usable facts. Once done, simple filtration, segregation, data linkage, and logged IDs would do.
Tired of closed AI and subscriptions? Don't worry, you can create your own AI agent from scratch. Check out this NFT.EU AI agent building guide.
Enterprise AI should always have governed contextual knowledge
Combining a state machine, Postgres, and graphical storage together can work great for enterprise AI memory setup
Brain in a Jar is what Siemens calls such context setups. The brain must identify when to call the ERP, when to call any MES component, or when to simply call any online database MCP. Everything needs to be set up with negative prompts.
Combining a state machine, Postgres, and graphical storage together can work great for enterprise AI memory setup, as this allows authority to the relevant rule makers.
Governance is mandatory, or else about 40% of agentic AI projects will be cancelled by next year due to the risks of data loss and output vagueness. This is why multi-agent systems with enough cloud storage, high-speed internet for reduced latency, and proper negative-prompted guardrails are mandatory.
Someone should always be there to monitor the writing and rewriting. They must be skilled enough to trigger rollbacks if any context poisoning or system policy overwrites happen.
