The Law of Contextual Decay

/A

Martijn Veldkamp

/A

August 15, 2025

I was on PTO in lovely Luxemburg (nature sure needs a lot of rain to look so good) and the campsite where we stayed had a folder for enticing people to come see this German Dino park (Teufelsschlucht). This made me reminiscent of times in the south of France were our family indeed visited those kind of parks. And that sparked something. This article is about exploring the connection between plastic dinos and LLMs.

Jurassic Park

For a very long time the Tyrannosaurus Rex stomped through our collective imagination as a green, scaly, cold-blooded monster. All the Jurassic Park movies that Michael Crichton inspired certainly helped cement those images.

(image credit: Murray Close/Getty Images)

We sort of take for granted these depictions of prehistoric animals. We even go to these touristic dino park to look at them. I still remember that one park in Rocamadour where some the Dinos could roar, my kids were terrified.

But piecing together Earth’s murky past is a lot harder than it sounds. We built this image from the only evidence we had: colossal bones. The skeleton was the hard data. From this framework, we extrapolated a creature that was powerful, terrifying, and distinctly reptilian. It was a logical conclusion drawn from an incomplete dataset. Scientists have to rely on fragmentary bones, weathered footprints and impressions in rock. These don’t always capture the fine details of the complex, living, breathing animal that passed through and died there.

New evidence

But in recent decades new evidence has begun to paint a different picture. One with feathers, complex social behaviors, and a metabolism closer to a bird than a lizard. The skeleton wasn’t wrong, it was just a fraction of the story. (Also there are so many discussion on fossils and skeletons and what is the right way to put together, this will not end soon, see Archaeopteryx). Sometimes, while doing this painstaking work, researchers get it wrong. And let’s not discuss the frauds (yeah you, Piltdown Man). The history of paleontology contains countless examples of soft tissue reconstructions that later proved wildly inaccurate, from T. rex’s arm positioning to the trunk shape of woolly mammoths.

Code skeleton

We are doing something similar today, not in paleontology, but in the digital realm of Artificial Intelligence. We are feeding Large Language Models the fossilized remains of human logic: millions of lines of code from repositories like GitHub. Like the onlookers of old, we are easily impressed by the skeletons these models can construct. They generate functional code that assembles into applications.

We are also discovering that the code is just the skeleton. The one thing of truly great software, the soft tissue of development, is missing. This is about the missing context, the ghosts in the machine that separate functional code from robust, and maintainable systems. In my earlier articles I tried to articulate that underlying something. I called it in a cynical mode:

• The First Law of Engineering Thermodynamics: Problem energy can neither be created nor destroyed, only changed in form. That was focussed on the engineers perspective. Then I came up with another one.
• The Unspoken Second Law of technological entropy: In a closed system, the relentless injection of new features increases complexity until the point of collapse.

I now have found a third law! Flowing directly from my first two, is The Law of Contextual Decay:

The context surrounding a component decays far more rapidly than the component itself. Over time, a system’s structure is mistaken for its purpose.

Reconstructing the Dinosaur (The Feathers)

When the first dinosaur bones were discovered, they were interpreted through the lens of what we knew. Scientists saw massive leg bones and vertebrae and thought, giant lizard. This led to the classic depiction of dinosaurs as slow, lumbering, tail-dragging behemoths.

However, our notion began to shift not because the original bones were wrong, but because new, more subtle forms of data were discovered. In China, paleontologists found fossils of small theropods, like Sinosauropteryx, surrounded by the unmistakable imprints of fine, downy feathers. And closer analysis of bone microstructure suggested rapid growth rates, more akin to warm-blooded birds and mammals than cold-blooded reptiles. Also examination of preserved feather fossils revealed melanosomes. These are pigment containing organelles. Which allows scientists to reconstruct the actual colors and patterns of some dinosaurs.

ar.inspiredpencil.com

A T-Rex may have been more than just green or grey. My inner child would have loved to play with a cockatoo cross over. We shouldn’t just be looking for bones, but for the subtle traces of life left around them.

Assembling the LLM (The Architecture Meetings)

Now, consider an LLM being trained to become a coding assistant. It ingests a staggering volume of data. Essentially the entire public digital fossil record of software development. It learns syntax, patterns, library usage, and common algorithms. Give it a prompt, and it can generate a Python script to parse a CSV file or a JavaScript function to call an API. It has reconstructed the skeleton.

The problem is, this model, while incredibly useful, is as incomplete as the scaly T-Rex. The code in a repository is the result of a development process, not the process itself. There is critical soft tissue context that the LLM is missing.

Code tells you what the system does. It rarely tells you why it was built that way. It’s one of the tells if when doing code reviews. When the comments in the code tell what is happening instead of why.

The LLM never sat in the architecture meeting where we debated the trade-offs between current architecture or microservices architecture. It wasn’t there when we decided to use a NoSQL database because of the specific nature of the expected data, even though a relational database was easier. It doesnt’ see the whiteboard diagrams, the design documents, or the business constraints that shaped the final product.

That leaves us with what?

A dinosaur reconstructed only from bones is a lifeless plastic monster. An LLM trained only on code is a syntax-perfect, context-blind machine. The consequences are remarkably similar: We imagined a creature that couldn’t possibly have survived in its environment. A cold-blooded, slow T-Rex would have been an ineffective hunter. Adding feathers and warm-bloodedness makes it a viable, terrifyingly efficient apex predator.

The LLM can generate code that is technically correct but architecturally naive. It might produce a solution that works for 10 users but collapses under the load of 10,000. It might re-introduce a security flaw that was patched years ago because the flawed pattern is more common in its training data than the fix.

My conclusion: The Human in the Loop is the Context

Our understanding of dinosaurs didn’t stagnate. Paleontologists actively sought out the missing context. They keep discussing Brontosaurus or a Apposaurus. They developed new tools and new ways of thinking to find the feathers hidden in the barnstone. Similarly, the future of AI in software development isn’t about replacing the developer. Its about recognizing the LLM as a powerful tool. I use it as a pair programmer with encyclopedic knowledge but zero experience. The developers role becomes more critical, shifting from writing boilerplate code to providing the essential context the model lacks.

The developer is the one who attends the architecture meeting, who understands the business needs, who remembers the lessons from past failures. The job is to be the senior guide for the AI, to ask the right questions, to validate the output against the deep, unwritten context of this project, but also the other one you’ve did.

We were wrong about the dinosaurs because we mistook the skeleton for the creature. Lets not make the same mistake with our code. The repository is not the project, and the code is not the context. True intelligence, whether biological or artificial, is about more than just the bones.