In Search of the Meta-Modeling Process
A search for the computational process behind model creation: how observations, purposes, values, and abstractions become reusable reasoning programs—and perhaps the primitive underlying creativity itself.
For the last several years, I’ve been trying to teach AI to do something that I haven’t been able to fully explain, even to myself.
I call it meta-modeling.
Ironically, the difficulty of explaining it is precisely the problem I’m trying to solve.
This essay isn’t a proof. It isn’t even a finished theory. It’s a research journal—a map of the territory I’ve explored, the pieces I believe I’ve found, and the missing piece that continues to elude me.
I don’t think the missing piece is another larger language model.
I think it’s a process.
What is meta-modeling?
Every discipline has structure.
Science has the scientific method.
Mathematics has proofs.
Economics has supply and demand.
Software engineering has architectures, design patterns, and algorithms.
Someone had to invent these.
Someone had to discover where one concept ended and another began.
Someone had to decide that these observations belonged together under a new abstraction.
That activity—the process that creates the model before the model exists—is what I mean by meta-modeling.
A useful analogy is map making.
Imagine walking into completely unexplored territory.
Before you begin, the land could contain anything.
Mountains.
Rivers.
Cities.
Nothing.
Everything is possible.
As you observe, the infinite possibility space begins collapsing.
You notice patterns.
You measure.
You compare.
You identify relationships.
Eventually you step back and realize:
“This isn’t just a collection of observations. It’s a forest.”
Or:
“This is a market.”
Or:
“This is natural selection.”
A new category is born.
A new compression appears.
The map becomes simpler than the territory while preserving what matters.
That act is meta-modeling.
Observation is constraint
I increasingly think observation itself is a computational operation.
Before observation there exists an effectively unbounded possibility space.
Observation constrains it.
Every observation rules out possibilities.
More importantly, the constraints themselves have structure.
The process isn’t simply collecting facts.
It is discovering which constraints matter.
When studying a flower, I don’t merely record its color.
I begin asking questions.
Why does it grow here?
What surrounds it?
What predators influence it?
How does rainfall matter?
What causal relationships shape its existence?
The quality of the eventual model depends on the quality of these observations.
Meta-modeling is therefore not passive observation.
It is active exploration of structure.
Models require purpose
Observation alone is insufficient.
Every model exists for a reason.
Maps illustrate this beautifully.
A road map is not a topographical map.
Neither is a political map.
None of them are wrong.
They simply optimize for different purposes.
The same reality can produce infinitely many representations.
Consider boiling water.
Why is it boiling?
One explanation says:
The heating element transfers energy into the water through molecular interactions.
Another says:
Someone wanted ramen.
Both explanations describe exactly the same physical event.
The observations haven’t changed.
Only the purpose has.
This realization became important for me.
Not only are there infinitely many observations.
There are infinitely many purposes.
Both sides of the modeling process are effectively open-ended.
The bridge
This led me to what I currently believe is the central structure.
Every model is a bridge.
On one side:
Observations.
On the other:
Intent.
The bridge between them is the abstraction.
It compresses the observations into a representation useful for achieving the intent.
This bridge is what makes information meaningful.
Not all bridges are equally good.
Some explain more with fewer concepts.
Some compress better.
Some transfer more effectively into other minds.
Some survive criticism longer.
Others collapse almost immediately.
The entire activity of sense-making is the construction and evaluation of these bridges.
Values constrain purposes
There is another layer.
If purposes are infinite, how do we choose between them?
Values.
Values constrain which purposes are even worth pursuing.
A business can make money through fraud.
Or by creating value.
Our ethical values eliminate entire regions of the possibility space before modeling even begins.
Values therefore shape intent.
Intent shapes representation.
Representation shapes action.
Meta-modeling operates across all three.
Why current AI falls short
For years I’ve tried teaching language models to perform this process.
One experiment was built around an intentionally underspecified request.
Make me money.
Humans immediately recognize how incomplete this request is.
Who is asking?
How much money?
By when?
Under what ethical constraints?
What skills exist?
What resources exist?
What country?
What decade?
What opportunities?
The request explodes into an enormous modeling exercise.
Current LLMs rarely do this.
Instead they tend to answer immediately.
The model jumps directly toward execution before constructing an adequate representation of the problem.
This isn’t a capability problem.
It is a modeling problem.
Execution is downstream of representation.
The representation comes first.
Neuro-symbolic AI is pointing in the right direction
The closest architecture I’ve found is neuro-symbolic AI.
The probabilistic side performs observation.
The symbolic side provides structure.
Together they resemble the two halves of meta-modeling.
I’ve built multiple neuro-symbolic systems myself.
Harmoniq Work.
Refine.
Others.
They work surprisingly well.
But every one of them shares the same limitation.
The symbolic structure was created by me.
The AI merely operated inside it.
Today’s agent frameworks do much the same thing.
The harness is fixed.
The model fills in the gaps.
What interests me is the level above the harness.
Can the system create the harness itself?
Can it revise its own representational structures?
Can it invent better abstractions?
Philosophy before verification
This is where I increasingly believe philosophy becomes indispensable.
Modern AI research understandably focuses on verification.
Benchmarks.
Evaluation.
Ground truth.
Empirical performance.
These are enormously valuable.
But they occur after the model already exists.
Before verification comes concept formation.
Before experiments comes ontology.
Before mathematics comes representation.
Before algorithms comes philosophy.
Karl Popper’s framework of conjectures and refutations has become increasingly influential in my thinking.
In an effectively infinite possibility space, certainty is unattainable.
Progress comes through proposing better explanations and criticizing them relentlessly.
The question therefore shifts.
How do we generate better explanations?
That question is exactly the meta-modeling problem.
Creativity
Here is my boldest conjecture.
I’m least certain of it.
But it increasingly feels true.
Creativity is the process of constructing new bridges between observations and intentions.
Once a system can freely change its intentions while generating new abstractions that effectively connect those intentions to observations, it has become creative.
Creativity is not randomness.
It is not novelty for its own sake.
It is successful bridge construction.
If this conjecture is correct, then meta-modeling is the computational process underlying creativity itself.
Programs instead of workflows
One idea I’ve become increasingly interested in is thinking in terms of programs rather than workflows.
Not programs as code.
Programs as executable reasoning structures.
Every scientific discipline can be viewed this way.
Logic.
Mathematics.
Physics.
Economics.
These are reusable reasoning programs built over centuries.
When encountering a new problem, humans don’t invent reasoning from scratch.
They retrieve and adapt existing programs.
Sometimes they compose several.
Sometimes they invent entirely new ones.
The important point is that these reasoning programs themselves become reusable bridges.
Future problems can retrieve them, adapt them, criticize them, and improve them.
Eventually a library of reasoning programs emerges.
Not answers.
Ways of thinking.
The primitive I’m searching for
This leaves me with one unanswered question.
What is the primitive operation behind all of this?
I can tell an AI to make observations.
That is not the same as operationalizing observation.
I can tell it to build a model.
That is not the same as operationalizing model construction.
I can tell it to compare explanations.
That is not the same as implementing comparison itself.
Somewhere beneath all of these lies a smaller computational process.
Something recursive.
Something reusable.
A primitive that generates meta-models the way meta-models generate models.
I suspect it exists.
I suspect it is computational.
I suspect today’s language models already possess nearly all of the knowledge required to execute it.
What they lack is the process.
Why this matters
If this primitive exists, I believe it changes how we think about intelligence.
Today’s AI systems execute remarkably well inside existing representations.
The frontier is no longer execution.
The frontier is representation itself.
The ability to construct better models.
Better abstractions.
Better bridges.
Perhaps this process becomes the missing ingredient for recursive scientific discovery.
Perhaps it becomes the missing ingredient for recursive self-improvement.
Perhaps it becomes the algorithm underlying creativity itself.
I don’t know.
That remains a conjecture.
But until I either discover the primitive or prove that it cannot exist, I don’t think I’ll be satisfied.
That has become the research.