What I Could Do to a Machine That I'd Never Do to a Brain

For most of my research life, the clearest way I knew to learn what a part of the brain does was to wait for it to be gone.

That is the strange logic at the center of a lot of neuroscience. If you want to know what a region is for, one of the sharpest answers comes from people who have lost it. A stroke, an injury, an illness: something takes a piece of the brain offline, and the gap it leaves behind tells you what that piece was holding up. It is a powerful idea. It is also a cruel one, because you never choose it. You wait for it to happen to someone, and then you study what remains.

This year I started studying language models, the systems behind the AI tools everyone is suddenly talking about. And I ran into a feeling I did not expect. For the first time, I got to choose.

Learning from what is missing

I should explain the habit of mind I brought with me. A lot of what we know about the brain, we learned backward, from absence. The patient who can no longer find words. The person who sees perfectly but cannot recognize a familiar face. Function revealed by its failure.

This way of thinking gets into you. After enough years you stop asking only "how does this work" and start asking "what would have to break for this to stop working." You learn to read a system by its damage.

When I turned that same instinct toward language models, the question came out almost unchanged. Not just how does this thing work, but what is each part holding up, and what happens when it is no longer there.

The freedom I was not ready for

Here is what surprised me. A model is not a person. It is a system I can open all the way up.

In the clinic, the damage is given to you. It is rare, it lands wherever it happens to land, it cannot be repeated, and it can never be undone. You take what nature offers and you are grateful for it, even while you wish the circumstances were anything else.

A model is the opposite in every direction. I can choose exactly what to take offline. I can choose the moment, the input, the piece. I can measure what happens, then put everything back the way it was and try again, slightly differently, a thousand times if I want to. Nothing is irreversible. Nothing is rare. I can run the experiment that, in a human being, I would never be allowed to run and would never want to.

The first time I really sat with that, it stopped me. The method I had only ever applied through misfortune, I could now apply on purpose, with control, with repetition, cleanly. The thing that in a person is a tragedy is, in a model, an experiment I can rerun before lunch.

The honest part

I want to be careful here, because this is exactly where it would be easy to fool myself.

A model is not a brain. It is tempting, especially for someone trained the way I was, to let the analogy do more work than it has earned. The words line up so nicely. Parts, functions, damage, loss. But lining up nicely is not the same as being the same thing, and the very freedom that makes a model so easy to study is what makes it so easy to over-read. When you can take anything apart at will, you can talk yourself into almost any tidy story about what you find.

So I try to hold two things at once. The control is real, and it is a gift. The control is also seductive, and a clean result inside a system that people built is not, on its own, a claim about a mind, or a brain, or a person. I am mapping a machine. That is worth doing. It is not permission to forget what it is.

There is something humbling in the comparison running the other way, too. The messiness I used to resent in human studies, the rarity, the noise, the fact that you cannot rerun anything, is part of why those questions are hard, and part of why the answers, when they finally come, mean something. A perfectly controlled system is easier to study. It is not automatically closer to the truth.

What I think I am really doing

I have come to think the gift is not that an AI is a brain. It plainly is not. The gift is that it is a place to rehearse the questions.

For years I could only ask my favorite question, what does this part do, in the slow, patient, opportunistic way that brains force on you. Now I can ask it directly, watch the whole thing from beginning to end, and ask it again. That does not turn the machine into a mind. But it lets me see what a clean version of the question even looks like, and it sends me back to the harder, realer version with more discipline and a lot more respect for how difficult it was the whole time.

I did not expect a language model to teach me anything about studying brains. It turns out what it taught me was mostly about the studying.

This is a personal reflection on how my research has shifted lately. The specifics of the work itself will come later, once it is ready to share.