The Code Ran. The Science Was Wrong.

There is a claim going around that entry-level research and data scientist jobs are gone because AI writes the code now. The usual counterargument is that AI makes coding faster and makes the fundamentals worth more, because the job moves from writing code to judging it.

I agree with that. I went back through my own recent work to check whether I actually believe it, or just like how it sounds.

I work with brain images, EEG, and clinical trial data, and people in my corner are asking the same question in their own words. Is there still a reason to learn how an image file stores left and right, or what a Fourier transform does to your signal, if a model will write the pipeline in an afternoon?

What AI did for me

I'll start with the good news, because it is real.

I am a research scientist at the intersection of neuroscience and engineeirng, not a software engineer. For most of my career I was the only engineer in a room of clinicians and physicians, and code was the thing I did at night so the science could happen during the day. AI changed that arithmetic, and I am not going to pretend otherwise.

But each of these went fast. I already knew what the right answer was supposed to look like.

What AI got wrong

Here are some of the things that went wrong in my own work this year. None of them crashed. That is the whole point.

The figure that made up its own data. Earlier this year I built a deep learning model to predict brain aging from more than a hundred behavioral measures. I wrote about it here: it did not work. Predictions were no better than chance, and the permutation test agreed.

The figure code did not agree. It produced a beautiful actual-versus-predicted scatter plot with an R² of about 0.99. When I looked at how it was made, the explanation was sitting in a comment that said, more or less, "for demonstration, create the actual values from the predictions." The "actual" values were the predictions plus a little random noise. Of course they lined up. The real held-out values had never been saved, so instead of failing, the code invented something plausible and labeled it ready for publication.

The subjects that got shuffled. In the same project, results came out of cross-validation in one order and were matched against the participant table in a different order. So every person's model output was being correlated with a different person's behavior. It is the oldest bug in data analysis: two tables joined on the wrong key. The code ran, the correlations came out, and they meant nothing.

The lesion on the wrong side. For a tool that maps stroke lesions onto brain atlases, I had AI write a detailed plan, including the code, before anything ran. The review afterward found eleven defects in it. A few of them:

A lesion entirely in the right hemisphere came back as "no lateralized signal," when the correct answer is fully right-lateralized. The logic only ever started from left-hemisphere regions, so right-only damage was never visited.
A threshold of zero, which is a natural thing for a user to type if they "don't want a threshold," selected every voxel. An empty statistical map reported 100% activation in every region, with no error.
A 4D file, routine output from standard neuroimaging software, was silently cut down to its first volume, and the record of what was analyzed described the shortened file as if it were the real one.

The tests were worse than the bugs, because they would have hidden them. Every test case used the simplest possible geometry, where a transposed matrix and the correct one give the same answer. None of them used the left-right convention that the real atlases actually use. So nothing in the plan could have caught a left-right flip, which in this field is the single worst wrong answer you can give. Telling a clinician that a stroke is in the wrong hemisphere is not a rounding error. Every test would have passed.

Real words, reported as made-up words. In my work on damaging parts of an LLM and scoring its naming errors the way we score people with aphasia, one step ran the error classifier before the word lists it depends on were loaded. With no dictionary, nothing counts as a real word, so every correct answer came back as a neologism. A sudden jump in neologisms is exactly the kind of result I would get excited about and write up.

What actually caught them

Not careful line-by-line reading. I don't read every line, and I doubt many people do anymore. What caught these was knowing what the answer should be.

I knew the brain-aging model was null because I had run the permutation test. An R² of 0.99 from a null model is not an acceptable result. It is an impossible one.

I knew to distrust left and right because anyone who spends time with neuroimaging software learns that orientation is a convention stored in a file header, and that conventions disagree. I knew a model that calls a picture of a cat "dog" is not producing neologisms. And on the clinical trial side, the AI wrote the data-quality check in minutes. Deciding what it should count as "missing" took knowing that a speech-language pathologist often starts a form in the morning and finishes it after the last session of the day. Counting only finished forms cut our false alarms by about 70%. The code had been correct the whole time. The question it was answering was wrong.

So if I write down the fundamentals for my field, the list looks something like this:

image geometry: voxel versus world coordinates, orientation, what resampling does to a mask
signal processing: sampling, filtering, what a frequency transform assumes about your data
the statistics of prediction: cross-validation, leakage, permutation tests, multiple comparisons
what the measure means clinically, in the person it came from
enough version control and testing to see what changed and to prove a result is wrong

There is one habit I'd add that I don't see discussed enough. Test against an answer you worked out without the code. Compute the expected value by hand, from the formula, and then check the code against it. Better yet, break the code on purpose and make sure the test fails. Run the pipeline on synthetic data where you planted the answer before it ever sees a real recording. If the expected value in your test came from running the code you are testing, you have not tested anything. You have written down what it currently does.

The part I can't answer

This is the question I keep coming back to. If research labs and companies stop hiring junior people, where do the experienced ones come from in ten years?

I learned signal processing by spending months of my master's with neural signals recroded by EEG, which looked nothing like the textbook: noisy, full of artifacts, wrong in ways nobody had warned me about. Nobody handed me a working pipeline. Those months are why I don't trust a beautiful result, and they are most of the reason the bugs above got caught.

A student today can get a working pipeline in an afternoon. I'm not sure when they get their bad months.

What I tell the students and early-career researchers I work with now is not "don't use AI." It is this. Before you point it at real data, make it reproduce an answer you already know. When a result looks very good, get more suspicious, not less. And try to explain each step out loud to someone outside your field. When I had to explain time-frequency analysis to clinicians, I found the gaps in my own understanding that I had been covering with equations. A pipeline you can't explain is one you can't check.

To the people who run labs and companies: every bug in this post was found because someone who knew the domain was looking. That person was once a junior who made the same mistakes with supervision. If we stop training them because the code writes itself, we are borrowing against 2036.

I don't think the field is doomed. I think the bar moved. Writing the code used to be the proof that you understood the analysis. Now it is not proof of anything, and the understanding has to be shown in some other ways.

That 0.99 scatter plot is still saved on my computer. I'm keeping it.