I was preparing for a panel. Financial services, people, AI. One question was what people still need to learn when AI can do the work.
My answer was a chain. Understand the process. Then you know what to ask. Then you know when to ask. What to ask came from V. Anantha Nageswaran, India’s chief economic advisor, who spoke at the IMF meetings in Bangkok this month.1 He said teachers must teach children to ask the right question.
Then I heard that someone I work with had used a tool called Reverse Engineer Anything. You give AI something that already exists. A running system. A product. Old code with no documents and nobody left who wrote it. AI works out how it was made.
My first thought was that my answer had just broken. If AI can recover the process from the thing itself, why would anyone spend time understanding it? Everything becomes result oriented. The path becomes predictable. Maybe we don’t even want the process, if AI can do it end to end.
This is not coming soon. It is happening now. So I tested the chain against it.
This is part of a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe to get each piece as it goes out.
The why is a guess
The normal path goes forward. You say what you want. AI builds it and writes the requirements along the way. Someone reads them, or nobody does.
Reverse engineering goes backward. You start from the result and AI finds the steps. This is new, and it removes an old excuse. We used to say understanding leaves the company with the person who built the system. Now AI can bring a lot of it back.
But only the how. AI can tell you which part calls which, which rule fires when, how each edge case was handled. Ask it why, and it will give you an answer too. I can ask AI why I asked my own questions, and it will list reasons. Some of them will even be right.
It is still an assumption. A well written one. Somebody has to confirm it, and that is the part AI cannot do for you. In Bangkok, Nageswaran said he is “against technology that takes away human agency.” This is exactly where agency lives. Not in finding the reason. In checking it.
The why also does not stop at code. Why did this team resist the last system. Why did that person leave. What did the customer actually mean when they asked for this field. AI can guess all of these. The only way to confirm a why about people is to be with the people, long enough that they tell you the real reason. That takes trust.
The formula that runs on its own
Back to the chain. What to ask can become a formula. A checklist. Someone writes the right questions once and everyone copies them. When to ask can become a formula too. A trigger. A threshold. An alert that fires when a number moves.
The first person who builds the system sets those up. Then they keep running, indefinitely. Nobody after them has to know why the threshold sits where it sits.
So where is the human? I see two places. The first is questioning the formula itself when the world changes. A new fraud pattern. A new regulation. A customer nobody planned for. The formula cannot notice it has become wrong. The second is the signature. When the regulator or the auditor asks, someone has to answer, and the answer they get will be one AI prepared.
This whole book says the human is always the variable. In this picture the variable got frozen at setup.
People still argue. Put all the data into AI and it can justify any decision, better than any CEO. That is true. It can also justify the opposite decision just as well. When justification is that cheap, it stops being worth much. Someone still has to choose and live with it.
And the data is never all the data. The engineer who felt for weeks that a design was wrong was not in any system.
Two companies
Imagine two companies with the same old system. Twenty years of rules, half of them written down nowhere.
The first company runs reverse engineering on it. AI produces the documents, the process maps, the manual, the test plan. It tells the team what to build and where to test. The team follows. Some steps by hand, some with another AI. The new system handles every edge case exactly the way the old one did. It ships. Users are happy. Management is happy.
The second company has a manager with one rule. You build only when you understand the process. So it moves slower. Much slower. On every progress report, the first company looks better.
Then something changes. A new product, a new rule, a new partner. The first company asks AI again. Where does this touch? What has to change? AI answers. The team clicks yes, yes, yes. And it works. Almost every time.
The first company is not stupid. It is fast and mostly right. That is why this is hard to argue.
But every yes approves a guess nobody inside can check. When one of those guesses is wrong, someone outside finds it. A customer. An auditor. A regulator. The second company finds its problems early, from inside. The first finds them late, from outside. That kind of failure is rare, which is why it never shows up on a progress report.
The first company also did not transform. It copied the old company onto new screens. That is a migration.
The manager’s rule is not against reverse engineering. Used well, it can make understanding much faster. The rule is against building before you understand.
What to call it
This is not only technical debt. The code can be perfectly clean. It is not quite cognitive debt either. Cognitive debt is what happens to one person’s thinking when they lean on AI too long.2
I call it understanding debt. The gap between what your organization runs and what anyone in it can actually explain.
The closest name in the research is epistemic debt. In Nature Machine Intelligence this year, three researchers in Barcelona described it as what happens when people use AI for thinking work without taking the time to understand the result.3 The distance grows between the work they show and what they can actually stand behind, and it grows faster as more work is built on top. That is close. Their focus is the person presenting the work. Mine is what happens inside an organization. AI gives you a why, and nobody confirms it. The first builder’s formulas keep running after everyone has forgotten why they were set. And at the end there is a signature with no understanding behind it.
There is a whole family of these debts now. Verification debt, when AI produces faster than anyone can review. Comprehension debt, when the code grows faster than anyone understands it.4 All of them are real. They also sit in different places, and that is how I ended up drawing a staircase.
The staircase
Building a product has steps. Each step can be handed to AI, and each handover borrows something. At the bottom the debt is quiet. At the top the bills arrive.
Step 1, understand the need. AI writes the why and nobody checks it. That is where understanding debt starts, and the organization carries it, because everything above is built on it. Pay as you go: write the why first, in your own words, and confirm it with the person who needs the thing. Then ask AI to question it.
Step 2, decide the design. AI picks the design and you nod. That is cognitive debt. It sits with the person who stopped deciding. Pay as you go: write your own option first. Then compare, and ask AI for the case against yours.
Step 3, build. AI writes the code and nobody reads it. Code debt, technical and comprehension both. The team carries it. Pay as you go: read before you merge. Keep changes small. Ask AI to explain the tradeoffs, not only to produce the code.
Step 4, verify. AI tests what AI wrote, and everything is green. Trust debt. Other people pay this one, usually the ones downstream who believed the green. Pay as you go: test against what the person needed, not against what was built. Someone who did not build it checks it.
Step 5, change. Something moves and the team clicks yes. This is the first bill, and it comes from inside. Interest grows with every change built on a guess. Pay as you go: name what the change touches. If you cannot say in one line what else it affects, stop.
Step 6, defend. The auditor or the regulator asks why. This is the second bill, from outside. You sign or you default. Pay as you go: explain it unaided. If you cannot, don’t sign.
AI is still there at steps 5 and 6, and people will use it. The old debts come due there, and AI lets you pay them by borrowing more. Ask AI to fix what AI built, then ask again next time. In finance that is called refinancing, and the finance words fit almost exactly. The principal is what you skipped at each step. The interest grows with every change. The due dates are the two bills at the top. Default is when nobody can explain and someone outside finds it first. Paying down is the slow work: asking, confirming the why, being there. Trust works like a credit rating. When people trust you, they tell you early, and you pay before the due date.
And you are not borrowing from AI. You are borrowing from the people who come after you, and they never agreed to lend.
Sometimes the person who comes after you is you. I tell my team this. The debt you create here, you will pay in your next career, like a sin. So try to make good sin. Then in your next job you live from the interest, instead of paying back the debt.
There is also a line that runs back down the staircase. Step 6 asks the why that step 1 should have kept. The same question, asked by someone else, later, with consequences.
Human agency
Under the staircase are two lines for human agency. On the first path you ask AI. You use it to question you, to explain, to show you what you missed. That line stays level. On the second path you hand the work over. That line falls, slowly at first, then steeply at the top where the bills are.
The gap between the two lines is the size of the debt.
There is evidence for the two paths from Anthropic, the company that makes the AI I use most. In a study published in January, 52 mostly junior developers learned a new programming library and then took a quiz on it.5The ones who handed the coding to AI scored under 40 percent. The ones who used AI to ask questions and understand what they were doing scored 65 percent or more.
How big the debt is at one step depends on two things. How much you handed over, and how much sits on top of it. That is why step 1 is the most expensive. A crack in the foundation is under everything.
The muscle
While I was building this, I noticed something about myself.
I asked AI to find the dots and connect them into a picture. I do that a lot. When the picture came back, I could tell whether it was good. Whether it would be easy for someone else to understand. But I had not connected it myself. I am losing the skill to connect them in the first place.
That is the debt I am creating. Step 2, in my own head.
I can still pay mine back, because I built the muscle before AI was there to do the connecting for me. Someone who starts their working life with AI may never build it at all. They get the debt without the muscle to pay it, and they cannot notice losing what they never had.
I know the better way is paper. Sit down, write it by hand, let the slow middle happen. I don’t have that time or that space. So I think in my head, which is worse than writing. Then I write by talking to AI, which is worse than writing on my own paper. The best I can do is give the rough shape in my own words first, before I ask AI for anything. This article started that way.
The chain from the start of this piece still holds, even for me. Understand the process. Know what to ask. Know when to ask. It only needs one more link. “Own it.”
Owning it is the signature. That is your human agency, and it is the one thing AI cannot borrow for you.
Nageswaran ended his talk with an image from an old Indian scripture. A chariot. The horses are the power that pulls us forward. The driver is our own thinking, choosing where to go. AI is a new horse, a very strong one, and we should welcome it. But without a driver, no horse takes you where you want to go.
The real risk, he said, is not that machines will start to think. It is that we will stop.
That’s the whole of this one. It belongs to a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe and I’ll send the next piece when it’s ready.
If it gave you something, pass it to someone.
I’m BØY (Chaiharan), based in Bangkok. I’ve spent 30 years in tech, building products, cleaning up disasters, and turning troubled projects around. Those years taught me to watch the people more than the technology. I write about what AI reveals about the humans who use it.
V. Anantha Nageswaran was a discussant at the 63rd Per Jacobsson Foundation Lecture, held in Bangkok on 17 October 2026 during the IMF and World Bank Annual Meetings, responding to Anthropic co-founder Jack Clark’s lecture “Inside the Event Horizon.” Event page. His remarks in print: Swarajya.
Kosmyna et al., MIT Media Lab, “Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task” (2025). arXiv.
Unceta, Subías-Beltrán and Pujol, “The epistemic debt of generative AI,” Nature Machine Intelligence 8, 1328 to 1330 (2026). Nature.
On verification debt: Sonar. On comprehension debt: O’Reilly Radar.