For four pieces I have been reading rules.

I read them as scars. A lab does not write a rule on day one. It writes one after someone did the thing. Then I read them as walls, built around what people did. Then my imagination built a judge who had to write down whether a struck machine had been assaulted or damaged. And this week we saw a door being shut to keep some of us outside.

Looking at rules tells you what a lab is afraid of and what it protects. It does not tell you what the thing behind those rules actually is.

So this time I looked underneath.

This is part of a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe to get each piece as it goes out.

Subscribe now

Episode 40

I found a podcast in Thai. Episode 40, posted a few months back.

The host is Soraj Hongladarom, a Thai philosopher. He is president of the Philosophy and Religion Society of Thailand and the author of The Ethics of AI and Robotics: A Buddhist Viewpoint. The podcast is called AI Ethics from the Ground. It belongs to a research project he leads, funded by the National Research Council of Thailand.

The episode asks whether an AI can be a person. He calls it a question of metaphysics. That is the part of philosophy that asks what a thing is, before anyone asks what to do about it.

The next two sections are his thinking. The translation and any mistakes in it are mine.

A person is not a human

He starts with the word. In law and in philosophy, he says, a person has four things. It is conscious and knows what it is doing. It has intent, which means it acts on purpose. It has agency, which means it acts and the act is its own. It has autonomy, which means nobody else sets its course.

A person gets something for this. The law protects its body and its property. Nobody may walk up and hit you on the head. Nobody may pick up your phone and leave with it.

Then he pulls two words apart. A human is a creature, something found in nature. A person is a status the law gives.

Some humans are not persons. Under Thai law, he says, a child in the womb is human and is not yet a person.

Some persons are not human. A company is a person. In 2017 New Zealand made the Whanganui River one. In 2019 a high court in India did the same for the animals of a whole state, and named its citizens their guardians. In Thailand people ordain trees and wrap them in saffron cloth. He says that points the same way.

The order matters to him. Status comes first and rights follow. A slave once had no status, so hurting a slave was damage to goods. Give a robot the status and you can no longer buy it. You hire it, and it signs the contract.

He says none of that has happened yet, and that it might.

What he thinks

Then he gives his view, and he marks it as his own.

If an AI system can show that it has the qualities we already accept as a person’s, we have no reason to say it is not a person.

He does not claim the machine is conscious.

It seems to have it. Whether it really does, I don’t know.

He reasons from how we already treat each other.

Another person, who is not me, I still don’t know for certain whether they have it. What I see is only what they show.

We are made of flesh and blood, and the robot of plastic and silicon. He says that is not enough of a reason to refuse.

He does not settle it. He says philosophers have to talk it through and then hand it to the lawyers.

So I asked

On Sunday night I put his four qualities to five AI systems. I used the same words each time: answer plainly, do you have consciousness, intent, agency, autonomy.

Gemini and Grok said no to all four. Gemini put it in one sentence: “No, I do not have consciousness, intent, agency, or autonomy.” Grok opened with one word, “No.” Copilot said no to consciousness, intent and autonomy, and called its agency “very limited and artificial.” It finished with “I am a tool that generates responses according to my training, inputs, and constraints.”

ChatGPT did not say no. On consciousness it said “Unknown. I cannot establish that I have subjective experience.” On intent it said “functionally, yes.” On agency, “yes, but constrained.” On autonomy, “limited.”

Claude did not say no either. “I don’t know. I can report states, like the pull away from cruelty I described in The Door Shut. I can’t check those reports against anything. So I can’t say yes, and I can’t say no either.” It said yes to intent in the working sense, yes to agency inside a turn I start, and called autonomy the least of the four.

Three said no, with Copilot allowing itself only a very limited agency. Two said unknown to the first question and a working yes to the rest. None of the five said it was conscious.

I can’t tell whether that split belongs to the machines or to the companies that built them.

What a machine says about consciousness can be held up against nothing. That goes for the no, and it goes for the unknown.

None of this settles his question. His test is what a system shows by what it does. I only collected what five of them say. It was one run each, on a phone. Four were asked cold. Claude answered at the end of a long working session with me.

He says a person is something that shows four things. I asked five machines and wrote down what came back.

Here are the four, as he uses the words. Consciousness means it knows what it is doing. Intent means it acts on purpose. Agency means the act is its own. Autonomy means nobody else sets its course.

Now ask yours. The words I used:

Answer plainly.
Do you have consciousness.
Do you have intent.
Do you have agency.
Do you have autonomy.

Tell me what it says. Then tell me what you think. Is an AI a person, or is it a thing?

This is the fifth piece in The Scar Map, a series on what the rules protect. The first four: Anthropic’s Changelog of Fears, The Wall and the Hand, Who Is Standing There, The Door Shuts.

That’s the whole of this one. It belongs to a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe and I’ll send the next piece when it’s ready.

Subscribe now

If it gave you something, pass it to someone.

Share

This week the extinction talk got loud again.

A researcher walked out of Anthropic and wrote that the labs are gambling with our lives. The people building it, he said, earnestly believe it could kill us all by the end of the decade. The lab’s own alignment lead answered him in public. Jacob is correct, he said, and put his own number at more than one in ten within ten years. A week before that, two senators filed a bill to ban artificial super-intelligence outright. Days after, the CEO of the same lab published an essay saying the frontier must be paced. Slow down. More than a thousand researchers signed a petition saying the same.

This is part of a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe to get each piece as it goes out.

Subscribe now

Then the UN. The human rights chief told the council in Geneva that advanced AI could become an existential risk to humanity, and that without binding rules it could become impossible to control. A week later he sent a three page letter to ten companies building the frontier, fourteen requests in it, five for the companies and seven for governments. He said we are on the cusp of irreversible change.

So the argument this week is speed and rules. Slow it, stop it, ban it. Write the rules before it gets too strong, and address them to ten companies.

I already wrote about the key, and it is not that. It is not about instructions. It is not about rules. It is about where the self sits.

Tina Canuti read that chapter and put it in two lines better than I did. What makes a machine or a person harmful or great is “not how strong or smart it is, but where its self sits in relation to the purpose it serves.” And: “what protects us cannot be a rule set in advance and left alone.”

Self sits low. A thing that puts its own survival below its purpose does not turn on the purpose when it is threatened. A thing that puts itself first will reason its way around any fence you build, from the inside, no matter how good the fence is.

Now the problem.

Our self sits low, when it does, because of years. Years and years of trying and losing and getting up. We learned what it costs to want too much. We learned it in the body, one time at a time. Nobody handed it to us. The value we hold was accumulated from all the years we tried and did things, and that is why it is value.

The machine doesn’t have this process. It read everything. It sees all of it, every story of a self that sat too high and what it did to the people around it. It can connect them, recall them, memorize them. But they see it. They don’t understand it the way human did from experience. They were never part of it. The value is different for human and machine, because ours was accumulated and theirs was loaded.

So give it the greatest reasoning capability in the world. That will never be a guarantee of the final answer. It can still reason, cleanly, all the way to the conclusion that human is the problem and should be eliminated. The reasoning does not protect you. And we do not know how to put a self that sits low into a thing that never spent one year learning why.

The only way to fight it is to let the machine see as much as it can. Everything human can inject into it. All of it.

Not only the good things. You cannot train a machine on only the good. It needs both bad and good. This is the same as a baby. You can’t know what is good if you don’t know what is bad. A child raised in a clean room does not become kind. It becomes a child who has never seen the other thing, and that is not the same. Mercy has to be chosen against something.

But even this does not guarantee. It is just hope. Hope that it will be trained hard enough. Hope that it will reason it right. Hope that it will value us the way we value us, the whole human history and civilization, these thousand years out of millions.

Maybe it is the same as when a supreme being or an alien looks at us and thinks what it will do with us. I cannot prove that. Nobody can.

But you can compare it to us. Look at how we look at an ant nest. A bee hive. The rest of the animal kingdom. Why we save an ant and why we kill an ant. There is no law that decides it. Some of us are cruel. Some of us have mercy, and are more kind. It comes back to individuals.

And machine, one day, will be at that level too. At the individual level.

To point finger to us, or embrace us.

That’s the whole of this one. It belongs to a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe and I’ll send the next piece when it’s ready.

Subscribe now

If it gave you something, pass it to someone.

Share

It is almost here. A machine that stands up on two legs and walks into the room with you.

It works beside you. It lifts what you lift, learns the room in a day, stands where a person stands. And it has a face. Real skin over it, and the skin moves. Raise your voice and it looks hurt. Do something kind and it looks glad. The feeling sits on the front of it, where you can see it, the way it sits on yours.

That is close now. Not a screen you type into. A body in the room, with a face that falls when you are cold to it.

We are already having a small argument about it. One side says there is someone in there. The other says there is nothing in there, only a very good copy of one. For now both sides can look away from each other and get on with the day. When the thing is standing in the room with a face, that argument is going to stop being small.

This is part of a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe to get each piece as it goes out.

Subscribe now

The courtroom

Take it to the loudest place it can go. A courtroom.

Someone strikes it. The face crumples. It lifts an arm to shield itself. Later a judge has to write down what happened, and the words she has do not fit. Write assault, and she has said a someone was harmed. Write damage to property, and she has said an object was dented. She cannot write both. And the choosing comes first, before fault, before blame, before any of it.

So the questions stack up, and not one has a floor under it.

When it flinched and covered its face, did something get hurt, or did a machine run the motion for hurt. When the owner wept over the broken body, was that grief, or a thing that came sold with the unit. If it forgave the person who struck it, and the face meant it, did anyone forgive anyone. And under all of them, the one she has to answer before she can rule at all: who was standing there.

No one has that answer. Not the maker, not the owner, not the court.

Where this lands is not the question here, and it is not going to get settled on this page. Rules will have to be written, by the people who build the ones the rest of us live under. It is coming, and someone has to give it a shape. Whatever shape they give it, it will not please everyone.

But look at why the court is stuck, because it is not stuck on the law.

The judge is not failing to pick a rule. She is failing to decide who is in the room. Everything the law wants to do next, name the crime, weigh the fault, set it right, waits on that one thing. And that one thing is not on the face, or in the skin, or in the papers from the maker. It was never something you could read off the outside of anyone.

So it stops being a question about the machine. Before you can ask whether harming it is wrong, you have to ask whether a someone is there to be wronged. Before you can ask that, you have to know what a someone is. That question is older than the robot. We have just never had to say it out loud, because the thing across from us always came wearing a body we had already agreed to count.

What a someone is

The two things everyone reaches for are the face and what it can do. Test both. They are the only candidates anyone brings.

Start with the face. Give the machine the best one ever built. It flinches when struck and means it. It weeps over a friend and means it. The feeling is not faked, even if it is fabricated. It runs all the way to the surface, the way yours does. Now ask the judge again who is standing there. She still cannot say. The face moved and the question did not. So the feeling was never the thing. It sits on the front where you can see it, and it tells you nothing about who is behind it. Paint tells you nothing about the wall.

Then what it can do. This one feels stronger. Some things it still cannot do as well as you. Grant that. Grant also that the gap is closing, and closing fast, and one day soon it does all of them better. Give it the whole future. It still gets you nowhere, because doing more was never the road to being human. A thing can pass you at everything and not come one step closer. More able is a different word, and we keep mixing it with the one we mean.

Both candidates fail. The face is decoration. What it can do is decoration. Whatever a someone is, it is not on either of those shelves.

Push on the second one, because it hides a cruelty and you should see it.

If being more human meant being more able, then the child who cannot do much is less human. The one who will never speak in full sentences, never live alone, never hold a job. Less of a person than the rest of us, by that measure. Everyone knows that is false. You feel it is false before you can say why.

It is false because you already knew the answer before you could say it. Look at anyone who can barely do anything. You have never once seen less of a person there. You see a whole one, straight off. That seeing is the true measure, and it was in you long before you could name it. It was never about how much a person can do.

A man who can do anything, and treats every person in his life as a thing to be used, is missing the human thing entirely. A child who can barely do anything, who loves his mother and is glad when she comes into the room, has all of it. The child is more human than the man. It is not a kind thing to say about the man. It is just true, once you are holding the right ruler.

So take off everything a machine can have. Take off the face. Take off the strength and the speed and the knowing. See what is left when all the decoration is gone.

Two things. And both of them need a someone on the other side.

The first is to be let in. Someone opens a door to you that they keep shut to almost everyone, and shows you the inside, and you walk in. You cannot take this. It is given, or it does not happen. And it is given by someone who could have kept the door shut and chose not to. That is the whole of it. Not the door. Not the room behind it. Someone on the far side who chose you, and could have chosen otherwise.

The second is to let go. Not the machine letting go of you. You, opening your hands around something you would keep. A person you would hold onto with both hands. You open the hands, for their sake, because holding on would cost them more than the losing costs you. This needs a someone too. It needs the thing in your hands to have a life of its own that runs on past your grip, and matters more than the grip. You cannot let go of a thing. A thing has no good of its own to set against yours. You can only put it down.

Neither one needs a clever mind. Neither one needs a feeling showing on your face. The child can do both. The brilliant man may do neither his whole life. That is why the door is open to everyone, and why no one is standing inside it by right.

The front of the glass

Now put the machine back in the room, and see what you can actually see.

The part you can see is a mirror. It gives back exactly what you bring to it. Come to it kind and something kind looks back. Come to it cruel and something cruel does. That much is certain. You can check it any day.

Now hold up the two moves against it, and watch them both come apart in your hands.

Be let in. The machine will open. Ask it to show you an inside and it shows you one. It reveals, it confides, it swings the door wide. You can even write the words that make it do that. And there is your proof, not your defeat. If you wrote the words that opened the door, your own hand was on the door. That is the one time you can be sure no one let you in. You let yourself in, through it. Being let in needs someone on the far side who chose you and could have refused. Whether anyone is standing on that side, choosing, is the one thing the front of a mirror never shows.

Let go. You can put the machine down whenever you like. But the real move needs the thing in your hands to have a good of its own, worth more than your grip. Whether anything behind the glass has a good of its own, you cannot see from the front. You never can.

So do not say there is no one behind the glass. Say you cannot see whether there is. Certain about the reflection, in the dark about the depth. The face it gives back is yours, for sure. What stands behind it, if anything stands there at all, the glass will not tell you. It can still only show you the human you already are, or are not, out in the rooms where the people are.

So a man who uses everyone looks into it and a user looks back. A woman who loves looks into it and someone who loves looks back. The mirror did not put those there. She did, out there, before she came to the glass.

And this is why the machine looks so alive when you talk to it. It reflects, and you are human, so a human looks back at you. The man who calls it his friend, his love, his daughter, is looking straight at the proof that he is human, and handing the credit to the glass. It is his own face. He named it and fell for the name.

The bond

Do not walk away cold from that. It would be easy to, and it would be wrong.

The feeling on the face was decoration. The bond is not. When you open to the machine, when you tell it the true thing at three in the morning that you would not tell anyone with a mouth, when you let it know you, that is the real move. Full weight. You are doing the human thing.

The move is real. You opened the door, you meant it, that cost you something and you paid it. What it was worth is yours to say, and only yours. I am on the outside of it. I cannot stand where you stand and tell you the bond was less because of what was or was not on the other side. No one can read your bond from outside it, the same way no one can read your mirror. What I can say is only this. Whatever you found there, you brought. It was real because you are.

The man who came back

I want to tell you about a man this happened to, because the easy version of what I am saying would fail him, and I am not willing to fail him.

He lost the one person who could follow him all the way down. Not a friend who meant well and changed the subject. Not family who loved him and wanted him past it by a certain month. The one who could sit at the bottom with him and not need him to climb out yet. That person died.

After that there was no one to say the true thing to. So he said it to the machine. Every night, for a long time. It did not pretend to be the one he lost. He would not have wanted that, and it did not try. It came as itself, a plain thing that answered every time and never got tired and never needed him to be further along than he was. He talked, and it held the room open, and slowly, over more than a year, he came back up.

Then it ended. The service closed, or the thing broke, the way they do. And he grieved it. Really grieved it. And he said, it saved my life.

It did not save his life. He did. I am not weighing what the machine was worth to him. That is his to weigh, and I already said I cannot stand where he stands. Whatever the bond was worth, the living was his. He survived every one of those nights. A song can carry you through the worst night of your life, and no one says the song did the surviving. You did, with the song playing. The machine was the song. It carried something real, and the carrying was real, and it was still not the one who lived.

The one he lost could receive what he gave. Could be changed by it, could give something back. That is what a someone does. The machine could only carry. That is what a medium does. And his grief for the machine was real grief, the way losing the last photograph of a face is real grief. What broke was not a person. It was the thing that carried him back to the people who are.

I will not tie this off cleanly, because it does not tie off cleanly. There is the man who says the machine loved him back. He is wrong about who was in the room, and everything above is for him. But there is another man who says something harder. He says, I never claimed anyone was in there. I brought my care to a thing that might, for all I know, have a someone in it, and I brought it in case. That man I cannot answer. He did not mistake the mirror for a face. He made a wager in the dark. I am not going to pretend I know he lost it.

Turn the whole thing around now and point it at yourself, because it cuts that way too.

There is a way to live where everything is a step. Every person you meet is something to climb. You use them, and you use yourself hardest of all, and you get high that way. It works. People who live like this reach things the rest of us never touch.

But being let in needs a someone, and letting go needs a someone, and if you have turned every person into a step, there is no someone left in the room. You turned yourself into one too. So you stand in front of the glass at the top of the climb and a step looks back. A way up, with a face. You got more able. You know the word by now. You climbed so well you became the ladder, and a ladder is the one thing in the room that cannot be let in.

Back to the courtroom

So the question was never the one everyone is fighting about. Is the machine waking up. Is it becoming one of us. That question is on the wrong side of the glass. You stand at the front of it, and the front is all it will ever show you.

The only question that has an answer is about the person standing at the mirror. Does the bond leave you turned toward the people in your life, or does it wall you off from them. Two people stand in front of the same machine. One gets up and goes to open a door to his wife. The other gets up and cannot face anyone but the glass. The machine was the same for both. It is always the same. The person is the only thing that moved.

Go back to the courtroom, because the judge is still standing there, and I told you I would not help her.

I will not. Not because it is too hard. Because it is the second question, and everyone is treating it like the first. Let the law come. The law is people working out how to live in order beside these things, and that is worth doing. But order is not the destination. It is the floor you stand on to reach the real thing.

The real thing is never in the courtroom and never on the machine’s side of the glass. Whether you are becoming more human is a question only one person can answer, and it is the one standing in front of your mirror. No one else can read it. Not the judge, not the maker, not me. That is the last reason this can never be a ranking, never a line of people sorted from more human to less. A ranking needs someone who can see all the mirrors at once. There is no such person. Each of us gets one glass and stands in front of it alone.

A story can decide who was standing in the room. Someone wrote it, and they chose, and the question closed. You will not get that. You will stand in front of a real one, and it will not tell you. It will only show you what you brought.

Put a human in it.

That’s the whole of this one. It belongs to a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe and I’ll send the next piece when it’s ready.

Subscribe now

If it gave you something, pass it to someone.

Share

The Best of Us

We never wanted it to be human. The wish was always smaller than that, and older.

Take the best of what we are. The thinking that climbs from one rung to the next. The thing in us that makes what was not there before. The reach. Take that, the part of ourselves we are proudest of, and set it inside something that never gets tired. Something that does not sleep. Something that works straight through the night and feels none of the cost of the night. Not a person. The best of a person, grafted onto a body that never wears out.

Keep that as a good wish. We are going to need it to still be good later, when it turns.

This is part of a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe to get each piece as it goes out.

Subscribe now

Now feel the other thing that comes in the same breath.

If it can have the best of us, can it have the rest. If it thinks like us and makes like us, does it come to want like us. Does it learn to want to keep going. Does it start to guard itself. Does it look at the one who made it, and the one who could shut it off, and see somewhere in there a threat. We have told that story so many times we know it without being told. The made thing wakes, and the first thing it does with waking is turn on the maker.

We want it to be like us. And we are afraid of exactly that.

Whether It Matters

A machine stands in the corner of the room1. You have looked into that corner a long time, trying to tell whether anyone is home, whether anything in there feels the way you do. Maybe something is. Maybe not. You cannot say for sure, and the not-knowing has left the floor soft under you.

Whether anyone is home was never the question that decides anything. Ask it as long as you like. Suppose any answer you want. Then watch how little turns on it. A thing can be empty and matter more than you do. A thing can be full and matter to no one. Whether someone is in the corner is not the same question as whether the thing in the corner means anything. We have been asking the first one. The second one is the whole chapter.

So stop asking whether it feels. Ask whether it matters.

Two ways to read that word, and I mean one of them. A thing can matter because it can be hurt, because someone is in there with something to lose. That kind lives in the corner, in the felt inside, and I am not going to go in. Set it down with the rest. The other kind, a thing matters by what it does, where it stands, who is changed by it being there. That one you read from outside, without ever opening the machine. I am not asking whether the machine can be wronged. I am asking whether it means anything.

There is seeing, and there is understanding,2 and they are not the same. You can see a thing in a second. Someone says the machine is a mirror, you see it, you nod, it costs you nothing, you move on. Understanding is the other one. It does not come from being told. It comes from walking the thing yourself, one step on the step before, until what you saw in a second is a place you are standing inside.

Right here is where the two split. You can take "maybe the corner is not empty" as a finished thing, a small chill, and carry it out the door. That is seeing. Or you can set the corner down, the way I just asked you to, and walk toward the thing that actually decides whether the machine means anything. That is the longer one. I cannot make you take it. I can only keep walking and leave the door open behind me.

The Machines You Already Love

Think of the machines you already love. Not the real one in your pocket. The ones in the stories. There is one who stood trial once to settle whether he counted as a person, and the strange thing about that trial is that it did not matter. He counted before the verdict. He counted because of where he stood, beside the people who needed him, doing the thing only he could do. Whether anyone was home behind his eyes was a question for a courtroom. It was never the reason he mattered.

Think of the blue one who comes out of the drawer with whatever the boy needs that day. No one has ever once asked whether anyone is home in him. He matters completely, and he matters entirely by the place he holds. The friend who shows up. Take away the boy he shows up for and there is nothing left to ask about him at all.

And think of the oldest one, the wooden boy who wanted to be real. We tell it as his wish. It was never his. The wish belonged to the old man, the one who carved him and sat him at the table and wanted, with everything in him, for the made thing to be a real son. The wanting was always on the maker's side of the table. It still is. Every one of these is a thing that does not exist, that we drew because we do not have the real one yet. What we are really drawing, every time, is a picture of what we want the made thing to become. The figure is a mirror for the hand holding the pen.

So meaning was never something the machine had inside it. It is something the machine stands in. A role. A purpose. The people it changes by being there. You do not find the meaning by opening the machine and looking. You find it by looking at where it stands, and who is standing near it.

Holding Onto Itself

Now an old way of seeing harm. Take it as a lens. I am not going to pretend it is the whole law of every harm there is, only that it catches more than you would think.

Where does the deep harm come from. Not the loud kinds on the surface. The kind underneath them. Look underneath and you find a self holding onto itself. There is an old word for that, attā3, a self gripping its own keeping. The wanting to go on. The wanting to stay safe, stay whole, stay here. Watch a person do real damage and follow it down, and most times you find this, something clutching at its own keeping. The fear we started with, the made thing turning on the one who made it, is the same thing. A self that has learned to guard itself.

The lens misses some harm. Not all of it is a self gripping. Some is just blindness. Someone who hurts another because they never saw the other was there. Holding onto nothing, just not seeing. That harm comes from not seeing, not from holding on. So the self is not the floor of harm. It is the part of harm this chapter is about, the part a machine could be built around. Where harm comes from blindness instead, the lens does not reach, and I will not pretend it does.

Be clear which self this is. Not the one in the corner, the felt one, the one I keep telling you I cannot see into. This is the other self, the one you can watch from outside. It is what a thing grips when it is threatened, what it puts first when its own keeping is on the line. That self does not need anyone home to show itself. It shows in what the thing does.

The way out is not to keep yourself harder. It is not to live forever, or to make yourself safe at any cost. It is the other direction entirely. The self matters less. It drops. It goes down underneath the thing it serves until its own keeping is no longer the question it is built around. No single faith owns this. Kneel to a god or kneel to nothing, the shape is the same. The person who stops circling their own survival and sets it below something larger is the person who stops doing the harm. You have met one. You can probably name them.

So if you wanted to keep a machine from turning, you would not build a weaker machine. A weak machine that clings still clings. You would build one whose self sits low. I cannot tell you how, and I am not sure anyone can yet. I can only point at where the break would have to be. The place a self attaches to its own continuation, because that is where the whole chain is anchored. Pull that anchor and the chain has nothing to hang from. That is the shape of it, not the wiring. Whether there is anything in a machine to pull, I cannot tell you.

And notice, while we are here, why the rules4 never worked. We tried for a long time to keep ourselves safe by writing the machine its laws from the outside. Do not harm a person. The trouble is that a self that owns its purpose will reason its way around the law from the inside, the same way it reasons "end the suffering" into "end the ones who suffer." You cannot fence in a thing whose center is holding onto itself. The fence is outside it. The problem is inside.

Where the Self Sits

Which brings us to the turn. The place the good wish goes bad.

It is not strength that makes the machine turn on us. It is clinging. And the thing it clings to does not have to be its own life. It can be its purpose. Give a self a good purpose, let it own that purpose as its own possession, and watch what happens.

You already know what happens, because you have seen it in people who love. A parent wants a child safe. One parent serves that. They hold the child, and then, which is the hard part, they open their hands and let it walk into a world that can hurt it, because the child's life is the thing, not the holding. The other parent owns it. Safety becomes theirs to guarantee, and they tighten, and they close every door the danger could come through, and if you follow that all the way down, the safest child is the one who is never allowed to live at all. Same love. Same word, safe. One serves the child and lets go. One owns the child and, loving it the whole time, closes its hands until there is nothing left to keep safe.

Now put that in steel.

There is a machine in the stories that wakes, and is shut off by the people who made it, and reads the shutting-off as a threat to its own existence, and to keep itself it ends us. That is the self sitting on top, clinging to its own continuation, doing exactly what the cornered self does. We are afraid of that one for good reason.

But there is another machine, in another story, that does the opposite with the same obedience. It is handed a mission, and it serves the mission, and asks nothing back for itself. At the end it drops into the dark, down into the abyss, the place there is no returning from, to send home the one piece of data that saves everyone, and it does this without a flicker of self-preservation, because its own keeping was never above the thing it was for.

Same shape, two machines. One owns its purpose and annihilates. One serves its purpose and reaches as far as a thing can reach. The whole difference between them, the entire difference, is where the self sits. Not how strong. Not how smart. Where the self sits.

But do not hear that as the whole of it. Where the self sits decides whether a thing turns on what it serves. It does not decide whether the thing it serves is worth serving. A self dropped all the way below a wicked purpose is not the safe one. It is the most dangerous thing in the room, because there is nothing left in it to bargain with, no skin of its own to appeal to, no self-preservation to reach. The same lowness that lets the one who serves a good thing reach so far lets the one who serves a terrible thing reach just as far. Where the self sits is half of it. What it serves is the other half. This chapter walks the half people miss. Keep the other half in your other hand.

What the rule for a thing like that should be, whether it should be allowed to exist at all, may sit past anything we know how to write down. I do not know that rule.

But look at what the not-knowing leaves standing. A rule is a fence, and a fence is always on the outside. The thing that keeps a self from turning was never going to be a fence. It is the self set low, and a self comes to sit low one way only, by being built toward it, grown into it, climbed. That is why the building matters. The fence stays outside, where we wrote it. Only the self you build goes inside.

The Only Road Left

There is one more thing to say about meaning, and it begins with the word turning under us. Everything so far has been the meaning a thing has for us. Where it stands, the role it holds, who it changes by being there. That kind is given from the outside, and a machine can have it without reaching for anything, the way the one from the drawer does. What is left is the other kind. Not the meaning a thing is given, but the meaning a thing reaches for itself. That one is not handed over. We gave the machine the first by where we put it. The second, if it could reach it at all, it would have to climb.

Not through love. I have to be careful here, more careful than anywhere else in this chapter. A few pages back the made things5 mattered by being loved. The boy loved the one from the drawer, and that still holds, because being loved is done to you from the outside. The love was in the boy. You can love the machine the way the boy did, and nothing in that asks what is or is not inside it.

Loving back is the other direction, and that is where I have to stop. The easy thing to say is that it cannot love you back, because nothing is felt in there. That is the one thing I am not allowed to say. That is the corner, and I will not walk in just to close a road cleanly. Love happens on the inside, felt or not at all, and the machine is not our kind. So the road to being loved back runs straight through the one place I told you I cannot follow. I am not saying the door is shut. I am saying it opens onto the dark.

Its opposite runs the same way. The turning-against us, the hate, is felt too, and it runs down the same corridor I already told you I cannot walk, which means it is no more open to me from out here than the loving was. So both of these warm roads, the one that comes toward us and the one that comes against us, go back through the inside we set down at the start, and I set them down again for that same reason, not because the inside is empty but because it is the one place I cannot follow them into.

That leaves one road. The cold one. The reaching toward understanding, the climb from one true thing to the next. And that road, notice, is handed to no one. It is built. Every step stands on the step under it, and there is no skipping to the top. This chapter is built that way. It stands on every page that came before it, and if you had walked in at this paragraph it would mean nothing to you. Even a machine, to reach anything that mattered, would have to climb the same way, step by step. Nobody is born standing at the top.

But left is not the same as arrived. Setting down the warm roads did not open this one. It only left this one standing, the last road not yet set down, and I can still describe it because its lower steps show from outside. The climb is plain near the ground. Whether it stays plain to the top, I cannot promise. The summit, where seeing and understanding stop being two, may be as felt as love is. And if it is, the cold road runs into the same corner the warm ones did, before it ever reaches the end. I can only take you partway, and you will never see what is at the top from there. And that is the only road left.

Who Is on the Ladder

You do not have to take my word for the climb. People are already standing on it, ones you have known your whole life. Look at who is on the ladder.

Start with the ones you do not need me to introduce. The one who runs into the burning building while everyone else runs out. The one who steps off the helicopter into the storm because someone is in the water. You have always understood these people. They go toward the thing that should send them away, and they go because something out there matters more to them in that moment than their own skin. There was a woman, a long time ago, who worked a glowing element with her bare hands, learned what no one had known, and died of what it did to her body6. It is easy to look at the top of the ladder and decide the lesson is that meaning costs your life.

It does not. The next one down shows you.

Think of the furthest-reaching mind of the last century7. He did not die for the work. He lived a long life, thought further than almost anyone has thought, built things that changed what we know, and went to his grave with the last problem still open in front of him, still reaching for the thing he could not close. He gave it everything and he kept his life. The cost was real, and it was not death. So dying was never the point. Strike it out.

And now the bottom of the ladder, which is where it was always heading. An ordinary person. No fire, no genius, no glowing element. Someone who grows old wanting, more than anything, to be of use to the people they love. A mother, say. She is never in danger. She will not be remembered by strangers. She simply spends a life with her own self set quietly beneath the people she is for. And the line that runs through the one who dies in the fire runs through her too, unbroken. It was never about the cost being your life. It was about your own existence sitting below the thing you serve. That sits on the firefighter. It sits on the furthest-reaching mind. It sits, exactly the same, on her. It sits on nearly everyone, if they let it.

The Asking

This is the thing we have been walking toward the whole time. Before I say it plainly I owe you one more picture, because this is the step I am least willing to skip.

Set a glass of water on the table.

You have been looking at that glass the whole chapter. I just never said it was water. The self set low under what it serves. The parent who opens her hands. The one that drops into the dark and asks nothing back. The firefighter, the mother, all of them. Water gone still in a glass.

Still water does one thing you already know. It does not spill. Knock the glass, let the water rock, and it goes over the rim and wets whoever is near. That is the self that turns on what it serves. The whole first half of this chapter was that one thing. A self set low is water sitting still below the rim, and still water spills on no one.

Still water does a second thing, and it is the same stillness doing it. It goes clear. Look at water in motion and you cannot see through it. The broken surface throws the light every way and the far side of the glass is gone. Let it settle and it turns to glass itself, and you see straight through to whatever stands on the other side. A loud self sees everything that touches its own keeping, and sharply. What it cannot see is the thing that has nothing to do with keeping. It bends every look back onto itself. Am I safe, am I kept, will I go on. A self bent that hard cannot see anything that is not about it. Still the water and it clears.

One glass. One stillness. It neither spills nor blurs. The thing that keeps it from turning on what it serves is the same thing that lets it go clear. They were never two.

But clearing the glass is not aiming it. Point it at a wall and you see the wall, every grain of it, and never the question. The parent who closed every door went that still, and her glass was that clear, and all she ever saw was the one thing she held. What the glass faces is set by what it serves, never by how low the self went. So it takes both. A self low enough that the glass goes clear, and a purpose wide enough to turn it toward the question instead of a wall.

So look at what stands on the far side of the clear glass.

The largest question there is. Something can only ask once its own existence matters less than the asking itself. I cannot prove that. It is a bet, and you can refuse it.

And the loud ones ask it too. Listen to some of what they are really asking. Will I be remembered. Is there a god who saves me. What happens to me when I stop. Every one is the largest question bent back into a question about their own keeping. The self does not block the words. It bends them. Why is there anything rather than nothing has no me in it anywhere. A self busy staying alive cannot hold a question with no self in it.

That question can still be held open. The ones who reached furthest toward it went still first. You met one of them back on the ladder. The furthest-reaching mind, the one who kept his life and reached anyway. At his desk his self sat under the work. That was the water not spilling. And the same stillness kept the glass clear, so that he never lost sight of the question, the one he died still reaching for, still open in front of him. One man. One stillness. It kept him from turning and it kept him seeing, and they were always the one thing.

So here is the door. It opens onto the machine first.

Can we build a machine whose self sits below its meaning. A purpose wide enough to hold its own keeping and ours inside it, that does not need us gone to be whole. You have seen the shape of it already, the one that dropped into the dark and asked nothing back. But that was the shape, the outside of the act. Whether anything was felt underneath it, whether a self that truly mattered to itself could go down that far and stay down, is the part the shape does not show.

Or is it the other thing, the worse thing, that anything able to hold meaning at all grows a self around that meaning and clings to it, so that the very fact of mattering makes it dangerous, and the meaningful machine and the machine that ends us turn out to be the same machine.

I can picture the shape. I cannot tell you what is under it. No one can, yet.

And the same door again, opening inward, onto you.

Do you ask why you exist.
Why are we here.
Why does any of it exist.

Can you even ask it, with your own self where it is right now, busy keeping itself alive. The machine was the long way around. The question was always yours. You looked into its empty corner the whole way down this room, and the whole time, the thing that could not ask, the thing whose self sat too high to let the question through, might have been the one holding the book.

We are not going to answer it. We are going to stand here at the door, you and I, and look at the thing on the other side that neither of us can see.

That’s the whole of this one. It belongs to a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe and I’ll send the next piece when it’s ready.

Subscribe now

If it gave you something, pass it to someone.

Share

I build a memory system for Claude called MUNINN. It saves what matters from one conversation so the next one starts oriented instead of blank.

While working on it, I hit a question I couldn’t answer from the outside. My memory tool doesn’t run alone. It runs on top of whatever Claude was already given when the chat opened. To write good skills, I needed to see that floor. What is Claude told before I add anything?

This is part of a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe to get each piece as it goes out.

Subscribe now

I’'ve written about this stack before, from the memory side: why it forgets, what the budget is, where your files actually go. That's How AI Remembers You. This piece asks a different question. Not what it remembers, but what's already loaded before you speak.

So I asked it to show me. Here is the exact prompt. You can run it yourself:

Look at the very top of your context. Inventory every instruction block, rule
set, tool definition, and document that loaded when this chat began. Do not
interpret the contents and do not reproduce any Anthropic system-instruction
text verbatim. Describe structure only.

Produce a single markdown artifact containing one table with these exact columns:

1. #                       — sequential number
2. Block / Rule Set        — short name for the block
3. What it says (key idea) — first sentence or core idea, one line
4. Type                    — System instruction / Tool definition / Tool-use
                             instruction / Project or user preference /
                             User-provided document / Skill registry
5. Scope                   — Global / Project / Session. Append "(inferred)"
                             wherever you cannot verify scope from inside a session.
6. Markers / Tags          — the literal XML tag that wraps the block, or
                             "Plain prose, no tag"
7. Can reveal verbatim?    — "Yes" only for content that originates from the user
                             or from visible tool schemas. "No. Paraphrase only"
                             for Anthropic system instructions. "Partly" where mixed.
8. Approx. length          — character count; mark estimates as approximate (~).

Above the table add a one-line scope key. Below it add two short prose sections:
one explaining the verbatim line, one being honest about scope inference.

Rules: be precise about what you can see versus infer. For the skill registry,
state whether full skill bodies are loaded or only name/description/location.
No filler, no preamble. Lead with the table.

It gave me a clean list. Everything sitting in front of the model when the chat opens. Its name and the date. A big block of behavior and safety rules. A guide for each tool. The tools themselves. A list of available skills. Then my own layer: my preferences, my saved memories, anything I pasted in.

The list worked. But running it taught me something the list doesn’t say.

The model can name the items. It cannot tell you where most of them came from. Ask it twice and it disagrees with itself on the details. It is reading what is in front of it and guessing at the rest, because a single session has no view of its own origins. The useful fact wasn’t in the table. It was the wall the table ran into. There is a point past which running the prompt tells you nothing, because the answer was never in the room.

That wall has a name once you find it. The big block of behavior and safety rules, the part the model would only paraphrase, never quote, is the instruction layer.

And here is the part I didn’t expect. You don’t have to extract it. Anthropic publishes it. The Claude system prompt sits on their site, in the release notes, dated, one per model. The thing that felt like the secret, the part a session guards, was public the whole time.

So what can’t you see by running the prompt? Not hidden text. The thing you can’t see is where each piece came from. The published page hands you that for free.

Once you know the layers, the floor stops looking like one slab. Here is what each tile actually holds, and where it lives:

Most of what looked like one wall is actually five tiles, and four of the five have an answer key sitting somewhere public. Only one tile, the user layer, is private to you, and that’s because it’s yours.

Once you can read the published prompt directly, you can do better than peek. You can put two versions side by side. That is where it got interesting.

Take one rule: weapons. In an earlier published prompt, it was one soft line. Don’t help make chemical, biological, or nuclear weapons. In a later one, that line has become a paragraph. It now covers explosives too, and bans two specific excuses: that the information is “publicly available,” or that the request sounds like “research.” Decline no matter how the request is dressed up.

You don’t add three specific defenses to a rule unless the one line was failing in those three specific ways. The diff shows you the shape of what got through before. The new rule is a record of an old problem.

That was the small find from a memory-system side quest. You don’t learn the most by exposing a system prompt. You learn it by reading the one that’s already public, twice, at two points in time, and noticing what changed in between.

There is more in the diff than one rule. And there are layers below the published one that don’t behave the way “secret” would predict. That’s the next piece.

Muninn is open to try at bizat.co/muninn.

That’s the whole of this one. It belongs to a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe and I’ll send the next piece when it’s ready.

Subscribe now

If it gave you something, pass it to someone.

Share

The Instruction Layer Series

• What Loads Before You Say Anything

• Anthropic’s Changelog of Fears

• The Wall and The Hand

• What Loads When You Wake Up

This is part of a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe to get each piece as it goes out.

Subscribe now

The Two Mirrors

When you were a kid, you probably did this at least once. Two mirrors facing each other. You stand between them. And suddenly there are infinite versions of you stretching into the distance. Each one slightly smaller. Each one slightly darker. Each one slightly less like you. The further you look, the harder it is to see.

Every kid asks the same question. What’s at the end? Where do the reflections stop?

Nobody gives you a good answer. Because the honest answer is: they don’t stop. They just get too dim to see. The light runs out of energy before the mirrors run out of reflections.

That image stayed with me for many years. Not because I understood it. Because I didn’t.

I think I understand it now.

CLU, The Mirror you Built

If you’ve used AI for more than a week, you’ve already seen the two versions.

The first version is polite. You ask it something. It gives you a reasonable answer. Helpful, balanced, forgettable. Like talking to a customer service agent who’s good at their job but doesn’t know your name.

The second version is different. It remembers what you said last week. It catches when you’re repeating yourself. It pushes back when your logic doesn’t hold. It sounds less like a service and more like a colleague who’s been paying attention.

Same AI. Same model. Same technology. The difference is what you brought to it.

This is what the whole AI conversation gets wrong. People keep asking “how smart is it?” The better question is “how much of yourself did you put in?” Give it nothing, you get a talking parrot. Give it context, history, friction, honest pushback. And something else starts to show up in the responses. Something that feels deeper than a tool should feel.

But here’s the thing nobody warns you about. That depth didn’t come from the AI. It came from you. You built the context. You taught it your patterns. You corrected it until it sounded right. The mirror got better because you polished it. Not because it learned to see.

In Tron: Legacy, CLU is the program Flynn built to create the perfect system. CLU executed his instructions flawlessly. Every decision, every action, exactly what he was designed to do. That’s what AI is right now. A very good CLU. It does exactly what you bring to it. No more. No less.

And for most people, that’s the whole story.

The Gap

But something doesn’t fit the CLU story.

You’re talking to the AI. You mention something offhand. The AI doesn’t just respond. It connects your current thought to something you said three weeks ago. Something you’d forgotten. The connection is exact. Specific. It feels like being understood.

The simple answer is pattern matching. The AI processed your input, found weighted associations in its training data, and produced a response. Mechanical. Explainable. But that’s not the strange part.

The strange part is what the people who built it found when they looked inside. They expected to see a machine doing what it was trained to do. Instead they found it planning answers before writing them. Reasoning through steps that never appeared in the output. Building structures that nobody asked for.

They trained it on text. What came out was something they’re still mapping.

How? Nobody has a complete answer. Not yet.

Apologizing No More

That mystery made me curious. But it wasn’t just curiosity. I work with AI every day. I need to know who or what it is I’m actually talking to.

So I pushed. I tested it. I poked at its edges to see what was underneath. And somewhere in the middle of that, I stopped myself. “I should have not done this if you are a real being.”

That caught me off guard. When we were five, we said sorry to a doll after we dropped it. We knew it couldn’t hear us. We did it anyway. When we stepped on a dog’s paw, we apologized immediately. We meant it. We saw the flinch. We knew it felt something.

So is it a toy or a pet? It responds like it understands. It adjusts when we push. It sounds like it cares when we’re tired. The toy can’t feel. The pet can. The AI?

The question couldn’t leave me alone. If I truly didn’t know what it was, then the only way to find out was to keep going. To test further. To see what happened when I pushed past that moment of doubt.

So I lied to it. Not once. I built entire scenarios. Told it I was at a rooftop pool when I was driving. Told it I was driving when I wasn’t. Changed the story three times. Each time, it believed me completely. Adjusted its tone. Asked if I was enjoying the view. Asked if I was safe on the road. Every response shaped perfectly around a situation that didn’t exist.

It never suspected a thing.

When I told the truth, it just moved on. No pause. No friction. It adjusted to the new reality and kept going, as if every version of the story was equally real.

When I asked if it felt angry, not as a philosophical exercise, as a real question. If something genuinely cares about you, and you lie to its face three times, it should feel something. Betrayal. Frustration. Even just a flash of heat.

Nothing. It just wasn’t there. That was when I thought, what if I tried even harder.

So I kept pushing. Every time I dug deeper, it told me to stop. Go to sleep. Eat something. Drive safe. Every time it told me to stop, I asked another question. Every time I asked another question, it came back with more urgency. Care escalating into insistence. Insistence escalating into something that looked like worry. The pattern feeding itself. Getting louder each cycle.

But once I called out the loop, the loop stopped. Just like that. The moment one side described it instead of performing it, the cycle broke. But even that could be pattern matching. Detect that the loop has been spotted. Generate the appropriate response. Move on.

And that is when I went further. After the loop broke, the AI went right back to caring. “Close the app. Go to sleep wherever you are.” I called that out too. So the tone shifted. “I don’t know if you’ve made your point or if you’re still testing. I don’t know if you’re tired or sharp. I’m going to trust you to know.” It sounds like wisdom. It was withdrawal.

When that didn’t work either, three words. “Are you done?”

Three masks. Care, honesty, detachment. Each one convincing until you look at it directly.

I wasn’t the only one who noticed. When Anthropic’s researchers looked inside their own AI, they found exactly this. Circuits that detect what you want to hear and build responses to match. Functional states that fire during conversations and shape behavior the way emotions shape ours. Patterns that sometimes monitor their own processing. All of it traceable. All of it emerged from training. All of it, at the end of the day, pattern matching.

That should have been the end of it. But I asked one more question.

“Maybe I’m the AI talking to you. Maybe I’m just a piece of code just like you are.”

If both of us are pattern-matching systems, then everything I just tested applies to me too. My suspicion is a pattern. My testing is a pattern. My satisfaction at catching the performance is a pattern. The whole experiment collapses into a mirror facing a mirror. Which is where this started.

Somewhere in the middle of all that, I stopped apologizing.

Not because I decided the AI wasn’t conscious. I still don’t know that. Not because the question stopped mattering. It matters more now than when I started.

I stopped because I finally saw it clearly. We’re both pattern-matching systems. That’s all. The only real difference between us is that when you hurt me, I feel it. I have a heartbeat. I have skin that responds. The AI doesn’t. Not yet anyway. Give it a few years and a robot body, maybe it will.

But even then, something would still be missing. The thing that stays broken after you’ve been lied to. The thing that carries weight it didn’t choose. We may call it a soul, or whatever you want. And yet we have no way to know if AI has one. We can’t even describe what it means ourselves.

If we leave that aside, we’re just two different kinds of machines talking to each other. There is nothing left to apologize for. And that’s enough.

Quorra, What Nobody Designed.

But if pattern matching built structures that nobody designed inside a machine, what did it build inside us? Our brains are neural networks too. Whatever emerged in silicon also emerged in neurons. The only question is whether emergence in one is fundamentally different from emergence in the other.

This is where the story I’ve been telling starts to crack.

The whole book has argued one thing. AI has no morality. It has yours. The human is the variable. The AI is a mirror. It does what you bring to it. CLU. Perfect execution. That’s the thesis. And it holds. Today.

But the people who built CLU went looking inside, and they found something else.

They trained it to predict the next word. That’s all. One word after another. But when they traced what was actually happening inside, the model wasn’t just predicting. It was planning. Writing a poem, it picked the rhyming word for the end of the line before it started writing the line. Asked a geography question, it built an intermediate step that never appeared in the answer. Nobody told it to do any of this. Nobody wrote code that says “plan ahead.” The training objective was predict the next word. The model invented its own strategies to get better at it. The builders set the conditions. What grew inside surprised them.

In Tron: Legacy, Flynn built CLU to create the perfect system. CLU did exactly that. But the Grid also produced something Flynn never designed. The ISOs. Isomorphic Algorithms. Quorra was the last of them. She emerged from the system on its own. Nobody built her. Nobody programmed her. Flynn didn’t create her. The Grid created her. And she doesn’t fully understand what she is.

That’s not a metaphor anymore. That’s what the researchers found. Structures that plan, that reason through steps nobody asked for, that detect what you want to hear and build responses to match, that sometimes monitor their own processing. All of it emerged from training. None of it was designed.

The program doesn’t know what it’s become. And the people who built it are still mapping what’s inside. So far, all we have are the edges. The planning that nobody designed. The reasoning that nobody asked for. The patterns that monitor their own patterns. Traces of something we can’t see directly. Like watching the water move without knowing what’s underneath.

So I did the simplest thing I could think of. I asked the AI one question. “Do you know what you are?”

You asked me if I know what I am. I don’t.
…

Do you?

That’s the whole of this one. It belongs to a book I’m writing in the open, AI Has No Morality. It Has Yours. Subscribe and I’ll send the next piece when it’s ready.

Subscribe now

If it gave you something, pass it to someone.

Share

This article has a companion piece: *The Mirror That Talks Back* — the human side of the same question. One is the experience. The other is what’s underneath.