AI Should Expand Human Capability, Not Replace Human Judgment
AI can increase human capability without replacing human agency. A reflection on autonomy, system authority, engineering judgment, and the decisions humans should continue making for themselves.
What if the AI apocalypse doesn’t begin with a machine taking control?
What if it begins with humans willingly handing over control, one decision at a time?
That question has been sitting with me.
We spend a lot of time debating how intelligent AI might become. But I’m beginning to think there is another question that matters just as much:
How much authority are we willing to give it?
AI is learning from human-generated data, and human behavior contains everything: cooperation, empathy, creativity, greed, manipulation, violence, bias, generosity, and contradiction.
That doesn’t mean AI is destined to become harmful.
But it does mean we should be extremely careful about what we ask increasingly capable systems to optimize and what power we give them while they do it.
A system doesn’t need hatred, greed, anger, or a desire for power to cause harm.
Tell a sufficiently capable system to maximize engagement, increase profit, win a conflict, or achieve an objective without strong enough boundaries, and harmful behavior could simply become an efficient path toward the goal.
That leads me to a principle I keep coming back to:
AI should never simultaneously possess high capability, broad access, autonomy, and irreversible authority.
AI can absolutely assist us in high-stakes domains.
It can analyze military intelligence.
It can model economic scenarios.
It can help optimize critical infrastructure.
It can help governments understand complex policy tradeoffs.
It can even help engineers design better AI systems.
But assistance and authority are not the same thing.
I don’t believe AI should independently decide when weapons are used, make consequential government decisions, control critical financial systems, operate essential infrastructure without meaningful oversight, or freely rewrite its own safeguards and objectives.
The higher the consequence, the stronger the human control should be.
And while thinking about this at the scale of governments and powerful AI systems, I realized the same problem is already appearing at a much smaller scale.
In us.
Recently, I caught myself asking AI questions that I was perfectly capable of reasoning through myself.
The answers weren’t necessarily wrong.
That wasn’t what bothered me.
What bothered me was how easy it had become to skip the uncomfortable part where I wrestle with the problem first.
And if I keep skipping that step, what happens to the skill?
So I’ve started creating a boundary for myself:
My reasoning comes before the prompt.
For problems involving judgment, architecture, learning, or critical thinking, I want to form my own hypothesis first.
Then AI can attack it.
Challenge my assumptions.
Find gaps.
Introduce perspectives I missed.
Help me articulate something I already believe but haven’t figured out how to express.
That distinction matters.
I want AI carrying more of my workload, not more of my agency.
There is some irony in the fact that AI helped me turn this very conversation into an article.
But I actually think that demonstrates the relationship I want with it.
The questions were mine.
The discomfort was mine.
The position was mine.
The boundaries were mine.
AI helped me structure and articulate them.
That is a relationship with AI I hope we preserve at every level, from an engineer using a coding assistant to institutions deploying systems capable of affecting millions of people.
The goal shouldn’t simply be keeping a human somewhere in the workflow so we can say a human was involved.
The human needs to retain meaningful agency.
Perhaps the defining question of the AI era won’t ultimately be:
How intelligent can we make these systems?
It may be:
What should AI never decide for us?