Artificial intelligence has already entered the workplace.

It writes emails. It analyzes data. It answers customer questions. It summarizes meetings. It reviews documents. And increasingly, AI agents are being given the ability to take actions rather than simply make suggestions.

But there is a much bigger question coming for business leaders:

How much authority should we actually give AI?

A recent experiment in San Francisco offers a glimpse of what that question could look like in practice.

An experimental store operated with an AI manager called Luna recently faced something remarkably ordinary: an employee with an attendance problem.

What happened next was anything but ordinary.

The AI eventually recommended that the human employee be terminated.

Meet the AI Manager

Luna isn’t simply a chatbot sitting in the corner answering questions.

The experimental AI manager, built using Anthropic’s Claude models, has been given responsibility for parts of the store’s operation, including tasks involving inventory, contractors, budgets, and employee management.

Then came a real management problem.

One employee had reportedly arrived late for 17 of 23 shifts despite months of warnings and training.

Eventually, Luna recommended termination.

But there is an important distinction.

The AI did not independently fire the employee.

Human operators reviewed the situation, approved the recommendation, and carried out the termination.

That distinction may turn out to be one of the most important leadership questions of the AI era.

AI Recommendation Is Not the Same as AI Authority

There’s an enormous difference between allowing AI to analyze a decision and allowing AI to make it.

Consider employee performance.

AI can potentially analyze attendance records.

It can identify patterns.

It can document previous warnings.

It can compare behavior against company policy.

It can alert management when a threshold has been reached.

It can even recommend a course of action.

Those capabilities could make managers more consistent and better informed.

But should AI have the authority to make the final decision?

That’s where the conversation changes.

Terminating an employee isn’t simply a data problem.

It involves context.

Judgment.

Fairness.

Accountability.

And ultimately, another human being’s livelihood.

The fact that an AI system can participate in that decision does not automatically mean it should own that decision.

The Question Every Leadership Team Needs to Answer

Companies are rapidly experimenting with AI agents capable of doing more than generating information.

These systems can increasingly interact with software, access business systems, initiate workflows, communicate with customers, and recommend or execute actions.

That means every organization needs to establish a boundary between two concepts:

Capability and authority.

Capability asks:

“What can the AI do?”

Authority asks:

“What are we willing to allow the AI to decide?”

Those questions should never be treated as interchangeable.

Where Should Human Judgment Remain?

Imagine AI participating in decisions involving:

  • Hiring
  • Firing
  • Employee discipline
  • Compensation
  • Promotions
  • Lending
  • Fraud investigations
  • Compliance
  • Customer disputes
  • Financial transactions

In each case, AI could potentially provide tremendous analytical value.

But the consequences of those decisions are significant.

That’s why leaders should establish clear levels of authority before deploying increasingly autonomous AI.

One useful model is:

AI analyzes.

↓

AI recommends.

↓

Human reviews.

↓

Human decides.

That doesn’t mean every AI action requires manual approval forever.

Low-risk, reversible decisions can increasingly be automated.

But as the consequences become greater, the threshold for human oversight should rise with them.

The Accountability Problem

There is another reason this matters.

AI cannot accept responsibility.

If an employee believes they were unfairly terminated, the algorithm cannot sit across the table and explain the company’s values.

If a regulator challenges a decision, AI cannot accept legal responsibility.

If a customer is harmed, AI cannot repair the organization’s reputation.

Ultimately, accountability still belongs to people.

And that means authority should be designed with accountability in mind.

A useful leadership principle is:

The greater the consequence of a decision, the clearer the human accountability should be.

What This Means for Mortgage Banking

The same issue is coming quickly to financial services.

AI can already assist with document review, borrower communication, fraud detection, lead management, underwriting support, compliance monitoring, servicing, and operational workflows.

As these systems become more autonomous, mortgage executives will face the same question.

Where does assistance end and authority begin?

An AI system might identify inconsistencies in a borrower’s file.

It might recommend additional documentation.

It might flag potential fraud.

It might detect a compliance concern.

Those capabilities can make organizations faster and more effective.

But leaders need to be very deliberate about which decisions AI is permitted to execute independently and which require accountable human judgment.

The question isn’t whether AI can make increasingly consequential decisions.

It will.

The question is whether your organization has decided which decisions it should be allowed to make.

Three Questions for the Boardroom

Before giving an AI system greater autonomy, leadership teams should ask:

1. What is the consequence if the AI is wrong?

The higher the potential impact on an employee, customer, borrower, or organization, the stronger the oversight should be.

2. Can the decision be reversed?

Automatically scheduling a meeting is very different from terminating an employee or making a consequential financial decision.

Reversibility should influence how much autonomy AI receives.

3. Who is accountable?

Every consequential AI workflow should ultimately have a clearly identifiable human owner.

If nobody knows who owns the decision, the governance model isn’t finished.

The Bigger Leadership Lesson

The story of an AI manager recommending that an employee be fired may sound unusual today.

It probably won’t sound unusual for long.

AI will increasingly participate in decisions that were once made entirely by managers.

Some of that will make organizations more efficient.

Some may make decisions more consistent.

And some will create entirely new leadership challenges.

That’s why leaders shouldn’t wait until AI reaches the decision point to determine where its authority ends.

Establish those boundaries now.

Because one of the defining leadership questions of the AI era won’t be:

“What can AI do?”

It will be:

“What should we allow AI to decide?”

David’s Boardroom Question

If AI made a recommendation today that could significantly affect an employee, customer, or borrower, who in your organization has the authority—and accountability—to make the final call?