Hesham Elmahdy

If my AI agent makes a mistake, who is legally responsible?

The short answer

In most cases, your company is. The agent is not a person the law can hold to account, so what it says or does on your behalf is treated as what your company said or did. A contract with the vendor may share the cost with you, but your client will hold you responsible. The real decision is not who pays after the mistake. It is what the agent is allowed to do before it.

The agent is not a separate party

In 2024, a tribunal in British Columbia, Canada, ruled on a case against Air Canada. The airline’s website chatbot told a grieving passenger that he could buy his ticket at the full fare and ask for a bereavement discount within 90 days. That was wrong: the airline’s own policy did not allow the discount once the trip was over. When he asked for the refund, the airline refused.

Air Canada argued that the chatbot was a separate legal entity, responsible for its own actions. The tribunal rejected this. The chatbot was part of the airline’s website, so the airline was responsible for what it said, and it was ordered to compensate the passenger.

That chatbot only answered questions. An agent does more.

An agent acts, so its mistakes are actions

A chatbot gives a wrong answer, and you can correct it. An agent sends the email, issues the refund, changes the price, places the order. A wrong sentence can be taken back. A payment that has left, or a commitment a client has accepted, often cannot.

The more an agent can do on its own, the more of its mistakes become your company’s commitments.

What a vendor contract can and cannot do

Your contract with the vendor decides how the cost of a mistake is shared between you and them. Read what the vendor accepts when its product is wrong, and where it caps what it will pay. But that contract is between you and the vendor. Your client signed with you.

What published AI principles say

The OECD AI Principles, which more than 40 countries and the European Union have adopted, say that organisations that develop, deploy or operate AI systems should be held accountable for their proper functioning. The direction is the same as the Canadian case: the organisation that uses AI answers for what it does.

What this means for you

Decide these before the agent starts work, not after its first mistake:

I am not a lawyer. For your own contracts and your sector’s rules, ask one. But no lawyer can make an agent responsible for itself.

References

  1. American Bar Association, BC Tribunal Confirms Companies Remain Liable for Information Provided by AI Chatbot, February 2024
  2. OECD, OECD AI Principles, adopted 2019, updated 2024
  3. OECD, AI Principle 1.5: Accountability

More questions

All questions