Can I give my AI agent instructions by voice?
The short answer
Yes. Most of my own instructions are dictated. But speech-to-text gets names wrong, so the agent must read through the errors, and must check with you before acting on any name where a wrong guess would change the result.
Why I talk instead of type
I dictate most of my instructions. It is faster than typing, and I can explain what I want the way I would explain it to a colleague, with the background and the reasons. Agents work better with that background than with short typed commands.
The problem is names
Speech-to-text software turns what I say into text, and it handles ordinary words well. It handles names badly: names of people, companies and products. An Arabic name spoken inside an English sentence can come out as an English word that has nothing to do with it.
Most of the time this does not matter. The agent reads the whole sentence, understands what I meant, and carries on.
Where a wrong guess matters
It matters when the name decides what happens: who receives a file, which client record is meant, which document is changed. If the agent guesses wrong there, the file goes to the wrong person.
So the rule in my system is simple. Read through the errors, but never act on a guessed name where the guess changes what happens. Check once, then act.
What this means for you
- Voice is a good way to give an agent background. Use it.
- Tell the agent, in its standing rules, that instructions may arrive dictated and with errors.
- Any action tied to a name needs a check when the name is not certain. Sending, filing and changing a record are the usual cases.