Unknown Calls in the Age of AI Voice Cloning: What Not to Say on the Phone
A few seconds of your voice is enough for AI cloning tools. Seven simple phone habits, a ready-to-print script and what to do if you already talked to a scammer.
Scammers can now clone a voice from a short audio clip, so every phone call with a stranger is also a chance for them to collect a sample of how you speak. You do not need to stop answering the phone. You need a few simple habits: let unknown numbers go to voicemail, let the caller speak first, keep unexpected calls short, never confirm personal or company details, and verify any surprising request by hanging up and calling back on a number you already trust. This guide explains why, and gives you and your team a script you can use today.
Why a few seconds of your voice now matter
The FTC has warned that a scammer only needs a short audio clip of someone's voice, which can come from content posted online, plus a voice-cloning program to make a call that sounds like a family member in trouble. The FBI has made the same point: criminals use AI-generated audio to impersonate relatives, executives and public figures, and it recommends listening closely to tone and word choice, because a cloned voice can be very convincing.
For a business, the risk is not limited to family emergency calls. A cloned voice of an owner, a manager or an office administrator can be used to:
- Call the bank, a vendor or an employee and ask for a "quick" change.
- Leave a voicemail that asks someone to approve a payment or share a code.
- Make an impersonation call more believable because it sounds like the right person.
The goal is not to panic. It is to stop giving away free material and to make sure that a familiar voice alone is never enough to move money, data or access.
What an unexpected call is really trying to collect
Many scam and robocall attempts are not trying to steal anything during the first call. They are collecting pieces they can use later:
- Proof that the number is live. Answering and talking tells an automated system that a real person picks up, which can lead to more calls.
- A voice sample. A long conversation, a custom voicemail greeting or a reading of your name, title and company gives a cloning tool more to work with.
- Details. Who handles payments, which bank you use, when the owner travels, which IT company you work with. These details make the next call or email convincing.
- A reaction under pressure. Urgency, secrecy and fear are how scammers push people to skip verification.
You may have heard that you should never say "yes" because a recorded "yes" can be used as a voice signature. AARP reports that experts consider that specific scenario unlikely today, because accounts rely on more than a single recorded word. The useful part of the advice still stands: a caller who opens with "Can you hear me?" is usually checking that the line is live and trying to start a conversation, and the less you say, the less they get.
Seven phone habits that protect you and your staff
1. Let unknown numbers go to voicemail
Legitimate callers leave a message. If a call matters, you will hear about it, and you can return it on your own terms. This one habit removes most of the risk.
2. If you answer, let the caller speak first
Answer with a neutral greeting, then wait. A pause or a click before anyone speaks often means an automated dialer. If nobody speaks, hang up.
3. Keep unexpected calls short
You can be polite and brief at the same time. Do not take part in surveys, do not repeat your full name, title and company, and do not tell stories about your day or your schedule. A short call gives away very little.
4. Never confirm codes, account details or "security" answers
No real bank, software company or IT provider needs you to read out a one-time code, a password or the answer to a security question over a call they started. If anyone asks, the call is over.
5. Hang up and call back on a number you already have
Caller ID can be faked, and a familiar voice can be cloned. When a call asks you to do something unusual, end it politely and call back on a number from your contacts, your invoice, the back of your card or the official website. The FTC and the FBI both give this exact advice.
6. Agree on a verification phrase
The FBI recommends creating a secret word or phrase with your family to verify identity. The same idea works in a small business: owners, managers and the people who handle payments agree on a phrase that is never written in email or chat, and that is required before anyone acts on an unusual request by phone.
7. Use the standard voicemail greeting
A recorded greeting in your own voice is a free sample for anyone who calls. AARP suggests using the default system greeting instead. For company lines, a generic greeting recorded once for the main number works well.
Quick reference: what to say on an unexpected call
| Instead of… | Say… |
|---|---|
| "Yes, this is Maria Lopez, office manager at…" | "Who is calling, please?" |
| "Sure, I can confirm the code you sent." | "I don't share codes by phone. I'll call the company directly." |
| "The owner is traveling until Monday." | "I can take a message." |
| "Our bank is…" or "We use … for IT." | "I'm not able to share that." |
| Staying on the line to argue | "Thank you, goodbye." Then hang up. |
Print this table and keep it near reception and accounts payable. People follow a script far more easily than a policy they read once.
Reduce the voice samples you publish
Webinars, videos and podcasts are good marketing, and you do not need to stop them. Just be deliberate:
- Keep personal social media accounts private where possible, as the FBI recommends, and limit followers to people you know.
- Be aware that executives and the staff who approve payments are the most valuable voices to copy, so their verification rules matter most.
- Do not post voice notes or videos that reveal internal processes, such as who approves wires or how vendors are paid.
The real protection is the process: if every payment, bank change and access request needs a call-back or a second person, a perfect voice clone still fails.
If you think you already talked to a scammer
- Do not call back the number that called you. Use a number you already trust.
- If you shared a code or password, change the password right away from a device you trust and check the account's sign-in activity and recovery settings.
- If money moved, call your bank immediately and ask for a recall. Speed matters.
- Tell your team and your IT provider, because the same caller often tries other people in the same office.
- Report it to the FTC at ReportFraud.ftc.gov and to the FBI's IC3 at ic3.gov.
Make it a team habit
A short training session, a printed script and a call-back directory with trusted numbers for your bank, key vendors and IT provider go a long way. If you want to go deeper on payment and video-call scams, read our guide to deepfake voice and video scams and the verification rules that stop them.
Core Florida IT helps businesses in Miami-Dade and South Florida set up phone and payment verification rules, security awareness training and the account protections that make a cloned voice useless. Request a free IT assessment and we will review where your team is most exposed and what to fix first. Learn more about our cybersecurity services.
Sources
- FTC Consumer Alert: Scammers use AI to enhance their family emergency schemes (March 20, 2023)
- FBI / IC3: Criminals Use Generative Artificial Intelligence to Facilitate Financial Fraud (PSA, December 3, 2024)
- AARP: Why You Shouldn't Answer Calls From Unknown Numbers (July 8, 2024)
- FTC: Report fraud (ReportFraud.ftc.gov)
