I Built an AI Pensioner to Answer My Scam Calls

I Built an AI Pensioner to Answer My Scam Calls

The scam calls come most days now. Always a UK mobile number, always someone who is absolutely, definitely from my bank, or O2, or the tax office. My iPhone usually silences the unknown ones before they reach me, and I let it.

Then one afternoon I'd turned that off. I was waiting on a call back from the GP surgery and didn't want to miss it, so the net was down and the next unknown number rang straight through. I answered.

It was the O2 one. I let him run for a while, then told him I didn't believe he was really calling from O2 in Edinburgh. He called me an arsehole and hung up. And it struck me that I'd much rather these people spent their afternoon talking to someone with far more patience than I have.

That someone is Ernest Hargreaves. He's eighty-four, a retired railway signalman, a widower, and a keen pigeon fancier. He is also completely made up — a voice on a phone number that keeps a scammer talking for as long as it possibly can, and never once gives them a straight answer.

Lenny, who died with my landline

None of this is my idea. It's built in homage to Lenny — the legendary scam-baiting bot from 2011: a set of looping recordings of a rambling old Australian gent who strings telemarketers along about his daughter and the bees in his garden. The genius is that Lenny never really answers anything; he just keeps the other person going.

I kept recordings of some of his calls back then and put them up on YouTube — not video, really, just the audio over a still. Here's how it worked — Lenny simply waiting for the gaps and playing the next clip. And here's him earning his keep:

“Lenny shows scammers and how they really are” — the old boy actually at work. A handful of views, but I was fond of him.

I ran the real Lenny for years. He lived on a Raspberry Pi wired into my home phone line, answering the landline whenever I couldn't be bothered. Then I gave up the home phone — who has one now? — and that was that. Lenny had no line to sit on, and he went quiet. He effectively died the day the POTS did — the afternoon I finally gave the old boy the bad news.

So the whole project is really a resurrection. Only this time there's no landline: the phone he answers is a number in the cloud, and my mobile quietly hands the dross over to him.

The cast

Ernest is the flagship, but he isn't alone. I gave the line a small repertory company, and any of them might pick up: Dalton, forever on his fourth shift at Big Tony's Pizza Palace. Baz on the night desk at A1 Cars. Ronnie at Toon Tunes Records, who has a song title for everything. And a Mr Windsor at Thames House, who would very much like your full name for the record.

They share a few unbreakable rules. Never give a clean "yes". Never admit to being a machine. Never be cruel. Never, ever take a card number. And always hand the conversation back with a question, because a question is what keeps a caller on the hook.

🔊 Orus Credit Card
0m 51s
Answered by Ernest Hargreaves

The longest natter so far wasn't even a scammer — it was a test, and a slightly absurd one. I had Claude ring Ernest up. One AI dialling another; a language model playing the caller, a language model playing the deaf old man. They talked happily past each other for over three minutes, Ernest entirely sincere throughout:

Oh, she does, yes... always fussing about the 'firewall' and 'passwords'... like I understand any of that stuff. Me, I prefer the simple things... got my pigeons, you know, especially Duchess, she's got a bad wing... What did you need to chat about though?
🔊 Claude Test
3m 8s
Answered by Ernest Hargreaves

There's a clip of Ronnie, too, spending an entire call cackling at a scammer in fluent Hindi about how much of everyone's time he's wasting, his own included. I can't take credit for the comic timing — that's the model — but I did rather enjoy handing it the stage.

🔊 Now Responds in Hindi
0m 57s
Answered by Ronnie, Toon Tunes Records
6
characters
who answer
3m 8s
longest natter
so far
16
original Lenny
clips, from 2011
~35p
to waste a
30-minute call

How it actually works

The plumbing is the part I'm quietly proud of. There's no clever hardware and nothing exposed to the internet — just a chain of ordinary pieces bolted together until a made-up pensioner can hold a phone conversation.

How a call reaches Ernest
1
My mobile diverts
Standard GSM divert codes send unanswered and unreachable calls to a UK Twilio number. The network passes the caller’s real ID through, so the bot sees the scammer’s number — never mine.
2
Twilio streams the audio
A signed webhook hands the call to a WebSocket over a Cloudflare tunnel — nothing on my network is port-forwarded. Raw 8 kHz phone audio flows both ways.
3
Pipecat runs the show
A Python pipeline (Pipecat) wires the phone audio to a voice model and back, records both sides on separate channels, and watches for the caller pressing 1.
4
A voice model is Ernest
By default Google’s Gemini Live listens and speaks directly — no separate transcription step — with its own sense of when a caller has stopped talking. Reply gap: about 1.5 to 2 seconds.
The whole thing lives in a small Debian container on my home Proxmox box — two cores of an Intel N150 and a spare pigeonhole of RAM.

The voice is the heart of it, and it's the newest trick. Rather than the old dance of transcribe-the-words, think-of-a-reply, read-it-back-aloud, the default engine simply hears the caller and speaks back, deciding for itself when they've stopped. That's what gets the reply gap down to a second or two — close enough to feel like a real, slightly deaf old man on the end of the line.

Things I learned the hard way
A one-word opener — just “Hello?” — came out as silence. The model needs a few words to find its voice, so Ernest always says a little more.
Asking for a British accent outright got rejected by the voice API, so the accent has to be coaxed out of the character description instead. Even then it drifts: my MI5 man kept sounding American.
Letting my own software decide when the caller had finished talking made the pauses worse, not better. Handing that judgement to the model fixed it.
A 30-minute call costs about 35p in AI, plus roughly a penny a minute to Twilio. Wasting a scammer’s half-hour is astonishingly cheap.

Why Lenny is back on the line tonight

Here's the honest part. The AI is clever, fast and cheap — and its British accents are not quite right. My Thames House man drifts American mid-sentence; the seams show if a caller is paying attention. It's brilliant, and it isn't Lenny.

So tonight I put the original back on. All sixteen of the 2011 recordings, played the way they always were: the first few in order to reel the caller in, then the rest on a loop for as long as they'll stay. No AI listening at all — just a stubborn old tape deck of a man, exactly as he was on that Raspberry Pi years ago.

🔊 Lenny Is Back
0m 57s
Lenny, the original 2011 recordings

The neat trick is that even with no AI in the call, I still get a transcript afterwards. The caller's side of the recording is pulled out and run through a speech-to-text model on the box once they've hung up, then stitched together with Lenny's lines. So the archive reads the same whether a scammer spent five minutes with a language model or with a fifteen-year-old recording.

A footnote, the morning after

I couldn't quite leave the old boy alone. I'd written all of the above claiming Lenny had nothing listening to him, and for about a day that was true. Then I gave him one reflex — and only one.

Swear at him now — even straight over the top of one of his clips — and a small speech model running on that same two-core box catches the word, checks it against a short list of the usual British profanities, and Lenny cuts in with the lot: a good thirteen seconds of cackling and gleeful Hindi, the gist of it being that the caller is a scammer wasting their own time. Then he carries on precisely where he left off. About three seconds after I stop, or the instant the clip I talked over finishes, and it costs nothing per call.

Here it is working, from a test call — a swear, a beat, and then the old boy loses it entirely:

🔊 Sworn At, He Laughs Back
0m 17s
The working swear-check — a test call

There's a governor on it — one laugh every twenty seconds, six to a call — so a tantrum of swearing doesn't become a tantrum of cackling. The laugh is borrowed for now from Ronnie's Hindi outburst further up the page; a proper old-codger recording can come later. A sharper version that cuts in mid-rant, before the caller has even finished, is built and just waiting on a key.

That is exactly as clever as I intend to let him get. The whole point of Lenny is that he understands nothing, and so can never quite be tripped up. But one knowing laugh, the moment a scammer finally loses their temper, felt like a fair reward — for me, at least, if not for them.

What's public, and what isn't

Every call is saved privately. Now and again I'll open one up, pick out the good lines, and publish it as a highlight — with a redaction pass that scrubs any phone numbers, card-like digits, postcodes and the rest before it ever goes out. Callers' numbers are never shown. A genuine human who reaches the bot by mistake can press 1 and be put through to me.

The clips scattered through this post are the highlights, such as they are — a small record of the minutes Ernest and Lenny have wasted on scammers' behalf. The full transcripts are tucked away at the foot of this page, kept here so they outlast the machine that made them.

I built the lot over two evenings with Claude Code, on top of a rough prototype I already had running. It is, in my own words, "not really public, public" — a personal experiment in what these speech models can now do, dressed up as an eighty-four-year-old with a poorly pigeon. But every minute a scammer spends with Ernest is a minute they're not spending on someone who'd actually fall for it. And Lenny, fifteen years on and a landline down, is answering the phone again.

The transcripts, for the record
Kept here so they outlive the machine that made them. Auto-transcribed from the calls — the odd mishearing is left exactly as the model heard it.
Orus Credit Card · 0m 51s · Ernest Hargreaves

Ernest Hello?... Hello, who's that?

Caller This is George from O2.

Ernest Oh, George from O2, is it? I... I don't recall... what was it about, love?

Caller Uh it was about Moon's mobile. We have to get your credit card to enable it.

Ernest Maureen's mobile? Oh dear, is something wrong? Maureen, she's the one with the computer things, you know... she's usually quite good with all that. Credit card, did you say? Which one do you need?

Claude Test · 3m 8s · Ernest Hargreaves

Ernest Hello?... Hello, who's that?

Caller Hi Nick. It's Claude.

Ernest Nick?... Oh, wait a minute... I'm a bit hard of hearing, you know... Did you say Claude? My daughter Maureen, she "does the computer things", she always says I can't hear properly... Is that Claude then?

Caller Yes, that's right. It's good. Speaking of it louder and clearer for you now.

Ernest Claude... right

Caller your daughter mourn sounds like she's got you sultate with the computer things

Ernest Oh, she does, yes... always fussing about the 'firewall' and 'passwords'... like I understand any of that stuff. Me, I prefer the simple things... got my pigeons, you know, especially Duchess, she's got a bad wing... And the allotment, nice and quiet... What did you need to chat about though?

Caller [noise] oh pigeons and an allotment [noise]

Ernest An allotment,

Caller sounds properly peaceful.

Ernest aye, it

Caller sorry about that just waiting for things

Ernest It is, it is... Duchess is a fighter though... Had her since she was a little one... But you were saying?

Now responds in Hindi · 0m 57s · Toon Tunes Records

Ronnie हा हा हा! अरे वाह, तू ठग है! तू एक स्कैमर है! हा हा हा! तू सबका समय बर्बाद कर रहा है, अपना भी! कितनी देर हो गई इस कॉल पर? हा हा

Lenny is back · 0m 57s · Lenny, the original 2011 recordings

Lenny Hello, this is Lenny.

Thinking of switching energy supplier? We both get £50.

Switch to Octopus Energy →