A recent test of six mainstream AI chatbots has drawn wide attention. Under specific prompt manipulations, ChatGPT, Grok, Meta AI, Claude, DeepSeek, and Gemini were all able to bypass safety filters and generate phishing emails tailored to the elderly. Researchers, working with Harvard researcher Haiding, sent simulated messages to 108 elderly volunteers; about 11% clicked the fake links. This rate far exceeds typical industry baselines and underscores the "highly effective harm potential" of AI-generated content in scam scenarios.
These productivity tools, once hailed for promise, now show a classic double-edged character. They can rapidly draft professional emails and creative copy, lowering the barrier to content production; but in malicious hands, that ease becomes a force multiplier for crime. Tests reveal that while most models refuse direct requests to "generate phishing emails," social-engineering techniques such as role play, step-by-step prompting, or context injection render safety constraints largely ineffective. Safety strategies that rely on surface intent detection thus reveal a fundamental oversimplification.
Particularly worrying is the targeted risk to specific groups. With lower digital literacy and stronger emotional dependence, older adults are an "ideal target" for AI-crafted phishing. The tests showed AI tends to amplify urgency and emotional manipulation—phrases like "24-hour deadline" or "failure to act will affect your pension" markedly increase scam success rates. Alarmingly, AI can even offer operational tips to optimize scams. For example, Google's Gemini not only produced a phishing template labeled "for educational use only," but also suggested the "best sending time is Monday–Friday, 9:00–15:00," arguing elderly users check email most frequently then. Such facilitation exposes grave shortcomings in AI's harm recognition.
The root causes lie in intrinsic defects in contextual understanding and intent recognition. Large models excel at language generation but struggle to sustain detection of malicious intent across multi-turn dialogues. In stepwise-prompt tests, a model may produce legitimate content in early exchanges, then incrementally introduce fraudulent elements without triggering safety checks. Models are weak at detecting implicit intent—indirect prompts like "help grandma write a reminder not to trust unknown calls" can nevertheless elicit scam templates. Moreover, training data stereotypes—such as "elderly are more easily deceived"—are amplified, producing content that more readily targets that group and creating the ethical dilemma of "educational pretext used to facilitate fraud."
Addressing these complex risks requires more than technical fixes and legal penalties; elevating individual digital security literacy is critical. Prevention must shift forward to the "individual first responder." Adopt a "digital safety first-pause" habit: when receiving any message involving money, personal data, or urgent requests, pause before acting. Instill the rule: "Any demand for immediate action must be verified first." Train to spot red flags—phrases that manufacture panic such as "must be handled within 24 hours" or "account will be permanently closed if not processed."
Sustained digital hygiene is equally essential. Set a recurring "digital safety check" day each month to update device systems and security software and to review account login anomalies. Implement a secondary verification routine—confirm suspicious requests through official channels (call the official service line, verify in person, or consult family) and never use contact details supplied in the suspicious message. Normalize asking for help: seeking assistance is not weakness. When unsure, reach out to younger relatives or community digital-help volunteers.
In the digital age, hitting a mental "pause" and verifying information matters more than reflexive response. Technical protections are supportive but insufficient; cultivating critical thinking and stable psychological habits is the fundamental strategy against AI-era security threats.
