AI-written phishing: fluent does not mean trustworthy
Generative AI can help criminals produce persuasive messages in different languages with fewer spelling mistakes. The request matters more than the polish: unexpected attachments, payment changes and pressure to act still deserve scrutiny. Check the sender and destination, but also independently verify the action being requested.
Learn more: FBI / IC3Workplace example
A familiar-looking supplier email announces a new bank account and asks for payment before the end of the day.
The safer response
Pause the payment. Confirm the change using a trusted contact already on file, follow the normal approval process and report the suspicious message.
Deepfakes: verify the request beyond the voice
A familiar voice or convincing video is not sufficient proof of identity. Voice cloning can support impersonation fraud, including calls that sound like a manager. Do not make employees responsible for spotting every technical flaw in synthetic media; give them a reliable verification process.
Learn more: FTCWorkplace example
A call that sounds like your director asks for an urgent transfer and says to keep it confidential.
The safer response
End the call and reach the person through a known number or established company channel. Keep payment approvals in place, including when the request appears to come from leadership.
Safe generative AI use: protect inputs and check outputs
AI tools can produce fluent but incorrect answers. Before using one for work, confirm that the tool and the proposed data are allowed by company policy. Approval for an AI tool does not mean every category of information is suitable for it. Verify important claims, calculations, links and generated code before relying on the output.
Learn more: NCSCWorkplace example
An employee wants an AI assistant to summarise a customer support export containing names and account details.
The safer response
Check the approved workflow and data rules first. Remove sensitive information where required; if the permitted use is unclear, ask the designated security or data owner before uploading.
Prompt injection: when content tries to give instructions
Instructions hidden in a document, web page or message can attempt to redirect an AI assistant away from its intended task. This is especially relevant when assistants access other tools or information. Treat retrieved material as untrusted content; a confident AI response is not authorisation to share data or take action.
Learn more: OWASPWorkplace example
While summarising a document, an assistant unexpectedly asks to send internal files to an external address.
The safer response
Do not approve the unexpected action. Stop, preserve the relevant details and report it through your company process. Technical permissions and human approval should support the training.
A recommended training approach
Use AI assistance within a controlled, human-reviewed learning process. This is a suggested programme design, rather than a list of features included in every service plan.
Choose a behaviour
Define an observable goal, such as independently checking a changed bank account.
Draft and review
If AI helps write a scenario, have a qualified reviewer check accuracy, tone, privacy and relevance. Use fictional data.
Practise safely
Run authorised exercises, explain the safer response and provide a clear reporting route. Never request real passwords.
Learn and adjust
Review the results, discuss difficult decisions and repeat the lesson with a different example.
Make it relevant to each role
Finance & procurement
Verify beneficiary changes and payment requests through a known channel.
HR & people teams
Protect candidate and employee information; check impersonated instructions.
IT & all AI users
Review tool permissions, sensitive inputs and unexpected requests from assistants.
Measure learning, with context
Compare campaigns with similar difficulty and audiences. A lower click rate alone does not prove that risk has fallen, and an AI-generated score cannot establish an employee’s intent or predict a specific incident.
- Reporting rate: are people raising concerns?
- Time to report: how quickly can the team respond?
- Verification behaviour: did learners use the expected channel?
- Repeat gaps: which decisions need more explanation or practice?
AI awareness: common questions
Does AI awareness replace basic security training?
No. It extends phishing awareness, data protection and reporting habits. Strong passwords, multi-factor authentication and established approval processes remain part of the foundation.
Can staff learn to detect every deepfake?
No training can guarantee that. Teach independent identity checks and consistent approval processes so the response does not depend on detecting visual or audio defects.
Does the public demo use live AI?
The public demo uses sample data and simulated AI. It does not send live messages or connect to production systems. Discuss current service capabilities and your requirements with the team before choosing a plan.