The Growing Risk of AI-Generated Images

In recent years, AI-generated images have attracted significant attention, with many people initially viewing them as an interesting new technology and, in some cases, being able to identify the tell-tale signs that an image was not quite real.

However, as AI-generated imagery continues to become more convincing, distinguishing between an artificially created image and a genuine photograph is becoming a much greater challenge. Creating a growing concern for organisations where visual identification forms part of their security and verification processes.

For financial institutions, businesses, educational institutions, governments and healthcare organisations, the implications are particularly important. If an image is being used as part of an identity verification or security process, the question is no longer simply whether the image looks real, it is whether the person behind that image can be reliably verified.

How Deepfakes Can Fuel Misinformation and Fraud

One of the most visible concerns surrounding AI-generated imagery is its ability to spread misinformation. For example, fake images depicting Donald Trump being arrested demonstrate how AI-generated imagery can be used to create misleading content, even though the original creator made it clear that the images are fake, there is no guarantee that the content will continue to be shared with that context attached.

This creates an environment where malicious individuals can use fabricated imagery to sow discord, exploit existing societal divisions and erode public trust. The concern therefore extends beyond whether an individual can identify an AI-generated image, it is also about what happens when that image is deliberately used to deceive other people.

Criminal syndicates can use deepfake imagery to impersonate other people, potentially creating opportunities to commit fraud. Where an organisation relies too heavily on a static image or visual representation as part of its verification process, a convincing fake could potentially undermine that process.

For businesses, this is an important security consideration, fraudsters are continually looking for opportunities to exploit weaknesses, and an organisation that does not regularly review its onboarding, identity verification and security frameworks may find that yesterday’s safeguards are not necessarily enough to address today’s threats.

The challenge is not just about keeping up with technology, it is about understanding how criminals could potentially use that technology against your business.

The Growing Business Risk Behind AI-Generated Images

The wider risk becomes clearer when AI-generated imagery is considered alongside existing impersonation and fraud trends. According to the Southern African Fraud Prevention Service (SAFPS), impersonation attacks increased by 264% for the first five months of the year compared to 2021. The South African Banking Risk Information Centre’s (SABRIC) Annual Crime Stats 2022 report also noted that banking app fraud cases increased by 36% between 2021 and 2022. An image of a person is often used to access essential services and resources, while increasingly accessible tools can generate hundreds of convincing AI images based on a handful of authentic images.

This gives criminals another potential avenue for digitally altering or recreating a person’s likeness. If a financial institution’s system relies on a static image, for example, fraudsters could potentially use AI-generated imagery to impersonate someone and attempt to access funds or a social grant. The potential risks extend beyond financial services, with impersonation potentially being used in SIM swap fraud, access to private medical records or voter fraud. Presentation attacks, where a criminal holds up a fake or hijacked image to a camera, are another concern, while deepfakes can also be used in digital injection attacks that bypass the camera on a device or are injected directly into a data stream.

For business owners, this highlights why understanding the changing nature of identity fraud matters. The risk is not limited to one industry, one type of transaction or one particular security process. When a person’s image can be manipulated or recreated, organisations need to think carefully about what they are actually relying on when verifying someone’s identity.

Strengthening Verification in an AI-Driven Landscape

As the technology behind deepfakes develops, businesses should consider investing in the latest technology that can help identify deepfake imagery and video that might otherwise go undetected.

Verification technology should form part of a broader onboarding security framework and be incorporated into established verification processes. By considering how identity information is assessed and verified, businesses can work towards creating greater confidence before allowing an application, transaction or business relationship to move forward.

For organisations concerned about fraud, impersonation and the risks associated with inadequate onboarding, this means regularly reviewing existing processes and considering whether they provide sufficient protection against emerging threats.

The objective is straightforward: make it harder for criminals to exploit your business.

Protecting Your Business When Images Can Lie

AI-generated images are about far more than funny pictures, memes or entertaining videos. In the wrong hands, the same technology can become a tool for deception, impersonation and fraud. That is why businesses cannot afford to assume that something is genuine simply because it looks genuine.

MarisIT understands the ever-growing fraud threat facing businesses across South Africa. Its all-in-one vetting and credit scoring platform gives businesses access to information that can support critical business decisions, while its Pay-As-You-Go approach provides a low-risk way to access the service without fixed contracts.

When criminals are constantly looking for new ways to exploit businesses, protecting your organisation starts with knowing who and what you can trust.