A new study shows that artificial intelligence programs that check medical images for evidence of cancer can be duped by hacks and cyberattacks. Researchers demonstrated that a computer program could add or remove evidence of cancer from mammograms, and those changes fooled both an AI tool and human radiologists.

That could lead to an incorrect diagnosis. An AI program helping to screen mammograms might say a scan is healthy when there are actually signs of cancer or incorrectly say that a patient does have cancer when they’re actually cancer free. Such hacks are not known to have happened in the real world yet, but the new study adds to a growing body of research suggesting healthcare organizations need to be prepared for them.

Hackers are increasingly targeting hospitals and healthcare institutions with cyberattacks. Most of the time, those attacks siphon off patient data (which is valuable on the black market) or lock up an organization’s computer systems until that organizations pays a ransom. Both of those types of attacks can harm patients by gumming up the operations at a hospital and making it harder for healthcare workers to deliver good care.

But experts are also growing more worried about the potential for more direct attacks on people’s health. Security researchers have shown that hackers can remotely break into internet-connected insulin pumps and deliver dangerous doses of the medication.

Many other studies have also demonstrated the possibility that a cyberattack on medical images could lead to incorrect diagnoses. In 2019, a team of cybersecurity researchers showed that hackers could add or remove evidence of lung cancer from CT scans. Those changes also fooled both human radiologists and artificial intelligence programs.

It should be noted that healthcare organizations and people designing AI models should be aware that hacks that alter medical scans are a possibility.