AI in health care has been successfully
Posted: Mon Feb 03, 2025 4:00 am
The intellectual debt that accrues from machine learning features is dangerous not only because it is created by old-fashioned trial and error. Because most machine learning models cannot provide reasoned reasons for their current judgments, it is impossible to tell when they are wrong (unless there is already an independent judgment about the answers they are giving). Misfires in a well-trained system are rare. But they can also be deliberately triggered by someone who knows exactly what data to feed into that system.
Consider image recognition. Ten years ago, computers couldn’t easily identify objects in photographs. Today, image search systems, like many other systems we interact with every day, are powered by exceptionally capable machine learning models. Google’s image search is powered by the Inception neural network. In 2017, a research team of MIT uae number data undergraduate and graduate students at LabSix succeeded in altering the pixels of a photo of a cat so that, despite it looking like a cat to human eyes, Inception was 99.99% certain it had been given a photo of guacamole. Inception, of course, can’t explain what led it to conclude that a cat is a cat; as a result, there’s no easy way to predict how it might fail when presented with specially crafted or corrupted data. Such a system likely has unknown gaps in its accuracy that are potentially vulnerable to a clever and determined attacker.
As we use the knowledge generated by machine learning systems, these gaps could become significant. trained to classify skin lesions as benign or malignant. However, research published this year by researchers at Harvard Medical School and MIT showed that it can also be tricked into making inaccurate judgments using the same techniques that turn cats into guacamole (among other things, attackers could exploit these vulnerabilities to commit insurance fraud). Intoxicated by the predictive power of such systems, we might reject the human judges they promise to replace. But they will remain susceptible to hacking.
Consider image recognition. Ten years ago, computers couldn’t easily identify objects in photographs. Today, image search systems, like many other systems we interact with every day, are powered by exceptionally capable machine learning models. Google’s image search is powered by the Inception neural network. In 2017, a research team of MIT uae number data undergraduate and graduate students at LabSix succeeded in altering the pixels of a photo of a cat so that, despite it looking like a cat to human eyes, Inception was 99.99% certain it had been given a photo of guacamole. Inception, of course, can’t explain what led it to conclude that a cat is a cat; as a result, there’s no easy way to predict how it might fail when presented with specially crafted or corrupted data. Such a system likely has unknown gaps in its accuracy that are potentially vulnerable to a clever and determined attacker.
As we use the knowledge generated by machine learning systems, these gaps could become significant. trained to classify skin lesions as benign or malignant. However, research published this year by researchers at Harvard Medical School and MIT showed that it can also be tricked into making inaccurate judgments using the same techniques that turn cats into guacamole (among other things, attackers could exploit these vulnerabilities to commit insurance fraud). Intoxicated by the predictive power of such systems, we might reject the human judges they promise to replace. But they will remain susceptible to hacking.