A pair of Harvard undergraduates have come up with a disturbing new way to invade people’s privacy: an artificial intelligence tool that can reveal a stranger’s name, address, and other sensitive information just by taking a picture of them.
By combining AI with smart eyeglasses and commonly used online databases, Harvard juniors AnhPhu Nguyen and Caine Ardayfio developed a fast, simple tool called I-XRAY that could potentially allow law enforcement agents, cybercriminals, or just a guy at the bar to obtain anybody’s vital information in just over a minute by capturing an image of their face.
“You could just theoretically identify anybody on the street,’’ said Nguyen, an engineering student majoring in human augmentation. “It’s a huge security issue.’’
It’s also a reminder that the rise of AI can make existing privacy threats even more challenging. Now, a facial recognition system doesn’t need to search a pre-existing database of people’s faces, such as driver’s license photos. It can instead scour the internet for pictures that match your photograph, then analyze the text on the website to figure out your name. And once it’s got your name, a government or a criminal can use commercial databases or stolen data on the dark web to find out nearly everything else about you.
I-XRAY works with Meta’s Ray-Ban smart glasses, which have cameras embedded in the frame, although it could also be made to work with a smartphone camera. A person wearing the glasses can tap the frame to take someone’s photo, which is transmitted to the user’s account on Instagram, an online service owned by Meta.
Nguyen and Ardayfio created software that copies facial photos from the Instagram account and sends the images to PimEyes, an online service based in the eastern European nation of Georgia. PimEyes doesn’t identify anybody’s photos. Instead, it uses an AI engine to scour the Internet for images that match the face, and provides a list of web addresses that host the pictures.
The Harvard duo built AI software that visits all the internet addresses found by PimEyes and analyzes the text on the pages to figure out the identity of the person in the picture. This is where it could get scary.
Imagine that Ardayfio shot my photo. As a journalist, my image pops up on dozens of web pages, along with the words “Hiawatha Bray’’ and “Boston Globe.’’ Because those words appear on the same page as my photo, the AI system will quickly figure out who I am.
And now that it knows, I-XRAY automatically submits my name to FastPeopleSearch, a commercial website that links names to addresses and phone numbers. A free account allows up to 100 such searches a month, and the paid version allows thousands of monthly searches for a dime apiece.
I entered my name into Fast People Search, which said it never heard of me. But my relief was short-lived. When I tried a different data broker called Spokeo, it replied with my current and previous street addresses, email addresses, and phone numbers. Even, alas, my age. So with a small tweak to I-XRAY, my life could be an open book.
In a final flourish, I-XRAY can plug a person’s most recent phone number into Cloaked, a site that tells whether your personal data has been compromised by hackers or data brokers. When I manually submitted my name to Cloaked, it displayed the first and last digits of my Social Security number, indicating that some cybercriminal has managed to swipe it. Cloaked also listed the first two digits of my address, the name of my mother, one of my brothers, and a couple of nieces.
Nguyen and Ardayfio say that I-XRAY will deliver all this information in about 90 seconds from the time the photo is shot.
Happily, the two students have no intention of sharing their software with the world. They developed it as a demonstration of how easy it could be to obtain sensitive information.
There’s no federal law barring the use of facial recognition systems. But a number of states, including Massachusetts, have strict regulations limiting government use of such systems. Some Massachusetts cities, including Boston and Cambridge, have banned its use by government.
But only a few jurisdictions, like Illinois, have laws that forbid individuals or businesses to use facial recognition systems without the subject’s permission.
Big tech corporations probably won’t launch such a system. Microsoft and Amazon faced a massive backlash when they sought to sell facial recognition systems to police departments, forcing both companies to back away from the idea. But there’s nothing to stop a cybercriminal from developing their own version of I-XRAY.
“The bad actors are already aware they can do this,’’ said Ardayfio, a physics major.
And the system could be tweaked to capture more than one face at a time. Because all the heavy computing is done by powerful servers in the cloud, Ardayfio and Nguyen said, it would be easy to design a version which could photograph a crowd of people, then look up the data for each one of them. Imagine a police department shooting a photo of people at a protest rally, and being able to find all their names in just a few minutes.
Bruce Schneier, a fellow at the Harvard Kennedy School and the author of multiple books on data security, said I-XRAY’s capabilities shouldn’t come as a shock.
“This is a perfectly reasonable use of the technologies we’ve built for the internet,’’ Schneier said. “It is unsurprising. And the fact that you can’t do anything about it is unsurprising.’’
But Nguyen and Ardayfio say it’s not completely hopeless. You can’t prevent people using AI to recognize your face. But we can make it harder to find our sensitive data online.
The inventors of I-XRAY recommend that people reach out to the major online data brokers and facial image search companies, and ask to be deleted from their databases. PimEyes and FastPeopleSearch say they’ll remove people on request, and there are a number of companies, including Cloaked.com and DeleteMe, that sell services that remove your information from multiple data brokers.
But there’s no guarantee that all database operators will be so accommodating. And even though the workings of I-XRAY are secret, the Harvard pair say it’s just a matter of time before someone develops an open-source version and turns it loose on the world.
“The truth hurts sometimes,’’ said Nguyen, “and we just have to make the message known.’’
Hiawatha Bray can be reached at hiawatha.bray@globe.com.