计算机视觉欺诈图像引发安全担忧

36大数据专稿，原文作者： Bill Steele 本文由36大数据翻译组Teradata大数据资深顾问黄民安 翻译，转载必须获得本站及译者的同意，拒绝任何不表明译者及来源的转载！

计算机识别物体的能力，正在接近人类的水平。然而，康奈尔大学的研究人员发现，计算机和人类一样，也会被光学幻象所迷惑，这就引起了安全问题。于是计算机视觉识别领域需要开辟新的途径。

康奈尔大学研究生Jason Yosinski和他在怀俄明州演进人工智能实验室的同事们，已经创建了一个计算机识别系统，能够明确区分那些人类无法识别的白噪声或随机几何图案差异。他们将在6月7日至12日波士顿举行的IEEE计算机视觉与模式识别大会上展示这项成果。

上面这些图像对于人类来说毫无意义。计算机能够清晰识别这些图像之间的细微差别。如同所示，上面两行图像为白噪声版本，下面两行图像为图案版本。

“我们认为取得这样的结果，是由于两个重要原因，”Yosinski说。 “首先，论文中他们强调基于现代机器学习的计算机视觉系统也可以被欺骗，它在许多方面存在安全隐患。其次，他们的方法提供了一个重要的调试工具，可以发现是哪些模块在进行网络学习。”

计算机可以被训练，通过目标物体的名称，来对他们对应的图片。针对同一目标物体，计算机从许多不同的角度去识别，并将这些不同的角度识别数据进行匹配，来构建某种模糊模型。近年来，计算机科学家正在使用称为深层神经网络（DNN）的系统，该系统能够模拟在人脑中的神经元，精确识别模糊的图像信息。 “深”网络使用模拟神经元的工作原理可以抽象成几个层次：首先识别到的是一只四条腿的动物，其次再识别成一只猫，这需要把图像相关的部分定义为一个完整的“连体”。

但是，计算机不会按照人类的方式来处理图像，Yosinski说。 “我们意识到，神经网络并不能对消防车这个产品构成进行解码，它只需要从很多物品中识别消防车的这个形象，”他解释说。物体的色彩、斑点线和图案可能就足够进行识别物体了。例如，计算机能通过给出的黄色、黑色条纹及图案形式，识别哪些是校车，哪些是电脑键盘。

在康奈尔大学创意机器实验室工作的利普森，是机械和航空航天工程的副教授。他说，研究人员会考虑DNN 系统中增加“进化”特点的图像识别。他们在接受过海量影像训练的数据库上使用DNN系统进行测试。用随机图像开始，他们慢慢改变图像的特征，如果一个新的识别特征被认定比原来有更大的确定性，研究人员将抛弃旧版本，并不断进行迭代。最终，人们对DNN系统认可度超过了99％，但这些研究并没有涉及到 识别人的视觉图像。

计算机科学安全方面的专家弗雷德•施耐德说， “研究表明，深度学习机制也可能被非真实事务所欺骗，因此我们需要研究这些非真实事务的原理，以吸取教训。这个原理可以用来对犯罪嫌疑人进行测谎。网络上的许多系统都在使用深度学习机制，试图从大数据中得到有益的结论。 DNN系统能帮助Web广告商来决定广告应该在哪些网站上进行展示。”

Yosinski指出，恶意网页可能包含虚假图像，来误导哪些图像搜索引擎，并且成功通过“安全搜索”的过滤。这就有可能被不法分子用来通过面部识别系统，从而成为合法的、被授权访客。

通过进一步的研究，研究人员试图“再培训”DNN系统，对各类虚假图像进行标记，以改善系统的识别水平。然而，道高一尺魔高一丈，新类型的虚假图像正在层出不穷，不断考验DNN系统的识别能力。

“在过去的几年里，图像识别领域在彻底改变。”Yosinski说。 “机器学习领域的研究人员出了很多成果，但在图像识别领域却没有显著的成果，我们仍然需要继续研究神经网络的工作机制。”

Yosinski正在与Jeff Clune、Anh Nguyen开展合作研究。Jeff Clune是怀俄明大学计算机科学助理教授，Anh Nguyen是怀俄明大学的研究生。这项研究得到了美国航空航天局空间技术研究奖学金的支持。

Images that fool computer vision raise security concerns

Computers are learning to recognize objects with near-human ability. But Cornell researchers have found that computers, like humans, can be fooled by optical illusions, which raises security concerns and opens new avenues for research in computer vision.

Cornell graduate student Jason Yosinski and colleagues at the University of Wyoming Evolving Artificial Intelligence Laboratory have created images that look to humans like white noise or random geometric patterns but which computers identify with great confidence as common objects. They will report their work at the IEEE Computer Vision and Pattern Recognition conference in Boston June 7-12.

“We think our results are important for two reasons,” said Yosinski. “First, they highlight the extent to which computer vision systems based on modern supervised machine learning may be fooled, which has security implications in many areas. Second, the methods used in the paper provide an important debugging tool to discover exactly which artifacts the networks are learning.”

Computers can be trained to recognize images by showing them photos of objects along with the name of the object. From many different views of the same object the computer assembles a sort of fuzzy model that fits them all and will match a new image of the same object. In recent years, computer scientists have reached a high level of success in image recognition using systems called Deep Neural Networks (DNN) that simulate the synapses in a human brain by increasing the value of a location in memory each time it is activated. “Deep” networks use several layers of simulated neurons to work at several levels of abstraction: One level recognizes that a picture is of a four-legged animal, another that it’s a cat, and another narrows it to “Siamese.”

But computers don’t process images the way humans do, Yosinski said. “We realized that the neural nets did not encode knowledge necessary to produce an image of a fire truck, only the knowledge necessary to tell fire trucks apart from other classes,” he explained. Blobs of color and patterns of lines might be enough. For example, the computer might say “school bus” given just yellow and black stripes, or “computer keyboard” for a repeating array of roughly square shapes.

Working in the Cornell Creative Machines lab with Hod Lipson, associate professor of mechanical and aerospace engineering, the researchers “evolved” images with the features a DNN would consider significant. They tested with two widely used DNN systems that have been trained on massive image databases. Starting with a random image, they slowly mutated the images, showing each new version to a DNN. If a new image was identified as a particular class with more certainty than the original, the researchers would discard the old version and continue to mutate the new one. Eventually this produced images that were recognized by the DNN with over 99 percent confidence but were not recognizable to human vision.

“The research shows that it is possible to ‘fool’ a deep learning system so it learns something that is not true but that you want it to learn,” said Fred Schneider, the Samuel B. Eckert Professor of Computer Science and a nationally recognized expert on computer security. “This potentially has the basis for malfeasants to cause automated systems to give carefully crafted wrong answers to certain questions. Many systems on the Web are using deep learning to analyze and draw inferences from large sets of data. DNN might be used by a Web advertiser to decide what ad to show you on Facebook or by an intelligence agency to decide if a particular activity is suspicious.”

Malicious Web pages might include fake images to fool image search engines or bypass “safe search” filters, Yosinski noted. Or an apparently abstract image might be accepted by a facial recognition system as an authorized visitor.

In a further step, the researchers tried “retraining” the DNN by showing it fooling images and labeling them as such. This produced some improvement, but the researchers said that even these new, retrained networks often could be fooled.

“The field of image recognition has been revolutionized in the last few years,” Yosinski said. “[Machine learning researchers] now have a lot of stuff that works, but what we don’t have, what we still need, is a better understanding of what’s really going on inside these neural networks.”

Yosinski collaborated with Jeff Clune, assistant professor of computer science at the University of Wyoming, and Wyoming graduate student Anh Nguyen. The research was supported by a NASA Space Technology Research fellowship.

End