Are you scared yet? Meet Norman, the psychopathic AI
你害怕了吗？见见诺曼，变态人工智能。
By Jane Wakefield Technology reporter
简韦克菲尔德科技记者
Image copyright MIT Image caption Norman was named after Alfred Hitchcock's Norman Bates from his classic horror film Psycho
图片版权 麻省理工学院 诺曼是以阿尔弗雷德·希区柯克的经典恐怖电影“惊魂记”中的诺曼·贝茨的命名的。
Norman is an algorithm trained to understand pictures but, like its namesake Hitchcock's Norman Bates, it does not have an optimistic view of the world.
诺曼是一种学习理解图片的算法，但就像它的同名人物希区柯克的诺曼·贝茨一样，它对世界没有乐观的看法。
When a "normal" algorithm generated by artificial intelligence is asked what it sees in an abstract shape it chooses something cheery: "A group of birds sitting on top of a tree branch."
当一个人工智能生成的“正常的”算法被问到它在抽象的形状中看到了什么时，它选择了一种欢快的东西：“一群坐在树枝上的鸟。”
Norman sees a man being electrocuted.
诺尔曼看到一个人正在受电刑。
And where "normal" AI sees a couple of people standing next to each other, Norman sees a man jumping from a window.
当“正常的”人工智能看见了有几个人站在一起时，诺曼看到了一个男人从窗户跳下来。
The psychopathic algorithm was created by a team at the Massachusetts Institute of Technology, as part of an experiment to see what training AI on data from "the dark corners of the net" would do to its world view.
这个变态算法是由麻省理工学院的一个团队创建的，作为一个实验的一部分，目的是研究用“网络的黑暗角落”的数据训练人工智能会如何影响其世界观。
The software was shown images of people dying in gruesome circumstances, culled from a group on the website Reddit.
研究者把人们在可怕的环境中死亡的图片展现给这个软件看，这些照片来源于Reddit网站上的一组人。
Then the AI, which can interpret pictures and describe what it sees in text form, was shown inkblot drawings and asked what it saw in them.
然后，可以解释图片和描述它在文本形式中所看到的人工智能，会看到墨迹图纸，并被询问它在其中看到了什么。
These abstract images are traditionally used by psychologists to help assess the state of a patient's mind, in particular whether they perceive the world in a negative or positive light.
这些抽象的图像通常被心理学家用来帮助评估病人的心理状态，特别是判断病人是以消极的还是积极的眼光看待世界。
Norman's view was unremittingly bleak - it saw dead bodies, blood and destruction in every image.
诺曼的观点始终是暗淡的-它从每一张图纸中都看到了尸体、鲜血和毁灭。
正常人工智能看到的：
一个装花的花瓶。
诺曼看到的：
一个男人被枪毙。
Alongside Norman, another AI was trained on more normal images of cats, birds and people.
和诺曼一起的另一个人工智能被训练看更正常的猫、鸟和人的图像。
It saw far more cheerful images in the same abstract blots.
从同样的抽象墨迹中，它看到了更多令人愉快的画面。
The fact that Norman's responses were so much darker illustrates a harsh reality in the new world of machine learning, said Prof Iyad Rahwan, part of the three-person team from MIT's Media Lab which developed Norman.
伊亚德·拉赫万教授说，诺曼的反应如此黑暗的事实说明了在机器学习的新世界中一个严酷的现实。该教授是麻省理工学院媒体实验室的三人团队的一员，该实验室开发了诺曼。
"Data matters more than the algorithm.
“数据比算法更重要。”
"It highlights the idea that the data we use to train AI is reflected in the way the AI perceives the world and how it behaves."
“它强调了这样一种观点：我们用来训练人工智能的数据令反映在人工智能对世界的感知方式和行为方式上。”
正常人工智能看到的：
一张小鸟的黑白照。
诺曼看到的：
男人被拖进和面机。
Artificial intelligence is all around us these days - Google recently showed off AI making a phone call with a voice virtually indistinguishable from a human one, while fellow Alphabet firm Deepmind has made algorithms that can teach themselves to play complex games.
人工智能就在我们身边—谷歌最近展示了当人工智能打电话时，它的声音和人的声音几乎没有区别，而另一家Alphabet公司的DeepMind 发明了可以教自己玩复杂游戏的算法。
And AI is already being deployed across a wide variety of industries, from personal digital assistants, email filtering, search, fraud prevention, voice and facial recognition and content classification.
人工智能已经被广泛应用于各个行业，从个人数字助理、电子邮件过滤、搜索、防欺诈、语音和面部识别以及内容分类。
正常的人工智能看到的：
一个人在打着伞。
诺曼看到的：
一个男人在他尖叫的妻子面前被打死。
It can generate news, create new levels in video games, act as a customer service agent, analyse financial and medical reports and offer insights into how data centres can save energy.
它可以编写新闻，在电子游戏中创造新的关卡，充当客户服务代理，分析财务和医疗报告，并提供关于数据中心如何节省能源的见解。
But if the experiment with Norman proves anything it is that AI trained on bad data can itself turn bad.
但如果诺曼的实验有证明了什么，那就是接受过糟糕数据训练的人工智能本身就会变得糟糕。
Racist AI
种族主义的人工智能
Norman is biased towards death and destruction because that is all it knows and AI in real-life situations can be equally biased if it is trained on flawed data.
诺曼倾向于死亡和毁灭，因为这是它所知道的一切，而现实生活中的人工智能如果在有缺陷的数据上训练，也可能同样有偏见。
In May last year, a report claimed that an AI-generated computer program used by a US court for risk assessment was biased against black prisoners.
去年5月，一份报告称，美国一家法院用于风险评估的人工智能生成的计算机程序对黑人囚犯有偏见。
正常的人工智能看到的:
桌子上的结婚蛋糕的特写。
诺曼看到的：
一个人被超速行驶的车撞死。
The program flagged that black people were twice as likely as white people to reoffend, as a result of the flawed information that it was learning from.
因为它正在学习的错误信息，所以该程序指出，黑人再次犯罪的可能性是白人的两倍。
Predictive policing algorithms used in the US were also spotted as being similarly biased, as a result of the historical crime data on which they were trained.
基于这些算法接受训练的历史犯罪数据，这些在美国使用的预测警务算法也被发现具有类似的偏见。
Sometimes the data that AI "learns" from comes from humans intent on mischief-making so when Microsoft's chatbat Tay was released on Twitter in 2016, the bot quickly proved a hit with racists and trolls who taught it to defend white supremacists, call for genocide and express a fondness for Hitler.
有时候，人工智能“学到”的数据来自于有意制造恶作剧的人类，因此，当微软的Chatbat Tay于2016年在推特上发布时，这个机器人很快就被证明是种族主义者和网络霸凌者们的产品。他教它如何保卫白人至上主义者，呼吁种族灭绝，并表达对希特勒的喜爱。
Norman, it seems, is not alone when it comes to easily suggestible AI.
诺曼似乎并不是唯一一个容易被暗示的人工智能。
And AI hasn't stopped at racism.
人工智能并没有止步于种族主义。
One study showed that software trained on Google News became sexist as a result of the data it was learning from. When asked to complete the statement, "Man is to computer programmer as woman is to X", the software replied 'homemaker".
一项研究显示，通过谷歌新闻培训的软件由于其正在学习的数据而成为性别歧视者。当被要求完成陈述“男人是计算机程序员，女人是X”时，软件回答“家庭主妇”。
Dr Joanna Bryson, from the University of Bath's department of computer science said that the issue of sexist AI could be down to the fact that a lot of machines are programmed by "white, single guys from California" and can be addressed, at least partially, by diversifying the workforce.
来自巴斯大学计算机科学系的乔安娜·布莱森博士说，人工智能的性别歧视问题可能是因为许多机器都是由“来自加利福利亚的单身白种男性”编程的，至少可以部分地通过多样化的劳动力来解决这个问题。
She told the BBC it should come as no surprise that machines are picking up the opinions of the people who are training them.
她告诉BBC，机器正在收集训练他们的人的意见，这一点也不奇怪。
"When we train machines by choosing our culture, we necessarily transfer our own biases," she said.
“当我们通过选择我们的文化来训练机器时，我们必然会转移我们自己的偏见，”她说。
"There is no mathematical way to create fairness. Bias is not a bad word in machine learning. It just means that the machine is picking up regularities."
“没有任何数学方法可以创造公平。在机器学习中，偏见不是一个坏词。这只是意味着这台机器正在收集规律。“
What she worries about is the idea that some programmers would deliberately choose to hard-bake badness or bias into machines.
她担心的是，有些程序员会故意选择对机器输入错误价值观或偏见。
To stop this, the process of creating AI needs more oversight and greater transparency, she thinks.
她认为，要阻止这一现象，创建人工智能的过程需要更多的监督和更大的透明度。
Prof Rahwan said his experiment with Norman proved that "engineers have to find a way of balancing data in some way," but, he acknowledges the ever-expanding and important world of machine learning cannot be left to the programmers alone.
拉赫万教授说，他与诺曼的实验证明，“工程师必须找到某种方式平衡数据”，但是，他承认不断扩展机器学习这一重要领域。不能只留给程序员。
"There is a growing belief that machine behaviour can be something you can study in the same way as you study human behaviour," he said.
他说：“越来越多的人认为机器行为可能是你在研究人类行为的过程中能以同样的方式学习的东西。”
This new era of "AI psychology" would take the form of regular audits of the systems being developed, rather like those that exist in the banking world already, he said.
他说，这个“人工智能心理学”的新时代将采取对正在开发的系统进行定期审计的形式，就像那些已经存在于银行业的系统一样。
Microsoft's chief envisioning officer Dave Coplin thinks Norman is a great way to start an important conversation with the public and businesses who are coming to rely on AI more and more.
微软首席规划官戴夫科普林认为，诺曼是与日益依赖人工智能的公众和企业开始重要对话的好契机。
It must start, he said, with "a basic understanding of how these things work".
他说，首先必须“对这些事情的运作方式有一个基本的理解”。
"We are teaching algorithms in the same way as we teach human beings so there is a risk that we are not teaching everything right," he said.
他说：“我们教算法的方式和我们教人类的方式是一样的，所以我们有可能无法把所有的东西都教对。”
"When I see an answer from an algorithm, I need to know who made that algorithm," he added.
“当我看到一个算法的答案时，我需要知道是谁制定了这个算法，”他补充道。
"For example, if I use a tea-making algorithm made in North America then I know I am going to get a splash of milk in some lukewarm water."
“例如，如果我用北美制造的泡茶算法，我就知道我会得到一杯温水里溅上一滴牛奶的茶。”
From bad tea to dark thoughts about pictures, AI still has a lot to learn but Mr Coplin remains hopeful that, as algorithms become embedded in everything we do, humans will get better at spotting and eliminating bias in the data that feeds them.
从坏茶到关于图片的黑暗想法，人工智能仍然有很多需要学习的地方，但科普林先生仍然希望，随着算法嵌入到我们所做的每一件事中，人们在发现并消除数据中的偏见这方面会做的更好。