17th January 2018 AI beats humans at reading and comprehension test Artificial intelligence developed by Microsoft and Alibaba has achieved better average performance on a Stanford University reading and comprehension test than human beings.
Researchers from Microsoft, along with Chinese technology company Alibaba, have created deep learning algorithms able to outperform humans in one of the world’s most challenging reading comprehension tests. The Stanford Question Answering Dataset (SQuAD) consists of more than 100,000 question-answer pairs based on over 500 Wikipedia articles. SQuAD is significantly larger than previous datasets and much more challenging – because the answers do not come from a small set of candidate answers and they have variable lengths. It is regarded by many as the world’s top machine reading-comprehension test and is widely used by research institutes and universities including the Allen Research Institute, Carnegie Mellon University, Facebook, Google, IBM, Microsoft and Stanford University. Earlier this month, Microsoft Research Asia – the U.S. company's research arm in the Asia Pacific region – achieved a score of 82.65 on SQuAD, beating the highest human score of 82.30 for the "Exact Match" test, i.e. providing exact answers to questions. An example of a question might be "What causes rain?" On Monday this week, Alibaba’s Institute of Data Science and Technologies (iDST) announced that it, too, had surpassed the human record, with its latest deep neural network model scoring 82.44. The iDST explains in a news release that the model "leverages the Hierarchical Attention Network" and its "accuracy was tied to its ability to read from paragraphs to sentences to words – locating precise phrases that contain potential answers." “It is our great honour to witness the milestone where machines surpass humans in reading comprehension,” said Luo Si, iDST’s chief scientist for Natural Language Processing. “We are thrilled to see NLP research has achieved significant progress over the year. We look forward to sharing our model-building methodology with the wider community and exporting the technology to our clients in the near future.” Alibaba, alongside other Chinese tech giants, such as Baidu Inc. and Tencent Holdings Ltd., are racing to develop AI that can enrich social media feeds, target ads or even aid in autonomous driving. Other potential uses of Alibaba's new program include guides for visits to museums and other public venues, or responses to online inquiries from medical patients. The Chinese government recently endorsed AI and automation technology in a national-level plan that calls for the country to become an industry leader by 2030.
Comments »
If you enjoyed this article, please consider sharing it:
|
