Deep Learning for Information Extraction

People

Supervisor

Research areas

Temporary Supervisor

Dr Lizhen Qu

Description

Could you send just a query to Google to find the answers to "who graduated from ANU and is both a comedian and a film producer?" and "How many web users commented on the UEFA Champions League final 2016 in the last week? ". The former requires logical reasoning to find links between ANU alumni and their occupations, which are usually stored in a structured knowledge base such as Google Knowledge Graph. In the knowledge graph, each node represents an entity such as The_Australian_National_University and Kevin_Hart, and each edge denotes the relations between entities such as graduateFrom. The latter needs both logical reasoning and information extraction techniques, which map unstructured text into a structured knowledge representations stored in a knowledge base.

Traditionally, researchers apply supervised learning techniques to build models with rich handcrafted features to detect and characterize entities and their relations in text. However, the building of training datasets is time-consuming and expensive.

Recent advances of IE techniques show that deep learning techniques have an edge over the traditional methods by the capability of learning rich feature representations from text. Pure supervised learning of deep learning models still requires substantial training data. Despite of that, in the family of deep learning, transfer learning and unsupervised pre-training are the techniques with large potential of reducing training data. Therefore, this project aims to explore novel deep learning techniques for information extraction by using large knowledge bases and freely available unlabeled corpora.

The scope of the project can be trimmed down to a specific subtask for student projects of different lengths, such as

a)    Unsupervised pre-training techniques for relation extraction.

b)    Named entity disambiguation for novel entities by transfer learning.

Requirements

Familiarity with linear algebra, probability, and basic knowledge of machine learning. Knowledge of online learning and deep learning would be a plus. Good coding skills in Python, Lua, Scala, Java or C++.

Background Literature

Deep learning models for relation extraction:
dos Santos, Cıcero Nogueira, Bing Xiang, and Bowen Zhou. "Classifying relations by ranking with convolutional neural networks." Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Vol. 1. 2015.
 
Background of deep learning:
 
Deep learnining for natural language processing:

Keywords

Deep Learning, Information Extraction, Relation Extraction, named entity disambiguation

Updated:  1 June 2019/Responsible Officer:  Dean, CECS/Page Contact:  CECS Marketing