In this seminar we will study recent papers on neural networks and deep learning applied to analysis of
language. Interest in neural networks in problems of natural language processing (NLP) is growing rapidly,
particularly over the last 5 years, they have become a very active area of research. These techniques now
demonstrated results that surpass many earlier approaches on important problems in language analysis.
This followed earlier successes in other areas, such as vision and image processing.
In semantics, researchers have pursued effective models of "meaning" in language, at different
levels -- meaning of words, phrases, sentences, or entire documents. Semantics is important in many tasks
in NLP, since it allows the computer to model understanding about the content of text. Modeling meaning
requires finding effective representations for linguistic objects. When considering words, we may wish to
find words that have similar meaning, or related meaning, or "opposite" meaning, etc. The same question
can be asked about higher-level objects, e.g., whether two sentences have "the same" or similar meaning.
For the meaning of a document, we can ask whether the document describes some particular kind of event
(e.g., a political event, a technological innovation, a bankruptcy, etc.), whether a document about
a company is describing the company in positive or negative terms, etc.
Many problems in NLP can be viewed in terms of semantic representation. Finding an appropriate representation
for the given task often determines in big part the level of success we can achieve on the task. Neural
networks provide representations that are flexible -- useful for a variety of tasks -- sometimes for a
surprisingly broad variety of tasks. A neural network trained for one task is often useful for totally
different, seemingly unrelated tasks. Intuitively, this means that we may be approaching a representation
of language that is in some sense universal, or "true".
On top of neural networks we can build many interesting applications, such as document classification,
sentiment analysis, etc.
In the seminar we will study research papers about recent applications of neural nets and deep learning to
language problems.
We will have several invited guest speakers from outside the class, presenting their own research.
Prerequisites
- Fundamental understanding of machine learning;
- Fundamentals of NLP, or agreement with instructor -- in case the chosen topic can be approach without
in-depth knowledge of NLP.