Seminar: Neural Networks for Language Applications

Algorithms and machine learning
Advanced studies
Year Semester Date Period Language In charge
2017 spring 23.01-27.02. 3-3 English Roman Yangarber


Time Room Lecturer Date
Mon 10-12 C220 Roman Yangarber 23.01.2017-27.02.2017



In this seminar we will study recent papers on neural networks and deep learning applied to analysis of
language. Interest in neural networks in problems of natural language processing (NLP) is growing rapidly,
particularly over the last 5 years, they have become a very active area of research.  These techniques now
demonstrated results that surpass many earlier approaches on important problems in language analysis.
This followed earlier successes in other areas, such as vision and image processing.

In semantics, researchers have pursued effective models of "meaning" in language, at different
levels -- meaning of words, phrases, sentences, or entire documents.  Semantics is important in many tasks
in NLP, since it allows the computer to model understanding about the content of text. Modeling meaning
requires finding effective representations for linguistic objects.  When considering words, we may wish to
find words that have similar meaning, or related meaning, or "opposite" meaning, etc.  The same question
can be asked about higher-level objects, e.g., whether two sentences have "the same" or similar meaning.
For the meaning of a document, we can ask whether the document describes some particular kind of event
(e.g., a political event, a technological innovation, a bankruptcy, etc.), whether a document about
a company is describing the company in positive or negative terms, etc.

Many problems in NLP can be viewed in terms of semantic representation. Finding an appropriate representation
for the given task often determines in big part the level of success we can achieve on the task. Neural
networks provide representations that are flexible -- useful for a variety of tasks -- sometimes for a
surprisingly broad variety of tasks. A neural network trained for one task is often useful for totally
different, seemingly unrelated tasks.  Intuitively, this means that we may be approaching a representation
of language that is in some sense universal, or "true".

On top of neural networks we can build many interesting applications, such as document classification,
sentiment analysis, etc.

In the seminar we will study research papers about recent applications of neural nets and deep learning to
language problems.

We will have several invited guest speakers from outside the class, presenting their own research.

- Fundamental understanding of machine learning;
- Fundamentals of NLP, or agreement with instructor -- in case the chosen topic can be approach without
in-depth knowledge of NLP.

Completing the course

Each participant should prepare to do the following:
- present (at least) two papers on her/his choice of topic to the audience; the two papers may cover the same
topic or separate topics,
- provide slides to the audience in advance of the presentation,
- answer questions from the audience,
- attend presentations by other members,
- participate in presentations of other members by reading their papers and asking relevant questions from
the presenter.

The grade is based on the presentations (60%), active participation in the presentations of others (30),
and attendance (10%).

Literature and material

Suggested readings/paper selection will be posted on the Course Wiki.