Language Technology
(A Machine Learning Approach)

UA
Department of Linguistics

Walter Daelemans and Antal van den Bosch

February - May 2006
Fridays, 13.30 - 17.00

Description

In the past two decades, many developers of natural language processing (NLP) systems have started to use probabilistic and machine learning methods for the automatic construction of NLP systems. Circumstantial reasons for this evolution are the growing availability of annotated language data, and the increased capacities of computers, but the main reason is that in many NLP areas these methods have alleviated the knowledge acquisition bottleneck that plagued NLP before, and have resulted in practical and accurate applications. On the other hand, due to their dependence on data, they have introduced a data acquisition bottleneck.

The course gives an overview of this recent evolution, and handles the following aspects of it:

historical motivation and background: pre-Chomskyan linguistics, pattern recognition, artificial intelligence
introduction into natural language processing, language technology, information extraction, and text mining
natural language processing applications: morpho-phonology, syntax, semantics, discourse
linguistic issues: memory, modularity, representation
data issues: scaling, automated/automatic annotation

Schedule

[February 24 2006] (Antal van den Bosch)
- Introduction machine learning for natural language processing
[March 3 2006] (Walter Daelemans)
- Shallow parsing
  - PPT slides (6-slide-per-page handout PDF)
  - Partial assignment 1
  - English PoS tagging ARFF file (5000 examples)
[March 10 2006] (Antal van den Bosch)
- Machine learning for phonology and morphology
  - A case study: Dutch word stress
  - Diminutive formation; word pronunciation; morphological analysis
  - POS tagging; chunking; function tagging
  - Rule induction
[March 17 2006] (Antal van den Bosch)
- ML for discourse - Powerpoint slides
  - Shallow understanding of users in dialogue systems
  - Class exercises
  - Naive Bayes, exemplified on spam filtering - Powerpoint slides, PDF handout
[March 24 2006] (Antal van den Bosch)
- Information extraction - Powerpoint slides, PDF handouts
  - Meta-learning: voting, stacking, boosting
[March 31 2006] (Véronique Hoste)
- ML for anaphora resolution - Powerpoint slides, PDF handouts
  - Guest lecture by Véronique Hoste
  - k-Nearest Neighbor classification
  - Assignment 3
[April 21 2006] (Antal van den Bosch)
- Issues (1): Modularity and more data
  - Minimal description length principle; software engineering
  - Learning curves; automatic or machine-aided annotation
  - Capita selecta 1: Wrapped progressive sampling search for optimizing learning algorithm parameters
  - Assignment 4
[April 28 2006] (Antal van den Bosch)
- Issues (2): Memory and representation
  - Computation versus memory; forgetting exceptions
  - Capita selecta 2: Word sense disambiguation
  - Capita selecta 3: Feature transformation through rule induction: A case study with the k-NN classifier
  - End assignment planning
[May 5 2006] (Walter Daelemans)
- Machine learning for document classification - Powerpoint slides, PDF handouts

Relevant literature

Jurafsky, D., and Martin, J. (2000). Speech and language processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice-Hall.
Mitchell, Tom (1997). Machine Learning. McGraw Hill.
Manning, C., and Schütze, H. (1999). Foundations of Statistical Natural Language Processing. MIT Press.
Witten, I., and Frank, E. WEKA Tutorial.

Language Technology(A Machine Learning Approach)

UA Department of Linguistics

Description

Schedule

Links

Relevant literature

Language Technology
(A Machine Learning Approach)

UA
Department of Linguistics