Text Classification and POS Tagging Using NLTK

The Natural Language Toolkit (NLTK) is a Python library for handling natural language processing (NLP) tasks, ranging from segmenting words or sentences to performing advanced tasks, such as parsing grammar and classifying text. NLTK provides several modules and interfaces to work on natural language, useful for tasks such as document topic identification, parts of speech (POS) tagging, sentiment analysis, and so on. For experimentation with various NLP tasks, NLTK also includes modules for a wide range of text corpora, from basic text collections to tagged and structured texts, such as WordNet. While the NLTK library provides a vast set of APIs, we will only cover the most important aspects that are commonly used in practical NLP applications.

We will cover the following topics in this chapter:

• Installing NLTK and its modules
• Text preprocessing and exploratory analysis
• Exploratory analysis of text
• POS tagging
• Training a sentiment classifier for movie reviews
• Training a bag-of-words classifier