Corpus

As of 2018, NLTK comes with over 100 collections of large and well-structured text datasets, which are called corpora in NLP. Corpora can be used as dictionaries for checking word occurrences and as training pools for model learning and validating. Some useful and interesting corpora include Web Text corpus, Twitter samples, Shakespeare corpus sample, Sentiment Polarity, Names corpus (it contains lists of popular names, which we will be exploring very shortly), WordNet, and the Reuters benchmark corpus. The full list can be found at http://www.nltk.org/nltk_data. Before using any of these corpus resources, we need to first download them by running the following codes in the Python interpreter:

>>> import nltk
>>> nltk.download()

A new window will pop up and ask us which collections (the Collections tab in the following screenshot) or corpus (the Corpora tab in the following screenshot) to download, and where to keep the data:

Installing the whole popular package is the quick solution, since it contains all important corpora needed for your current study and future research. Installing a particular corpora, as shown in the following screenshot, is also fine:

Once the package or corpus you want to explore is installed, we can now take a look at the Names corpus (make sure the names corpus is installed).

First, import the corpus names:

>>> from nltk.corpus import names

We can check out the first 10 names in the list:

>>> print(names.words()[:10])
['Abagael', 'Abagail', 'Abbe', 'Abbey', 'Abbi', 'Abbie',
'Abby', 'Abigael', 'Abigail', 'Abigale']

There are, in total, 7944 names, as shown in the following output derived by executing the following command:

>>> print(len(names.words()))
7944

Other corpora are also fun to explore.

Besides the easy-to-use and abundant corpora pool, more importantly, NLTK is also good at many NLP and text analysis tasks including tokenization, PoS tagging, named entities recognition, word stemming, and lemmatization.