官术网_书友最值得收藏!

Semantics and topic modeling

Gensim is famous for its powerful semantic and topic modeling algorithms. Topic modeling is a typical text mining task of discovering the hidden semantic structures in a document. Semantic structure in plain English is the distribution of word occurrences. It is obviously an unsupervised learning task. What we need to do is to feed in plain text and let the model figure out the abstract "topics". We will study topic modeling in detail in Chapter 3, Mining the 20 Newsgroups Dataset with Clustering and Topic Modeling Algorithms.

In addition to robust semantic modeling methods, gensim also provides the following functionalities:

  • Word embedding: Also known as word vectorization, this is an innovative way to represent words while preserving words' co-occurrence features. We will study word embedding in detail in Chapter 10, Machine Learning Best Practices.
  • Similarity querying: This functionality retrieves objects that are similar to the given query object. It's a feature built on top of word embedding.
  • Distributed computingThis functionality makes it possible to efficiently learn from millions of documents.

Last but not least, as mentioned in the first chapter, scikit-learn is the main package we use throughout this entire book. Luckily, it provides all text processing features we need, such as tokenization, besides comprehensive machine learning functionalities. Plus, it comes with a built-in loader for the 20 newsgroups dataset.

Now that the tools are available and properly installed, what about the data?

主站蜘蛛池模板: 台湾省| 论坛| 翁源县| 资溪县| 中牟县| 邵阳县| 靖宇县| 阳泉市| 夏河县| 个旧市| 江西省| 修文县| 大兴区| 武隆县| 绍兴县| 汉中市| 太湖县| 富宁县| 阜城县| 阳春市| 蒙城县| 武功县| 扎鲁特旗| 辰溪县| 霍城县| 静乐县| 衡水市| 汶川县| 霍山县| 镇远县| 苏州市| 云霄县| 泰州市| 中西区| 湖北省| 丰原市| 绥江县| 大理市| 涪陵区| 菏泽市| 沾化县|