.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINXGALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/applications/plot_topics_extraction_with_nmf_lda.py"
.. LINE NUMBERS ARE GIVEN BELOW.
.. only:: html
.. note::
:class: sphxglrdownloadlinknote
Click :ref:`here `
to download the full example code or to run this example in your browser via Binder
.. rstclass:: sphxglrexampletitle
.. _sphx_glr_auto_examples_applications_plot_topics_extraction_with_nmf_lda.py:
=======================================================================================
Topic extraction with Nonnegative Matrix Factorization and Latent Dirichlet Allocation
=======================================================================================
This is an example of applying :class:`~sklearn.decomposition.NMF` and
:class:`~sklearn.decomposition.LatentDirichletAllocation` on a corpus
of documents and extract additive models of the topic structure of the
corpus. The output is a plot of topics, each represented as bar plot
using top few words based on weights.
Nonnegative Matrix Factorization is applied with two different objective
functions: the Frobenius norm, and the generalized KullbackLeibler divergence.
The latter is equivalent to Probabilistic Latent Semantic Indexing.
The default parameters (n_samples / n_features / n_components) should make
the example runnable in a couple of tens of seconds. You can try to
increase the dimensions of the problem, but be aware that the time
complexity is polynomial in NMF. In LDA, the time complexity is
proportional to (n_samples * iterations).
.. GENERATED FROM PYTHON SOURCE LINES 23158
.. rstclass:: sphxglrhorizontal
*
.. imagesg:: /auto_examples/applications/images/sphx_glr_plot_topics_extraction_with_nmf_lda_001.png
:alt: Topics in NMF model (Frobenius norm), Topic 1, Topic 2, Topic 3, Topic 4, Topic 5, Topic 6, Topic 7, Topic 8, Topic 9, Topic 10
:srcset: /auto_examples/applications/images/sphx_glr_plot_topics_extraction_with_nmf_lda_001.png
:class: sphxglrmultiimg
*
.. imagesg:: /auto_examples/applications/images/sphx_glr_plot_topics_extraction_with_nmf_lda_002.png
:alt: Topics in NMF model (generalized KullbackLeibler divergence), Topic 1, Topic 2, Topic 3, Topic 4, Topic 5, Topic 6, Topic 7, Topic 8, Topic 9, Topic 10
:srcset: /auto_examples/applications/images/sphx_glr_plot_topics_extraction_with_nmf_lda_002.png
:class: sphxglrmultiimg
*
.. imagesg:: /auto_examples/applications/images/sphx_glr_plot_topics_extraction_with_nmf_lda_003.png
:alt: Topics in LDA model, Topic 1, Topic 2, Topic 3, Topic 4, Topic 5, Topic 6, Topic 7, Topic 8, Topic 9, Topic 10
:srcset: /auto_examples/applications/images/sphx_glr_plot_topics_extraction_with_nmf_lda_003.png
:class: sphxglrmultiimg
.. rstclass:: sphxglrscriptout
Out:
.. codeblock:: none
Loading dataset...
done in 1.921s.
Extracting tfidf features for NMF...
done in 0.405s.
Extracting tf features for LDA...
done in 0.396s.
Fitting the NMF model (Frobenius norm) with tfidf features, n_samples=2000 and n_features=1000...
/home/circleci/project/sklearn/decomposition/_nmf.py:1422: FutureWarning: `alpha` was deprecated in version 1.0 and will be removed in 1.2. Use `alpha_W` and `alpha_H` instead
warnings.warn(
/home/circleci/project/sklearn/decomposition/_nmf.py:289: FutureWarning: The 'init' value, when 'init=None' and n_components is less than n_samples and n_features, will be changed from 'nndsvd' to 'nndsvda' in 1.1 (renaming of 0.26).
warnings.warn(
done in 0.440s.
Fitting the NMF model (generalized KullbackLeibler divergence) with tfidf features, n_samples=2000 and n_features=1000...
/home/circleci/project/sklearn/decomposition/_nmf.py:1422: FutureWarning: `alpha` was deprecated in version 1.0 and will be removed in 1.2. Use `alpha_W` and `alpha_H` instead
warnings.warn(
/home/circleci/project/sklearn/decomposition/_nmf.py:289: FutureWarning: The 'init' value, when 'init=None' and n_components is less than n_samples and n_features, will be changed from 'nndsvd' to 'nndsvda' in 1.1 (renaming of 0.26).
warnings.warn(
done in 1.311s.
Fitting LDA models with tf features, n_samples=2000 and n_features=1000...
done in 4.832s.

.. codeblock:: default
# Author: Olivier Grisel
# Lars Buitinck
# ChyiKwei Yau
# License: BSD 3 clause
from time import time
import matplotlib.pyplot as plt
from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer
from sklearn.decomposition import NMF, LatentDirichletAllocation
from sklearn.datasets import fetch_20newsgroups
n_samples = 2000
n_features = 1000
n_components = 10
n_top_words = 20
def plot_top_words(model, feature_names, n_top_words, title):
fig, axes = plt.subplots(2, 5, figsize=(30, 15), sharex=True)
axes = axes.flatten()
for topic_idx, topic in enumerate(model.components_):
top_features_ind = topic.argsort()[: n_top_words  1 : 1]
top_features = [feature_names[i] for i in top_features_ind]
weights = topic[top_features_ind]
ax = axes[topic_idx]
ax.barh(top_features, weights, height=0.7)
ax.set_title(f"Topic {topic_idx +1}", fontdict={"fontsize": 30})
ax.invert_yaxis()
ax.tick_params(axis="both", which="major", labelsize=20)
for i in "top right left".split():
ax.spines[i].set_visible(False)
fig.suptitle(title, fontsize=40)
plt.subplots_adjust(top=0.90, bottom=0.05, wspace=0.90, hspace=0.3)
plt.show()
# Load the 20 newsgroups dataset and vectorize it. We use a few heuristics
# to filter out useless terms early on: the posts are stripped of headers,
# footers and quoted replies, and common English words, words occurring in
# only one document or in at least 95% of the documents are removed.
print("Loading dataset...")
t0 = time()
data, _ = fetch_20newsgroups(
shuffle=True,
random_state=1,
remove=("headers", "footers", "quotes"),
return_X_y=True,
)
data_samples = data[:n_samples]
print("done in %0.3fs." % (time()  t0))
# Use tfidf features for NMF.
print("Extracting tfidf features for NMF...")
tfidf_vectorizer = TfidfVectorizer(
max_df=0.95, min_df=2, max_features=n_features, stop_words="english"
)
t0 = time()
tfidf = tfidf_vectorizer.fit_transform(data_samples)
print("done in %0.3fs." % (time()  t0))
# Use tf (raw term count) features for LDA.
print("Extracting tf features for LDA...")
tf_vectorizer = CountVectorizer(
max_df=0.95, min_df=2, max_features=n_features, stop_words="english"
)
t0 = time()
tf = tf_vectorizer.fit_transform(data_samples)
print("done in %0.3fs." % (time()  t0))
print()
# Fit the NMF model
print(
"Fitting the NMF model (Frobenius norm) with tfidf features, "
"n_samples=%d and n_features=%d..." % (n_samples, n_features)
)
t0 = time()
nmf = NMF(n_components=n_components, random_state=1, alpha=0.1, l1_ratio=0.5).fit(tfidf)
print("done in %0.3fs." % (time()  t0))
tfidf_feature_names = tfidf_vectorizer.get_feature_names_out()
plot_top_words(
nmf, tfidf_feature_names, n_top_words, "Topics in NMF model (Frobenius norm)"
)
# Fit the NMF model
print(
"\n" * 2,
"Fitting the NMF model (generalized KullbackLeibler "
"divergence) with tfidf features, n_samples=%d and n_features=%d..."
% (n_samples, n_features),
)
t0 = time()
nmf = NMF(
n_components=n_components,
random_state=1,
beta_loss="kullbackleibler",
solver="mu",
max_iter=1000,
alpha=0.1,
l1_ratio=0.5,
).fit(tfidf)
print("done in %0.3fs." % (time()  t0))
tfidf_feature_names = tfidf_vectorizer.get_feature_names_out()
plot_top_words(
nmf,
tfidf_feature_names,
n_top_words,
"Topics in NMF model (generalized KullbackLeibler divergence)",
)
print(
"\n" * 2,
"Fitting LDA models with tf features, n_samples=%d and n_features=%d..."
% (n_samples, n_features),
)
lda = LatentDirichletAllocation(
n_components=n_components,
max_iter=5,
learning_method="online",
learning_offset=50.0,
random_state=0,
)
t0 = time()
lda.fit(tf)
print("done in %0.3fs." % (time()  t0))
tf_feature_names = tf_vectorizer.get_feature_names_out()
plot_top_words(lda, tf_feature_names, n_top_words, "Topics in LDA model")
.. rstclass:: sphxglrtiming
**Total running time of the script:** ( 0 minutes 13.924 seconds)
.. _sphx_glr_download_auto_examples_applications_plot_topics_extraction_with_nmf_lda.py:
.. only :: html
.. container:: sphxglrfooter
:class: sphxglrfooterexample
.. container:: binderbadge
.. image:: images/binder_badge_logo.svg
:target: https://mybinder.org/v2/gh/scikitlearn/scikitlearn/main?urlpath=lab/tree/notebooks/auto_examples/applications/plot_topics_extraction_with_nmf_lda.ipynb
:alt: Launch binder
:width: 150 px
.. container:: sphxglrdownload sphxglrdownloadpython
:download:`Download Python source code: plot_topics_extraction_with_nmf_lda.py `
.. container:: sphxglrdownload sphxglrdownloadjupyter
:download:`Download Jupyter notebook: plot_topics_extraction_with_nmf_lda.ipynb `
.. only:: html
.. rstclass:: sphxglrsignature
`Gallery generated by SphinxGallery `_