Selecting N topics from M clusters

[vbertalan]

Hello. I have a quick question about selecting N topics. If I say that I want N topics from my original documents, am I saying, mathematically, that my documents are clustered into N clusters, and c-TFIDF is selecting the tokens with the highest values? If so, should the number of N topics always be the same as the number of HDBSCAN clusters generated? If I set HDBSCAN to find N clusters, and request M topics, are the results compromised?

[MaartenGr]

If I set HDBSCAN to find N clusters, and request M topics, are the results compromised?

No. What happens is when you adjust any parameter of HDBSCAN, BERTopic will still model all the topics that come out of that (N) using c-TF-IDF. If you have set nr_topics in BERTopic (M), then the N topics will merely be reduced to M topics by iteratively merging them.

Do note though that HDBSCAN does not allow you to select the number of topics directly.

[vbertalan]

Thank you for the answer. One last question: if M > N (i.e., nr_topics is greater than the N clusters from HDBSCAN). Is the difference (M-N) discarded in this case, and N topics returned?

[MaartenGr]

It would return the N topics since only N topics will be created. The nr_topics (M) is completely ignored in that case.