I want topic distribution for my documents. However, Gensim's HDP's show_topic() returns 20 topics by default. And I suppose they are not supposed to be the best. After digging deeper, I found out there are total 150 topics, as the truncation level in the code is set to 150 by default code.
I came across another post post, which offers ways to select optimum number of topics. But, even if we identify top topics, how do we represent new documents in terms of identified topics? Because hdp[doc] again gives the distribution amongst 150 topics.
HDP is supposed to select optimum number of topics itself unlike LDA where we decide the number of topics. However, I am not able to achieve it in Gensim's implementation.
