feat: add soft_clustering_temp parameter for lightweight soft topic assignments

[pidefrem]

What does this PR do?

feat: add soft_clustering_temp parameter for lightweight soft topic assignments

Getting soft (probabilistic) topic assignments currently requires calculate_probabilities=True (expensive, HDBSCAN-specific) or approximate_distribution() (requires re-running the vectorizer). This PR adds a lightweight, clustering-agnostic alternative.

Changes:

Add a soft_clustering_temp parameter to transform():

# Hard assignment (current default)
topics, probs = topic_model.transform(new_docs)

# Soft assignment with temperature scaling
topics, probs = topic_model.transform(new_docs, soft_clustering_temp=0.5)
# probs is now a (n_docs, n_topics) matrix of soft assignments

How it works:

• Compute distances from each document embedding to all topic centroids
• Apply softmax(-distance / temperature) to get a probability distribution
• Low temperature → sharper (near-hard) assignments; high temperature → softer distributions

Clustering-agnostic — works with any clustering model, not just HDBSCAN. Default is None — existing behavior unchanged.

Fixes #2505

Coordination note: this adds one new optional parameter to transform(). I know you prefer to keep the hyperparameter surface small, so it stays opt-in (None = today's behavior) and the temperature is the only knob. If you'd rather expose this as a separate method or a different name than soft_clustering_temp, I'm glad to adjust before you spend time reviewing.

Before submitting

• This PR fixes a typo or improves the docs (if yes, ignore all other checks!).
• Did you read the contributor guideline?
• Was this discussed/approved via a Github issue? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes (if applicable)?
• Did you write any new necessary tests?