Core Concepts

• Random variable
• Probability distributions (Gaussian, Bernoulli, multinomial)
• Expectation
• Variance
• Covariance
• Conditional probability
• Bayes' theorem
• Maximum Likelihood Estimation (MLE)
• Maximum A Posteriori (MAP)
• Hypothesis testing
• Confidence interval

Applications in Large Models

Language Modeling

• P(next token | context) is conditional probability.

Loss Function

• Cross-entropy loss originates from information theory and measures differences between probability distributions.

Sampling and Generation

• Top-k and Top-p (nucleus) sampling are both based on probability distributions.

Uncertainty Quantification

• Confidence estimation for model predictions.

Reinforcement Learning (RLHF)

• Optimization based on probabilistic policies.

---

贡献者

• Mira190贡献 2 次 · 最近 2025/09/13
• longsizhuo贡献 0 次 · 最近 2026/04/18

---

最近更新

Involution Hell© 2026 byCommunityunderCC BY-NC-SA 4.0