Bạn đang xem: The dirichlet distribution: what is it and why is it useful?
Since the likelihood function is usually defined from the data generating process, the difference choices of prior can make the integral more or less difficult to calculate. If the prior has the same algebraic khung as the likelihood, then often we can obtain a closed-form expression for the posterior, avoiding the need of numerical integration.
Motivating the Dirichlet Distribution
Here’s how the Dirichlet distribution can be used lớn characterize the random variability of a multinomial distribution. I’ve borrowed this example from a great blog post on visualizing the Dirichlet distribution.
Suppose we’re going lớn manufacture six-sided dice but allow the outcomes of a toss to be only one, two or three (so the later visualization is easier). If the die is fair then the probabilities of the three outcomes will be the same và equal lớn 1/3. We can represent the probabilities for the outcomes as a vector θ =(θ₁, θ₂, θ₃).
θ has two important properties: First, the sum of the probabilities for each entry must equal one, & none of the probabilities can be negative. When these conditions hold, we can use a multinomial distribution khổng lồ describe the results associated with rolling the die.
In other words, if we observe n dice rolls, D=x₁,…,x_k, then the likelihood function has the form:
Where N_k is the number of times the valuek∈1, 2, 3 has occurred.
We expect there will be some variability in the characteristics of the dice we produce, so even if we try to produce fair dice, we won’t expect the probabilities of each outcome for a particular die will be exactly 1/3, due to variability in the production process. To characterize this variability mathematically, we would lượt thích to know the probability density of every possible value of θ for a given manufacturing process. To vày this, let’s consider each element of θ as being an independent variable.
That is, for θ =(θ₁, θ₂, θ₃), we can treat θ₁, θ₂ & θ₃ each as an independent variable. Since the multinomial distribution requires that these three variables sum lớn one, we know that the allowable values of θ are confined to a plane. Furthermore, since each value θᵢ must be greater than or equal lớn zero, the mix of all allowable values of θ is confined to a triangle.
What we want lớn know is the probability mật độ trùng lặp từ khóa at each point on this triangle. This is where the Dirichlet distribution can help us: We can use it as the prior for the multinomial distribution.
The Dirichlet distribution defines a probability mật độ trùng lặp từ khóa for a vector valued đầu vào having the same characteristics as our multinomial parameter θ. It has support (the phối of points where it has non-zero values) over
K is the number of variables. Its probability density function has the following form:
The Dirichlet distribution is parameterized by the vector α, which has the same number of elements K as our multinomial parameter θ. So you can interpret p(θ|α) as the answer to lớn the question “what is the probability mật độ trùng lặp từ khóa associated with multinomial distribution θ, given that our Dirichlet distribution has parameter α?”
Get More From Built In’s Data Science ExpertsThe Poisson Process and Poisson Distribution, Explained (With Meteors!)
Visualizing the Dirichlet Distribution
We see the Dirichlet distribution indeed has the same form as the multinomial likelihood distribution. But what does it actually look like?
To see this, we need to cảnh báo that the Dirichlet distribution is the multivariate generalization of the beta distribution. The beta distribution is defined on the interval <0, 1> parameterized by two positive shape parameters α & β. As you might expect, it is the conjugate prior of the binomial (including Bernoulli) distribution. The figure shows the probability density function for the Beta distribution with a few α & β values.
As we can see, the beta mật độ trùng lặp từ khóa function can take a wide variety of different shapes depending on α and β. When both α and β are less than one, the distribution is U-shaped. In the limit of α = β → 0, it is a two point Bernoulli distribution with equal probability 1/2 at each Dirac delta function ends x=0 & x=1, and zero probability everywhere else. When α=β=1 we have the uniform <0, 1> distribution, which is the distribution with the largest entropy. When both α and β are greater than one the distribution is unimodal. This diversity of shapes by varying only two parameters makes it particularly useful for modeling actual measurements.
For the Dirichlet distribution Dir(α) we generalize these shapes to a K simplex. For K=3, visualizing the distribution requires us to vì the following:Generate a set of x-y coordinates over our triangle Map the x-y coordinates to the two-simplex coordinate spaceCompute Dir(α) for each point
Below are some examples, you can find the code in my Github repository.
We see it’s now the parameter α that governs the shapes of the distribution. In particular, the sum α₀=∑αᵢ controls the strength of the distribution (how peaked it is). If αᵢ for all i, we get spikes at the corners of the simplex. For values of αᵢ > 1, the distribution tends toward the centre of the simplex. As α₀ increases, the distribution becomes more tightly concentrated around the centre of the simplex.
Xem thêm: Bảng Đầy Đủ Các Công Thức Đạo Hàm Và Đạo Hàm Lượng Giác, Bảng Công Thức Đạo Hàm Và Đạo Hàm Lượng Giác
In the context of our original dice experiment, we would produce consistently fair dice as αᵢ → ∞. For a symmetric Dirichlet distribution with αᵢ > 1, we will produce fair dice, on average. If the goal is to lớn produce loaded dice (e.g., with a higher probability of rolling a three), we would want an asymmetric Dirichlet distribution with a higher value for α₃.
Built In Machine Learning TutorialsA Step-by-Step NLP Machine Learning Classifier Tutorial
Now you"ve seen what the Dirichlet distribution looks like, và the implications of using it as a prior for a multinomial likelihood function in the context of dice manufacturing.