Understanding the Normal or Gaussian Distribution

The PDF of the Normal (aka Gaussian) distribution is:


$$ \mathcal{N}(\mu,\sigma) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} $$


This equation looks slightly intimidating and it is really not clear where it comes from. In this post I give a simple explanation (and I hope my explanation is correct) of why it looks the way it does and hopefully this will make it far less scary.

The most important thing to bear in mind is that the Normal distribution is designed to have certain properties. First and foremost it must have the familiar bell-curve shape. Secondly, the area under the curve must be equal to 1 otherwise it would not be a valid PDF. Let's start the explanation by considering the standard normal distribution:

$$ \mathcal{N}(0,1) = \frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}} $$
There are obviously two parts to this function. The first is the coefficient of \(\sqrt{2\pi}\) and the second part is the exponential term \(e^{-\frac{x^2}{2}}\). We focus on the second part first. Notice that it is in the form of an exponential function. The basic function would be \(e^x\). As \(x\) increases, this increases (as the name suggests) exponentially; rapidly running away to infinity. With negative values of \(x\), however, the value would quickly run to zero. The figure below shows what this most basic form looks like:

Using \(x^2\) rather than \(x\) has the effect of making the value run away to infinity as \(x\) becomes very small as well:

However, for the Normal distribution what is wanted is a bell curve. Now, we know that anything raised to 0 is 1, so we know that \(e^0=1\). In order to create the bell curve effect we need the value of our function to run away to zero as \(x\) either increases or decreases. With the basic effects above we see that we have the opposite with the value running away to infinity. If we take the reciprocal (1 divided by the value) of that value we would have what we want. We can achieve this by using:

$$ e^{-x^2} = \frac{1}{\exp(x^2)} $$

This gives:

Immediately we see that we have the desired shape for our curve. However, the area under this curve is \(\sqrt{\pi}\) and we need it to be 1. We therefore scale the entire curve by a factor of \(\frac{1}{\sqrt{\pi}}\) so that the area becomes 1.

It's worth noting that the bell curve shape can be derived using any constant, not necessarily Eurler's constant, \(e\). However, using any other constant gives an area that contains a term with \(ln(y)\) in whereas using Euler's constant makes that term equal to 1.

At this point we are almost done. One problem remains. Whilst the mean of our new curve is 0, as desired, the variance is now equal to 1/2. We would like the variance to equal 1 and so we have to change the scaling factor to include the square root of 2 and change the exponential term to include a half and that gives the desired results and the form of the standard normal distribution that we are used to.

The general form is the result of scaling the curve to provide a desired variance and shifting it to provide a specified mean.

Hopefully this curve is a lot less scary now that we better understand where it comes from and what each part of the equation does.