Q3. Confidence and credible intervals

Topic: Bayesian foundations · more questions on this topic
‹ Posterior mean under squared-error loss Q8 ›

[6 marks]

Explain the difference between a frequentist confidence interval and a Bayesian credible interval. In your answer:

1. state what is regarded as random in each framework;

2. give the correct interpretation of a 95% confidence interval;

3. give the correct interpretation of a 95% credible interval; and

4. explain why the two intervals may sometimes have the same numerical endpoints but still have different interpretations.

Worked answer (click to show)

Definitions. Let X denote a random sample and x its observed value; C(X)=(L(X),U(X)) is an interval-producing procedure. Its coverage is \mathbb P_\theta\{\theta\in C(X)\}, calculated over samples at fixed \theta. A posterior credible interval instead has probability content \int_{C(x)}p(\theta\mid x)\,d\theta under the specified model and prior.

1. What is random?

In frequentist inference, the parameter \theta is fixed but unknown. Before observing the sample, the data X are random, so the interval endpoints L(X) and U(X) are also random.

In Bayesian inference, uncertainty about \theta is described by its posterior distribution p(\theta\mid x). Once we condition on the observed data x, those data are treated as fixed and \theta has a posterior probability distribution.

2. Interpretation of a 95% confidence interval.

A 95% confidence-interval procedure satisfies \mathbb{P}_{\theta}\!\left\{L(X)<\theta<U(X)\right\}=0.95, where the probability is taken over repeated samples generated with the same fixed value of \theta. Therefore, in repeated sampling, 95% of the intervals produced by the procedure would contain the true parameter.

After observing the data, the realised interval is fixed. In the frequentist framework, it is therefore incorrect to say that there is a 95% probability that the fixed parameter lies in this particular observed interval.

3. Interpretation of a 95% credible interval.

A Bayesian 95% credible interval (a,b) satisfies \mathbb{P}(a<\theta<b\mid x)=0.95. Conditional on the observed data, model, and prior, there is 95% posterior probability that \theta lies between a and b.

4. Why identical endpoints do not imply identical meanings.

The two methods can produce the same numerical endpoints in some simple models, or approximately the same endpoints in large samples. Their interpretations remain different because they use different probability statements:

5. confidence refers to the long-run coverage of a random interval procedure under repeated sampling;

6. credibility refers to posterior probability for \theta conditional on the observed data and the assumed prior and model.

Key result. Exam-safe sentence. A 95% confidence interval has 95% long-run coverage under repeated sampling, whereas a 95% credible interval contains 95% of the posterior probability for the parameter given the observed data.