Bayesian credible interval

Definition taught in Bayesian foundations, part of Bayesian statistics

Bayesian credible interval

Two 95\% summaries of the same skewed \operatorname{Beta}(3,9) posterior. From Bayesian foundations.
Type Definition
Taught in Bayesian foundations
Part Bayesian statistics
Properties 1
Worked examples 2
Questions using it 12

Bayesian analysis introduces a probability distribution for the unknown parameter. Conditional on observed data x, a statement such as

\mathbb{P}\!\left(a(x)<\mu<b(x)\mid x\right)=0.95

is meaningful. The interval (a(x),b(x)) is a 95\% credible interval. This is not a reworded confidence interval: the two calculations can coincide in large samples, but their probability statements condition on different objects.

Properties

Posterior summaries from draws

Main article: Posterior summaries from draws

Let \theta^{(1)},\ldots,\theta^{(L)} be posterior draws and let \theta_{(1)}\leq\cdots\leq\theta_{(L)} be their order statistics. Then \begin{align*} \widehat{\mathbb{E}}(\theta\mid x) &=\frac1L\sum_{l=1}^L\theta^{(l)},\\ \widehat{\operatorname{Var}}(\theta\mid x) &=\frac1{L-1}\sum_{l=1}^L \left(\theta^{(l)}-\overline\theta\right)^2,\\ \widehat{\mathbb{P}}(\theta<c\mid x) &=\frac1L\sum_{l=1}^L\mathbf{1}\{\theta^{(l)}<c\}. \end{align*} The posterior median and equal-tailed interval are estimated by the corresponding empirical quantiles. With L=1000, the median can be estimated by the average of \theta_{(500)} and \theta_{(501)}, and a direct 95\% interval approximation by [\theta_{(25)},\theta_{(975)}]. The precise indices vary slightly with a software package’s quantile convention.

Examples

Q3. Confidence and credible intervals

Question page: Q3. Confidence and credible intervals

[6 marks]

Explain the difference between a frequentist confidence interval and a Bayesian credible interval. In your answer:

1. state what is regarded as random in each framework;

2. give the correct interpretation of a 95% confidence interval;

3. give the correct interpretation of a 95% credible interval; and

4. explain why the two intervals may sometimes have the same numerical endpoints but still have different interpretations.

Definitions. Let X denote a random sample and x its observed value; C(X)=(L(X),U(X)) is an interval-producing procedure. Its coverage is \mathbb P_\theta\{\theta\in C(X)\}, calculated over samples at fixed \theta. A posterior credible interval instead has probability content \int_{C(x)}p(\theta\mid x)\,d\theta under the specified model and prior.

1. What is random?

In frequentist inference, the parameter \theta is fixed but unknown. Before observing the sample, the data X are random, so the interval endpoints L(X) and U(X) are also random.

In Bayesian inference, uncertainty about \theta is described by its posterior distribution p(\theta\mid x). Once we condition on the observed data x, those data are treated as fixed and \theta has a posterior probability distribution.

2. Interpretation of a 95% confidence interval.

A 95% confidence-interval procedure satisfies \mathbb{P}_{\theta}\!\left\{L(X)<\theta<U(X)\right\}=0.95, where the probability is taken over repeated samples generated with the same fixed value of \theta. Therefore, in repeated sampling, 95% of the intervals produced by the procedure would contain the true parameter.

After observing the data, the realised interval is fixed. In the frequentist framework, it is therefore incorrect to say that there is a 95% probability that the fixed parameter lies in this particular observed interval.

3. Interpretation of a 95% credible interval.

A Bayesian 95% credible interval (a,b) satisfies \mathbb{P}(a<\theta<b\mid x)=0.95. Conditional on the observed data, model, and prior, there is 95% posterior probability that \theta lies between a and b.

4. Why identical endpoints do not imply identical meanings.

The two methods can produce the same numerical endpoints in some simple models, or approximately the same endpoints in large samples. Their interpretations remain different because they use different probability statements:

5. confidence refers to the long-run coverage of a random interval procedure under repeated sampling;

6. credibility refers to posterior probability for \theta conditional on the observed data and the assumed prior and model.

Key result. Exam-safe sentence. A 95% confidence interval has 95% long-run coverage under repeated sampling, whereas a 95% credible interval contains 95% of the posterior probability for the parameter given the observed data.

Q16. Interpreting identical interval endpoints

Question page: Q16. Interpreting identical interval endpoints

[4 marks]

A Bayesian analysis produces a 95% posterior credible interval [0.3,0.6] for a parameter \theta. A frequentist analysis happens to produce a 95% confidence interval with the same endpoints.

Explain the interpretation of each interval. Identify what is treated as random in each probability statement and explain why identical endpoints do not imply identical interpretations.

Definitions. C(X) denotes an interval computed from a random sample X; C({\boldsymbol{x}}) is its realised value. \mathbb P_\theta describes repeated sampling with \theta fixed. In contrast, \mathbb P(\,\cdot\mid{\boldsymbol{x}}) for the parameter is calculated from its posterior under the chosen model and prior. The stated 95% levels refer respectively to coverage and posterior probability.

For the Bayesian interval, \boxed{ \mathbb{P}(0.3\leq\theta\leq0.6\mid{\boldsymbol{x}})=0.95. } Conditional on the observed data, the assumed model, and the prior, there is 95% posterior probability that \theta lies between 0.3 and 0.6. The observed data are fixed, while uncertainty about \theta is represented by its posterior distribution.

For a frequentist 95% confidence-interval procedure C(X), \boxed{ \mathbb{P}_{\theta}\{\theta\in C(X)\}=0.95. } Here \theta is fixed and the interval is random because it is calculated from the random sample X. Under repeated sampling at the same true value of \theta, 95% of the intervals generated by the procedure would contain \theta. Once the data have been observed, the realised interval is fixed; the frequentist statement is not that this particular interval contains \theta with probability 0.95.

Therefore identical endpoints do not imply identical interpretations. The credible interval assigns posterior probability to values of \theta given the observed data, whereas the confidence interval describes the long-run coverage of a random interval-producing procedure.

Used in

See also

Confidence interval · Credible-region constructions · Marginal Student posterior · Bayes’ rule · Proper prior and proper posterior · Posterior point summaries · Posterior summaries from draws · R’s distribution interface

Glossary and index