Introduction

Active inference agents learn by observing the world and updating their beliefs. The mathematics that makes this possible follows a remarkably clean chain of generalization:

Each step is a single, clean generalization of the previous one. By the end, you will understand exactly why incrementing a count is the same as learning, why the Gamma function appears everywhere, and how Dirichlet distributions parameterize the tensors in a POMDP generative model.

This tutorial assumes familiarity with basic probability (random variables, expectation, distributions) as covered in the Probability Theory tutorial. It serves as a bridge to the Scale-Free Active Inference tutorial, where these distributions are composed into a hierarchical generative model.

Bernoulli Trials & the Binomial Distribution

The simplest possible random experiment: flip a coin. It lands heads (1) or tails (0). Nothing else. This is a Bernoulli trial.

The probability mass function is:

When x = 1: P(X = 1) = p. When x = 0: P(X = 0) = 1 − p. The formula encodes both cases in a single expression.

Repeating the experiment: the Binomial

Now perform n independent Bernoulli trials with the same probability p. The number of successes k follows a Binomial distribution.

This has two distinct parts, and understanding them separately is key to everything that follows:

What we know and what we want to know

The Binomial answers: “Given that I know p, what is the probability of seeing k successes in n trials?”

But in practice, we have the opposite problem: we observe k successes in n trials and want to learn p. We need to go from data to parameter. This is the inverse problem, and it requires a distribution over p itself — the Beta distribution.

The Beta Distribution

The Binomial treats p as a fixed, known constant. The Beta distribution treats p as a random variable — it represents our uncertainty about the coin's bias.

Immediately notice the structural parallel:

The shape part is almost identical: powers of p and (1 − p). The normalization part is where the Gamma function replaces the factorial. We will see exactly why in the Gamma Function section.

What α and β mean

The parameters α and β are called concentration parameters (or shape parameters). They encode our prior belief about the coin:

• α — the “pseudo-count” of successes we've observed (or believe we have)
• β — the “pseudo-count” of failures
• α + β — total “pseudo-observations.” This is the precision of our belief. Higher = more confident.
• α / (α + β) — the expected value of p. This is our best guess for the coin's bias.

Pólya's Urn: The Intuitive Picture

Here is the crucial property: after observing k successes and n − k failures, the updated urn contains (α + k) red balls and (β + n − k) blue balls. The posterior is:

Learning is just adding counts. No matrix inversions, no gradient descent, no optimization. Just add. This property has a name.

Conjugate Priors

The Beta distribution is conjugate to the Binomial likelihood. This means:

Why this matters

Without conjugacy, Bayesian updating requires computing:

The denominator P(data) = ∫ P(data | p) P(p) dp is an integral over all possible values of p. For most prior-likelihood combinations, this integral has no closed form — you need numerical methods (MCMC, variational inference, etc.).

With conjugacy, you never need to compute the integral. You already know the posterior's functional form (it's the same family as the prior), so you just update the parameters directly:

Concrete examples of conjugate pairs

The pattern: every conjugate pair has an update rule that reduces to incrementing counts or accumulating sufficient statistics. The functional form is preserved; only the parameters change.

The Gamma Function

The factorial n! = n × (n−1) × ... × 2 × 1 is only defined for non-negative integers. But the normalization constants of the Beta and Dirichlet distributions require something more general — a factorial that works for any positive real number.

The key properties

Property 1: It generalizes the factorial. For positive integers: Γ(n) = (n − 1)!

Note the shift: Γ(n) = (n − 1)!, not n!. This is a historical convention. Some authors define Π(z) = Γ(z + 1) = z! to avoid the off-by-one, but Γ is standard.

Property 2: The recursion relation.

This is the continuous analogue of n! = n × (n − 1)!. It means the Gamma function satisfies the same recursion as the factorial, but for all positive reals, not just integers.

Property 3: It handles half-integers.

The √π appearing here is not a coincidence — it connects the Gamma function to the Gaussian distribution. The normalization constant of the Gaussian involves √(2π), which is 2 · Γ(1/2)² / 2.

Why the Gamma function appears in distributions

Recall the Binomial's combinatorial component: C(n, k) = n! / (k!(n−k)!). The Beta distribution's normalization constant is:

Compare directly:

Anatomy of a Distribution

Before generalizing to K outcomes, let's crystallize the two-part structure we've seen. Every distribution in this chain has the same anatomy:

The normalization constant ensures the distribution integrates (or sums) to 1. It answers: “how many ways?” or “what fraction of the total?” It uses factorials (discrete) or Gamma functions (continuous).

The shape function encodes the probability of each specific outcome. It uses exponentiation: some base raised to a power that depends on the data and parameters.

Read each row left to right: the normalization generalizes via factorial → Gamma. The shape generalizes via products of powers. Reading down: 2 outcomes → K outcomes. This table IS the entire generalization chain.

Categorical & Multinomial Distributions

The Bernoulli has 2 outcomes. The Categorical distribution generalizes to K ≥ 2 outcomes with probabilities p₁, p₂, ..., p_K where Σ p_k = 1.

The bracket notation [x = k] equals 1 when x = k and 0 otherwise (the Iverson bracket). So for any particular outcome, exactly one p_kis selected.

Repeating the experiment: the Multinomial

Perform n independent Categorical trials. Count how many times each outcome occurs: k₁, k₂, ..., k_Kwith Σ k_i = n.

The Dirichlet Distribution

The Dirichlet distribution is a distribution over probability vectors — points on the (K−1)-simplex where all components are non-negative and sum to 1.

Compare directly with the Beta:

When K = 2 and p₂ = 1 − p₁, the Dirichlet reduces exactly to the Beta. The Beta is just a special case.

The concentration parameters

• α_k — pseudo-count for category k. How many times we've “observed” category k (including the prior).
• α₀ = Σ α_k — total pseudo-observations. The precision of the distribution. Higher = sharper, more concentrated around the mean.
• α_k / α₀ — expected value of p_k. Our best guess for the probability of category k.

Conjugacy: the Dirichlet-Categorical pair

Pólya's Urn with K colors

Tensors

A tensor is a multi-dimensional array of numbers. That's it. There is no additional mysticism.

Tensors come from physics (specifically, differential geometry and general relativity), where they describe quantities that transform in specific ways under coordinate changes. In machine learning and active inference, we use the word more loosely: a tensor is just a multi-dimensional array used to store parameters or data.

How tensors store probability distributions

Consider a matrix A with shape [3 observations × 4 states]:

Each column of A is a probability distribution (sums to 1). Column j answers: “If I'm in state j, what is the probability of each observation?”

Each column can be independently learned via a Dirichlet distribution. The Dirichlet concentrations for column j are stored in a parallel array concentrationsA[obs][j]. The expected probabilities are concentrationsA[obs][j] / sum_obs(concentrationsA[obs][j]).

A 3D tensor B with shape [states × states × actions] stores a probability distribution per (state, action) pair: “If I'm in state s and take action a, what is the probability of transitioning to each next state?” Each (s, a) slice is a separate distribution, separately learnable.

The POMDP Generative Model

We now have all the ingredients. A Partially Observable Markov Decision Process (POMDP) describes an agent that:

• Has hidden states it cannot directly observe
• Receives noisy observations related to the hidden states
• Takes actions that change the hidden states
• Has preferences about what observations it wants to experience

The generative model specifies how these elements relate. It is encoded as five tensors, each parameterized by Dirichlet distributions:

How learning works in the POMDP

The agent observes the world, infers which state it's in, and updates the relevant Dirichlet concentrations. Here is the complete learning algorithm:

From here to Scale-Free Active Inference

The Scale-Free Active Inference tutorial covers what happens next:

• Variational free energy — how to quantify how well the model fits the data (Section 3)
• Expected free energy — how to select actions that balance exploration and exploitation (Section 5)
• Coarse-graining — how to build a hierarchy where each level has its own POMDP with its own Dirichlet-parameterized tensors (Section 7)
• Boundary detection — how free energy spikes signal regime changes between levels (Section 9)

All of these build on the foundation established here: Dirichlet distributions parameterize the tensors, learning is count accumulation, and the Gamma function provides the normalization that makes everything integrate to 1.