Probability notation and random variables
Random variable vs. realization
$$ X : \Omega \to \mathbb{R}^d, \qquad x = X(\omega) \ \text{ for a specific outcome } \omega. $$
$X$ is the map; $x$ is one realized output.
Intuition: $X$ is a number not yet seen, $x$ is one already seen.
Example
Flip a coin: $\Omega=\{H,T\}$, $X(H)=1$, $X(T)=0$. $X$ is the variable; $x=1$ is what happened on a given flip.
Events vs. values: $P(X=x)$ vs. $p(x)$
$P(\cdot)$ always eats an event, a subset of outcomes, and returns a probability: a number in $[0,1]$. $\{X=x\}$ and $\{X\in A\}$ are events, so $P(X=x)$ and $P(X\in A)$ are well-formed; $P(x)$ on its own is not, since $x$ is a number, not an event.
$p(\cdot)$, lowercase, is a different object: the probability mass function (pmf, discrete $X$) or probability density function (pdf, continuous $X$), a function of a value:
$$ \text{discrete: } p(x) := P(X=x), \qquad \text{continuous: } P(X\in A) = \int_A p(x)\,dx. $$
$$ \text{discrete: } \textstyle\sum_x p(x)=1 \ \text{(itself a probability)}, \qquad \text{continuous: } \int p(x)\,dx=1, \ p(x) \text{ can exceed } 1. $$
A continuous density gives relative likelihood: $p(x)\,dx\approx P(X\in[x,x+dx])$, and $P(X=x)=0$ for every point $x$.
Intuition: $P$ measures sets, $p$ measures local concentration (made rigorous in Lebesgue measure and KL divergence).
Example
Fair coin: $p(1)=p(0)=\tfrac12$, each a genuine probability. Standard uniform $X\sim\text{Unif}[0,2]$: $p(x)=\tfrac12$ for $x\in[0,2]$, a density, and $P(X=1)=0$ even though $p(1)=\tfrac12>0$.
Random vectors: joint, marginal, conditional
Everything above extends to a random vector $X=(X_1,\dots,X_d)\in\mathbb{R}^d$, with a joint density $p(x)=p(x_1,\dots,x_d)$ over all $d$ coordinates at once.
Shapes
$$ \underbrace{x}_{d\times1}=(x_1,\dots,x_d), \qquad \underbrace{p(x)}_{\text{scalar}}=\underbrace{p(x_1,\dots,x_d)}_{\text{one joint density, }d\text{ arguments}}. $$
Intuition: $p$ takes the whole vector at once and hands back a single number; it is one function of $d$ variables, not $d$ separate functions.
Three operations connect a joint to smaller pieces:
$$ \text{marginal: } p(x_1)=\sum_{x_2}p(x_1,x_2) \text{ or } \int p(x_1,x_2)\,dx_2, \qquad \text{conditional: } p(y\mid x)=\frac{p(x,y)}{p(x)}, \qquad \text{independence: } p(x,y)=p(x)\,p(y). $$
Intuition: marginal sums a variable away, conditional restricts and renormalizes, independence means conditioning changes nothing.
Repeated conditioning gives the chain rule, true for any ordering of the variables:
$$ p(x_1,\dots,x_n) = \prod_{i=1}^n p(x_i\mid x_1,\dots,x_{i-1}). $$
Intuition: generate one coordinate at a time, each conditioned on what came before: the mechanism behind autoregressive models.
Example
Two coin flips, independent: $p(x_1,x_2)=p(x_1)p(x_2)=\tfrac14$ for every outcome. If instead $x_2$ always copies $x_1$: $p(x_2\mid x_1)=1$ when $x_2=x_1$ and $0$ otherwise, so $p(x_1,x_2)=p(x_1)p(x_2\mid x_1)=\tfrac12$ exactly when $x_1=x_2$, and $0$ otherwise; no independence, but the chain rule factorization still holds.
Expectation notation
$\mathbb{E}_{x\sim p}[f(x)]$ averages $f$ over $x$ drawn from $p$:
$$ \mathbb{E}_{x\sim p}[f(x)] = \sum_x p(x)f(x) \quad \text{or} \quad \int p(x)f(x)\,dx. $$
Intuition: the subscript names which distribution the randomness runs under: never drop it once two distributions are in play.
The data-generating distribution
Machine learning starts from a dataset, $\{x_i\}_{i=1}^n$ or $\{(x_i,y_i)\}_{i=1}^n$, and the standard working assumption is that these are i.i.d. (independent, identically distributed) samples from some fixed but unknown true distribution:
$$ x_i \overset{\text{i.i.d.}}{\sim} p_{\text{data}}(x), \qquad \text{or} \qquad (x_i,y_i)\overset{\text{i.i.d.}}{\sim} p_{\text{data}}(x,y). $$
$p_{\text{data}}$ is observed only through finitely many samples. A classifier approximates $p_{\text{data}}(y\mid x)$; a generative model approximates $p_{\text{data}}(x)$ itself.
Intuition: $p_{\text{data}}$ is the target; the dataset is a flashlight shining a few samples of light on it.