Markov Chain (Continuous Time)

1. Definition

t-step transition probability: Let $P_{ij}(t)$ be the probability that the system is in state j in t time units, given the system is in state i now.

$P_{ij}(t) = P(X(t+s) = j | X(s) = i)$
                 = $P(X(t) =j | X(0) = i)$  (by stationarity)

2. Properties

Lemma 6.2 $\lim_{t \to 0} \frac{1-P_{ii}(h)}{h} = v_i$

Lemma  6.2 b: $\lim_{h \to 0} = \frac{P_{ij}(h)}{h} = q_{ij} = v_i p_{ij}$

Lemma 6.3: 

3. Forward Chapman-Kolmogorov Equations

$P'_{ij}(t) = \sum_{k \neq j} q_{kj} P_{ik}(t) - v_j P_{ij}(t)$

Proof:

Define $q_{jj} = -v_j$