On the Fundamental Matrix of Finite State Markov Chains, its Eigensystem and its Relation to Hitting Times

For a finite state reversible and ergodic Markov chain we prove an intimate relationship between its fundamental matrix and its hitting time matrix. From this we derive hitting time identities. Relating the eigensystem of the fundamental matrix to the eigensystem of the transition matrix yields a new characterization of equivalence classes of states indicated by piecewise constant eigenvectors. Since the latter are used for spectral clustering the paper gives a hitting time interpretation for the resulting clusters.