When considering the space of all homogeneous matrix product states (with arbitrary boundary conditions) of a given bond dimension, one finds that the Hilbert space spanned by such states is much smaller than one might initially expect. Indeed, one can find many linear equations vanishing on this space using a beautiful connection to matrix algebras and polynomial identities of matrices. This connection was first pointed out by Werner in 2006, and later rediscovered by Navascues and Vertesi. In my talk I will give an introduction to polynomial identities of matrices, explain the connection to homogeneous matrix product states, and sketch an approach to obtain similar results for translation-invariant matrix product states with closed boundary conditions.