In this paper, a novel face recognition system using partitioned hidden Markov models is introduced to deal with partial occlusion problems. The proposed subface based system divides the face into forehead, eyes, nose, mouth, and chin, five subregions, which are characterized by five separated subface HMMs such that we can reconfigure these subface HMMs to achieve partially occluded face recognition. Moreover, we also suggested a facial grammar network to manipulate these subface HMMs to form various composite face HMMs. The Viterbi algorithm is used to estimate the likelihood score to perform face recognition with maximum likelihood criteria. Experiments are carried out on George Tech (GT) and AR facial databases. Experimental results reveal that the proposed system outperforms the embedded HMM (EHMM) and demonstrates promising abilities against partial occlusions and robustness against different facial expressions and illumination variations.