This paper presents a Human Partner System (HPS) that is composed of three sub-systems: a hearing system, a speaking system, and a vision system. Coordinated with these systems, HPS can recognize people and dialog with them. To attract a human's attention for interaction, this paper combines the difference image method and the GrabCut algorithm for HPS. To assist humans in their daily lives and to enhance their interactive fun, HPS has a short-term memory function and can read out new email when a known person appears in her line of sight. To allow HPS to search for a target object in a complicated environment, this paper proposes the double peak continuously adaptive mean shift (DPCAMS) algorithm, which combines the contours and color information of target object. This algorithm is easy to implement and the experimental result demonstrates that this algorithm is faster than the speeded up robust features (SURF) algorithm.