Power assisting devices can be found in many different applications, most notably, exercise/fitness equipment, rehabilitation of spinal cord injury or stroke patients, limb muscle enhancement of workers, and bionic limbs of amputee patient. Since the power assisting device has a direct contact with the human user, in addition to providing suitable amount of assisting force, the highest priority in the design of power assisting devices is to ensure the safety of the human user. In order to fulfill such a requirement, in this paper an impedance controller and a passive velocity controller are combined to provide moderate amount of assisting force and also ensure the safety of the user simultaneously. Moreover, a torque observer is used to estimate the torque generated by the user's muscle rather than using a force sensor to measure the contact force between the user and the power assisting device. Experimental results verify the effectiveness of the observer-based impedance controller and the passive velocity controller.