This study proposes a new framework for controlling servo motor angle based on body gesture. In the proposed framework, a novel algorithm based on Kinect SDK processes the given image from camera; and generates commands for fitting the angle of servo motor in the right position. Further, for estimating the performance of proposed algorithm, this paper introduces a hardware structure, which is designed and implemented for generating standard empirical results. Experimental study on this hardware demonstrates that the proposed approach achieves superior performance to other state-of-the-art methods.