Variable block size motion estimator (VBSME) for scan rate up-convertor (SRUC) based on the algorithm/architecture co-exploration (AAC) design methodology is presented in this paper. Due to the concurrent exploration of both algorithm and architecture, the designed system requires comparatively less computations and hardware cost but is capable of enhancing the accuracy of motion vector (MV) by refining MV from coarse-grained to fine-grained. The proposed algorithm generates the fine-grained MVs to produce the high quality results especially for the videos with high motion. Benefiting from AAC, we back-annotate the architectural information to algorithm to revise the proposed algorithm and then make the proposed algorithm be mapped onto the targeted platform smoothly. Hence, the SRUC system is able to convert the frame rate from 60 fps up to 120 fps at full HD (1920×1080) resolution was successfully implemented and verified on field-programmable array gate (FPGA). This SRUC system's performance has been shown to surpass those state-of-arts and its hardware cost is less than the related works as stated in the literature.