A low-complexity divider-free architecture is proposed to implement the direct prediction technique of sub-pixel motion estimation. We investigate the characteristic of the error surface to reduce the computation complexity. The flatness of the error surface around the best integer-pixel motion vector (MV) position is measured to determine whether the prediction is skipped to save computation. The dividers are replaced by comparators to shorten the critical path. We also develop a simplified process to find the best prediction inside the well-condition error surface. Experimental results show that the performance of the proposed low-complexity sub-pixel MV prediction scheme is close to the conventional full interpolation-and-search method.