Embedded processors are ubiquitous in today's system-on-chip design. In addition to designing digital signal processors (DSPs) for various applications, developing efficient test methods with little overhead and desired fault coverage for DSPs are also crucial and practical. Compared with the scan based test methods, the software-based self-test (SBST) method does not suffer from area overhead and performance degradation, and can provide at-speed test for DSPs with the potential drawbacks of lower fault coverage and a larger amount of test vectors. This paper explores techniques to improve the fault coverage of SBST methods for the developed DSP core with instructions fully compatible with those of the TI TMS320C54x. Experimental results exhibit that applying the developed SBST test flow obtains more than 96% fault coverage for our DSP core, which is higher than the reported values in related works.