Series of numerical study have for the numerical solution of the coaxial supersonic combustion and its stability. LES simulation of supersonic combustion flow fields necessitates the upwind-based high resolution schemes to capture the fine scale instabilities those might be dissipated with conventional schemes. High resolution schemes with fifth-order space discretization have been implemented. A validation study has been carried out for a coaxial supersonic combustor experiment. A numerical study is carried out to identify the instability capturing characteristics of high resolution schemes for compressible fuel-oxidizer mixing layer with combustion. Results show that the higher order schemes show clear improvement in capturing fine scale instabilities, while oMLP scheme exhibits unbeatable performance without a big expense. Parametric studies have been carried to investigate the effects of the combustor divergence angle and the length of the constant area combustor section on the stability and lift-off of supersonic flames.