Multinomial selection for comparison with a standard

The multinomial selection problem is considered under the formulation of comparison with a standard, where each system is required to be compared to a single system, referred to as a “standard,” as well as to other alternative systems. The goal is to identify systems that are better than the standard, or to retain the standard when it is equal to or better than the other alternatives in terms of the probability to generate the largest or smallest performance measure. We derive new multinomial selection procedures for comparison with a standard to be applied in different scenarios, including exact small-sample procedure and approximate large-sample procedure. Empirical results and the proof are presented to demonstrate the statistical validity of our procedures. The tables of the procedure parameters and the corresponding exact probability of correct selection are also provided.