Reliability of adjudicators responsible for choral performance assessment has long concerned music educators. For example, Radocy (1989) argued, “any measure that involves human judgment is inherently subjective because it involves human impressions (p. 30).” He concluded music educators must recognize that all measurement procedures are inherently subjective, either in construction, application, or interpretation. To address such matters, numerous assessment forms have been employed to enhance consistency and reliability of performance adjudication. Recently, performance assessment rubrics, which contain narrative descriptions of various categories, have come into use. The purpose of this study was to longitudinally investigate the validity and reliability of one such rubric employed in the Large Group Choral Festivals in a Midwestern state in the United States. This paper/presentation will report choral adjudicator reliability and validity findings from 45 different adjudicators assessing 350 different choral performances, which occurred over two years. To that end, the following research questions will be addressed: (a) What was the level of agreement among choral adjudicators in assigning the global rating (that is, I, II, III, IV, V) when utilizing this rubric? (b) What was the level of agreement among choral adjudicators in assigning scores for individual performance categories (that is, tone, interpretation, rhythm, and so on) when utilizing this rubric? (c) What was the level of correlation between individual performance categories and global ratings, or specifically, which performance category tended to be the best indicator of the global score? And (d) what is the perceived efficacy of this performance assessment rubric among both adjudicators and directors?