No.060

Criterion-Referenced and Norm-Referenced Testing

Is 18 out of 25 good? Compared with what?

AssessmentFoundation2 min read
In one line

Norm-referenced testing interprets a score by comparing it with other test takers’ scores; criterion-referenced testing interprets it against a defined body of skills or content, showing what a learner can and cannot yet do.

A learner gets a test back marked 18 out of 25 and asks, “Is that good?” The honest answer is that a number means little until you say what you are comparing it with.

A norm-referenced interpretation compares the score with other test takers’ scores: 18 might put this learner above most of the group. A criterion-referenced interpretation compares it with a defined area of skill or content. It says what the learner can and cannot yet do, such as write a clear email making a request, whatever anyone else scored. Robert Glaser named the contrast in 1963, arguing that teaching needs to know whether learners have reached a goal, not just where they rank.

The difference shapes how tests are built. Norm-referenced tests, used for selection and some proficiency purposes, need items that spread people out, so items nearly everyone gets right are usually dropped. Criterion-referenced tests keep such items if they sample the target skill, because a class where everyone succeeds is good news. Popham and Husek (1969) pointed out that traditional reliability statistics, which measure how consistent scores are, depend on that spread, so they can make a good criterion-referenced test look weak.

The line is not perfectly clean. Many tests are reported both ways, and deciding where to set a pass mark often draws on how real learners actually perform. Still, most classroom tests exist to answer criterion questions. Reporting them only as rankings hides the information that makes formative use possible.

In actionOne score, two reports

Norm-referenced report: 18/25. Higher than about three-quarters of the class. Criterion-referenced report: Can write a short email making a request: yes. Can tell a story in the past simple with few errors: yes. Can link ideas with although and however: not yet.

Both reports describe the same paper. Only the second tells the learner, and the teacher, what to work on next, which is why it suits formative assessment.

In the classroom

  1. 1

    Build each unit test from a short list of can-do statements, then report results against each statement (yes, nearly, not yet) instead of only a total or a rank.

  2. 2

    Keep easy items that check core skills. If everyone gets them right, that is evidence your teaching worked, not a sign the item is weak.

  3. 3

    Avoid grading classroom tests on a curve. If fixed shares of the class must get each grade, a strong group is marked down regardless of what it can do, and the washback rewards competing over helping.

Myth and reality

The myth

Norm-referenced and criterion-referenced are fixed labels for tests, so every test is simply one or the other.

What the evidence says

The labels mainly describe how scores are interpreted. A test is usually designed with one purpose in mind, but the same score can often be read both ways, and choosing the right reading is part of assessment literacy.

Reflect

If all your learners scored above 90% on your next unit test, would you conclude that the test had failed or that your teaching had worked, and why?

Worth a minute before your next lesson, or a conversation in the staffroom.

SourcesChecked against Crossref and library records

Where it comes from

Glaser introduced both terms in this short 1963 paper on instruction. Popham and Husek’s 1969 paper spelled out the measurement consequences. Hudson and Lynch (1984) were among the first to apply the approach to language testing.

Glaser, R. (1963). Instructional technology and the measurement of learning outcomes: Some questions. American Psychologist, 18(8), 519–521.

Key readings
  1. Popham, W. J., & Husek, T. R. (1969). Implications of criterion-referenced measurement. Journal of Educational Measurement, 6(1), 1–9.

    The classic paper on why tests meant to show mastery need different construction and statistics from tests meant to rank.

  2. Hudson, T., & Lynch, B. (1984). A criterion-referenced measurement approach to ESL achievement testing. Language Testing, 1(2), 171–201.

    An early, detailed case of building criterion-referenced achievement tests for an ESL programme, from the first volume of Language Testing.

  3. Brown, J. D., & Hudson, T. (2002). Criterion-referenced language testing. Cambridge University Press.

    The standard book-length guide for language teachers: writing, analysing, validating and reporting criterion-referenced tests.

  4. Sawaki, Y. (2016). Norm-referenced vs. criterion-referenced approach to assessment. In D. Tsagari & J. Banerjee (Eds.), Handbook of Second Language Assessment (pp. 45–60). De Gruyter Mouton.

    A clear, up-to-date overview of both approaches in language assessment, including where the boundary blurs.

ConnectionsSee it in the graph

Know a teacher who’d like this?

Send them the card. It’s made for Facebook, Zalo, LinkedIn and Instagram, and links back here.

Concepts
Criterion-Referenced and Norm-Referenced TestingNo. 060 · Assessment
Lexical DensityNo. 059 · SFL
Presentation–Practice–ProductionNo. 058 · Pedagogy
The Academic Word ListNo. 057 · Corpus
Postmethod PedagogyNo. 056 · Teacher Dev.
Code-SwitchingNo. 055 · Socioling.
Negotiation of MeaningNo. 054 · SLA
Dialogic TeachingNo. 053 · Discourse
Linguistic RelativityNo. 052 · Cognitive
Consequential ValidityNo. 051 · Assessment
Appraisal TheoryNo. 050 · SFL
The Lexical ApproachNo. 049 · Pedagogy
Lexical BundlesNo. 048 · Corpus
Action ResearchNo. 047 · Teacher Dev.
Linguistic ImperialismNo. 046 · Socioling.
The Interaction HypothesisNo. 045 · SLA
Display and Referential QuestionsNo. 044 · Discourse
Image SchemasNo. 043 · Cognitive
Assessment LiteracyNo. 042 · Assessment
Genre PedagogyNo. 041 · SFL
Communicative Language TeachingNo. 040 · Pedagogy
ConcordanceNo. 039 · Corpus
English as a Lingua FrancaNo. 038 · Socioling.
Comprehensible InputNo. 037 · SLA
Wait TimeNo. 036 · Discourse
Exploratory PracticeNo. 035 · Teacher Dev.
Thinking for SpeakingNo. 034 · Cognitive
The Validity ArgumentNo. 033 · Assessment
Grammatical MetaphorNo. 032 · SFL
The Involvement Load HypothesisNo. 031 · Pedagogy
Semantic ProsodyNo. 030 · Corpus
Native-SpeakerismNo. 029 · Socioling.
Processability TheoryNo. 028 · SLA
Epistemic Status and Epistemic StanceNo. 027 · Discourse
Teacher CognitionNo. 026 · Teacher Dev.
Construction GrammarNo. 025 · Cognitive
Data-Driven LearningNo. 024 · Corpus
The Teaching–Learning CycleNo. 023 · SFL
InvestmentNo. 022 · Socioling.
Dynamic AssessmentNo. 021 · Assessment
Focus on FormNo. 020 · Pedagogy
The Output HypothesisNo. 019 · SLA
The Apprenticeship of ObservationNo. 018 · Teacher Dev.
Prototype TheoryNo. 017 · Cognitive
The Idiom PrincipleNo. 016 · Corpus
Theme and RhemeNo. 015 · SFL
TranslanguagingNo. 014 · Socioling.
RecastsNo. 013 · Discourse
Formative AssessmentNo. 012 · Assessment
Task-Based Language TeachingNo. 011 · Pedagogy
The Noticing HypothesisNo. 010 · SLA
Reflective PracticeNo. 009 · Teacher Dev.
Conceptual MetaphorNo. 008 · Cognitive
CollocationNo. 007 · Corpus
Register: Field, Tenor and ModeNo. 006 · SFL
Communicative CompetenceNo. 005 · Socioling.
IRF / IRE SequenceNo. 004 · Discourse
WashbackNo. 003 · Assessment
ScaffoldingNo. 002 · Pedagogy
InterlanguageNo. 001 · SLA