Meet our international blog: Sisto Andriessen International
A randomised test can be totally fair but it can also be biased.
A test is biased when the results have consequences that unfairly advantage or disadvantage test takers.
Is it possible to determine whether a test is fair? Whether it is equally difficult for all candidates?
Yes it is. But only in hindsight.
An analysis of the average p-value of the test is of great help in establishing the fairness of the test. When the average p-values are spread across a broad range then it is highly likely that several tests had varying levels of difficulty.