In the Assessment category on the National Counselor Examination (NCE), the Counselor Preparation Comprehensive Examination (CPCE), or another counseling exam, having a good understanding of reliability will be important.
You might wonder, what is reliability? We’ll cover that and review test-retest reliability, equivalent forms reliability, the split-half method, inter-rater reliability, and more.
In terms of assessments, reliability is whether a test can consistently produce similar results. Meaning that a person could take the test one week and get a score of 89. Assuming they have not put in a bunch of time studying or done anything else in that time, if they were to take the test again the week after and get a 90, this test would likely be seen as reliable because the results were roughly the same. Now, if they score an 89 one week and tested again the next and scored a 37, the reliability of this test would be questionable.
Reliability is, of course, an important thing for an assessment to have as you want to make sure you can rely on an assessment to provide stable results. Or, said another way, you want to be confident that the test will be consistent in its ability to measure whatever is being tested. Often times, when people talk about reliability, they will also discuss validity. Validity is whether a test is measuring what it is supposed to measure. If you want to learn more about that, make sure you check out my content on validity.
Something to remember with reliability and validity is that a reliable test is NOT always valid, but a valid test is always reliable. Again, a reliable test is NOT always valid, but a valid test is always reliable.
When it comes to reliability, there are multiple ways that reliability can be assessed. Let’s go over these.
Test-Retest Reliability: The first type of reliability to cover is test-retest reliability, sometimes just called retest reliability. With this, the test is given twice to the same group of people. Their scores between the two administrations are then compared and checked to make sure they are consistent. For example, if we created a test and gave it to a class, then had them take the same test two weeks later, we would expect a reliable test to have similar results between the two administrations.
Equivalent Forms Reliability: A second type of reliability is equivalent forms reliability. With this, the same group is given two versions of a test and the scores are correlated. This type of reliability can go by other names as well, which includes alternate forms reliability and parallel-forms reliability. An example of this type of reliability would be creating version A of an assessment and a version B that have similar test items. The scores between the two versions would then be correlated to make sure they are similar.
Split-Half Method: A third type of reliability is called the split-half method. With this, the entire test is split in half. This may be done randomly or split into even and odd questions. Either way, the questions should be split. Then, each test taker will be given both halves of the test. Once they’ve taken both halves, the results of each are then correlated. The closer the scores are, the greater the internal consistency of the test.
Inter-Rater Reliability: The last type of reliability we’ll cover is called inter-rater reliability. This type of reliability only applies to subjective tests or questions, such as an essay. For inter-rater reliability, the test taker will only complete the test once. Then, two different raters will grade the test. Once they’ve graded it, the scores they gave are compared. Ideally, those rating the test should have similar scores that they provided.
When the reliability of a test is being determined, this is measured by the reliability coefficient. This tells the error or variance. If the reliability coefficient is a positive 1.00, this would be perfect reliability. A reliability coefficient of above .80 is generally considered acceptable. When it comes to psychological tests, a reliability coefficient of .90 is considered good. What a .90 reliability coefficient means is that 90% of the score is considered accurate and 10% of the score is seen as error.
