Rater Calibration

Rater calibration requires that potential raters view and score videotaped School AMPS observations during the course and then complete 10 live observations of students in their typical classroom settings after the course. Rater calibration allows AMPS Project International to determine each rater’s severity and whether or not he or she is scoring the School AMPS in a valid and reliable manner.

To become a School AMPS Calibrated Rater, you must complete the following:

Complete a five-day School AMPS training course
Test 10 students who perform 2 or 3 School AMPS tasks each
- Interviewing each student's teacher before classroom observation is required
- Testing each student in his or her typical classroom setting is required
- At least 5 of the 10 students must be well.
- A classroom setting is one that includes the teacher and at least four students, including the one that the occupational therapist plans to observe and evaluate.
Independently evaluate "live" students; the use of video tapes is not acceptable
Two of the ten students may be co-scored. Coscoring is defined as two therapists observing a student at the same time, but scoring the student's performance independently.
Enter the data into the computer using the School AMPS computer-scoring program
Submit exported data to AMPS Project International using the Rater Calibration Center within three months of taking the course.

For detailed instructions, use the Rater Calibration Requirements pages of the course handouts.

You will receive a confirmation number immediately from the web browser when you have successfully submitted your data. If you do not receive a confirmation number, AMPS Project International did not receive your data. Review your data entry for accuracy. It is your responsibility to make sure your data is entered correctly and accurately. Errors may result in having to retake the course, test additional students, or pay additional fees. Do not submit backup copies of your data - only exported data will be accepted.

Once data is received, AMPS Project International will complete a detailed analysis of rater severity and therapist reliability as a School AMPS rater. When finished analyzing rater data, AMPS Project International will provide a detailed letter regarding the results of the analysis. The letter will be made available for download from the Rater Calibration Center within 2-3 weeks as indicated by the Rater Calibration Center when data is received. If the rater calibrates reliably, a calibration code specific to the rater's severity will be included in the letter. If the analysis reveals that the rater does not demonstrate valid and reliable scoring, he or she will be asked to test an additional ten clients or, in the rare case of extreme (and invalid) rater severity, to take the course again.

Submission of data for an additional ten clients provides the potential School AMPS user with an additional opportunity to learn to administer and score the School AMPS in a valid and reliable manner. Approximately 95% for those who resubmit calibrate successfully. Those who do not calibrate as valid and reliable raters with the second set of data must retake the course if they want to use the School AMPS in clinical practice or research.

If you need technical assistance with your software, please contact the AMPS Project International by e-mail: info@AMPSIntl.com.

This page was last modified on 11 January 2010