First off, thank you to everyone who provided constructive feedback about the new teacher rating system! We read, considered, and discussed all of the feedback we received and worked to find solutions to some of the main pain points. Here were the top 3 critical pieces of feedback we received and the solutions we’ve put in place for each one.
Problem: It’s not fair for students to rate 30-minute trials as they are not full lessons, but more of a meet and greet to decide if it’s the right student/teacher match.
Solution: Students may no longer rate 30-minute trials.
Problem: Students rate teachers poorly when the problem has to do with audio or video, which may not be the teacher’s fault.
Solution: Now students are first prompted to rate only audio and video like so:
This way, audio and video ratings are cognitively distanced from teacher rating.
If either the audio or video is rated below a 3, the "Teacher Rating" does not appear at all, and the "Continue" button will change to "Done". This is so to ensure that the student doesn't reflect poor audio/video in the teacher rating.
If audio and video are rated with a 3 or higher, then the "Teacher" rating does pop up. Like so:
Problem: If star ratings are anonymous, then teachers do not have the proper feedback needed to improve. However, lack of anonymity was causing inflated ratings, resulting in a useless rating system in which nearly everyone had 5 stars.
Solution: If the teacher rating is fewer than 4 stars, a mandatory box appears where students must explain to Verbling what the teacher should do to improve.
These comments come to us at Verbling. We reach out directly to the teachers who have had complaints. If the student has not provided a proper explanation for the low score, we investigate and possibly remove the rating.
How are average ratings calculated?
Lastly, we wanted to provide clarity about how rating averages are calculated. Ratings are weighted as follows:
5 Stars: 1
4 Stars: 0.75
3 Stars: 0.5
2 Stars: 0.25
1 Star: 0
If a teacher receives a 5, it is fully weighted. The lower it goes, the less the rating matters to the overall score. A 1 will not affect the average rating as it’s not counted. It’s a system designed to heavily favor good scores and remove outliers like 1s. Weighting averages is an industry standard.
The average rating is used in combination with a Wilson Score, or Binomial Proportion Confidence Interval, to determine how teachers are listed on the teacher listing page.