Suggestion for change in review scoring

[Anirudh Badri]

Hello All.

Currently, when the reviewer score is calculated, the system considers the current editorial review score to be the average of all review scores to date. As a result, this considers the score for all reviews equally, regardless of how long ago that review was.

If however, there was a look-back similar to the popularity score for the BOTD (where the editorial review score is the average of the most recent 4-5 reviews), the system would be much more reflective of the current performance of the reviewer. This would thus enable a reviewer who has improved dramatically and is now consistently scoring well to have a good score without a couple of bad scores from the beginning weighing them down. Conversely, it might also ensure that reviewers take special care before submitting a review because it is going to be much more relevant. This might prevent complacency because of previous performances.

Of course, I do not know the reasons for the current system, so it is possible that this suggestion is unfeasible or counterproductive. This is just a suggestion that I hope will make the community better. What do you all think?

[Doaa Wael]

yes I think taking the last couple of reviews 10 for example, is a good idea, but anyway, if you have a lot of good reviews towards the end, the bad ones in the beginning don't really affect ur score significantly

[Anirudh Badri]

It is not just about how much the old scores affect the current level. Whether that is beneficial or not, the fact remains that judging based on outdated scores is not ideal.

Of course, if there are other design considerations in play, that would be a major factor as well.

[Gingerbo0ks]

yes I think taking the last couple of reviews 10 for example, is a good idea, but anyway, if you have a lot of good reviews towards the end, the bad ones in the beginning don't really affect ur score significantly

I believe it does. I have a very low scoring review (not sure why it even ended up being published) from ages ago that is "dragging down" my average. I've been hitting 80s and 90s recently and this poor review is still affecting my score.

Not sure if I 100% agree the formula should be changed though.

[MsTri]

yes I think taking the last couple of reviews 10 for example, is a good idea, but anyway, if you have a lot of good reviews towards the end, the bad ones in the beginning don't really affect ur score significantly

I believe it does. I have a very low scoring review (not sure why it even ended up being published) from ages ago that is "dragging down" my average. I've been hitting 80s and 90s recently and this poor review is still affecting my score.

Not sure if I 100% agree the formula should be changed though.

I WOULD like to see the formula changed somewhat, perhaps only encompassing the last 10 at the most. My last several reviews have been in the mid-90s, yet I can't seem to get above an average of mid-80s, due to a couple of scores several reviews ago. I'm honestly starting to despair that all these great numbers will ever budge me from where I've seemed to settle.

[MrsCatInTheHat]

I WOULD like to see the formula changed somewhat, perhaps only encompassing the last 10 at the most. My last several reviews have been in the mid-90s, yet I can't seem to get above an average of mid-80s, due to a couple of scores several reviews ago. I'm honestly starting to despair that all these great numbers will ever budge me from where I've seemed to settle.[/color]

Mid-80s is a great reviewer score! You should be proud of your work. If the old ones didn't count, then those who did well early on would be on the same footing as those who did not; that doesn't seem particularly "fair" either.

[bookowlie]

Your suggestion should probably be discussed privately with Scott rather than publically airing an issue related to your personal review scores.

That being said, I thought Scott changed the formula last year so that reviews prior to a certain timeframe were given a little less weight in the average score.

[ashley_claire]

I'm kind of in the opposite boat. I've never been a consistent reviewer, and have gone months without doing a review. Ever since the formula changed for reviews it seems like I lost my previous scores, which is a shame because they were pretty good. Now it feels like I'm starting at the bottom again. But like I said, I've been inconsistent with the amount of reviews I do so I know I need to work on that.

[va2016]

Hello All.

Currently, when the reviewer score is calculated, the system considers the current editorial review score to be the average of all review scores to date. As a result, this considers the score for all reviews equally, regardless of how long ago that review was.

If however, there was a look-back similar to the popularity score for the BOTD (where the editorial review score is the average of the most recent 4-5 reviews), the system would be much more reflective of the current performance of the reviewer. This would thus enable a reviewer who has improved dramatically and is now consistently scoring well to have a good score without a couple of bad scores from the beginning weighing them down. Conversely, it might also ensure that reviewers take special care before submitting a review because it is going to be much more relevant. This might prevent complacency because of previous performances.

Of course, I do not know the reasons for the current system, so it is possible that this suggestion is unfeasible or counterproductive. This is just a suggestion that I hope will make the community better. What do you all think?

Totally agree! Suggest 10 or 15 reviews, as the count.

Thanks!