Score system

[va2016]

I understand. However if the authors rate the quality of the reviews according to the rates given or criticism of their work, it kind of defeats the purpose of writing honest reviews in my view. I myself found parts of the system unfair and it discourages me.

The author's do not rate the reviews. It is based on other factors. At this time, the biggest thing that would negatively impact the editor's score is if a board warning is issued for a review that does not meet the guidelines or request a rewrite. In theory, average reviews should get a rating of 5 out of 10, and almost no reviewers if any should get a score of 10 out of 10 as that would be perfection and leave no room for improvement.

I'm thinking maybe the 'editorial' rating is subjective and thus can come off as offensive. Even a rating of above average but not perfect can come off as an insult. I'm not completely sure the best way to avoid that. Originally, the rating was supposed to default to 5 unless there was something for which to give an above average rating or a penalty recorded in the database (such as a review being rejected as not being spellchecked, etc.). I'm adjusting the formula so that the rating will tend to be higher unless there are explicit issues for which the score will be penalized, the idea being that lack of problem should be taken as more of a credit.

I was thinking about this a while back, and subtracting points for mistakes seems like a required
component in order to keep in check the reviewers getting slack, although I feel there could be
levels of mistakes (severe ones like clear spoilers, minor ones like a missing article), and corresponding
reduction of points.

I do believe that there can be objective assessment of giving additional points for reviews which stand out
above an average review, come up with the total score first, and then subtract points for the mistakes.

For example,

1) 50 points for an average review that covers the basics
2) add 10 points for say, reference to similar books and comparing this book with them.
3) add 10 points for say, paying good attention to grammar, punctuation, mechanical errors
and pointing them out.
4) add 10 points to say, language assessment (figures of speech, command over English etc) and
how the reviewer has recognized and written about them in the review.

5) subtract 2 points for say, a missing article
6) subtract 15 points for a severe mistake of not following the guideline
7) subtract 10 points for a clear spoiler

80 points - 27 points = 53 points

As of this writing, the scoring system has changed, and may be the editorial team
is already doing what I described, but just thought of putting that out. But, I observed
silly errors as well as the serious errors carry same reduction of 10 points, which I think
is not fair.

-- 21 Apr 2017, 05:24 --

I’d like to share some of my contemplations about the current formula for getting scores. When I saw it, I didn’t like the formulation “unlimited potential loss of points for penalties”. In my opinion, it should be defined more precisely (one or five or…? points for a reminder email, e.g.). I was really astounded when I rechecked my score and saw that from 12 points I went down to -4 (I got two reminder emails after which I reacted promptly). Does anyone else have similar experience?

It's possibly not because of the reminders, but because of a recent review that you submitted
which got negative editorial comments. You might want to check your scores on each reviewer
score sections thoroughly so that you know where the points went down.

[Scott]

The Review Team editing system has already been updated since this topic was created to make it more objective and clear.

This topic was created in 2014, so I will lock this topic.