Score system

[gali]

What are you think about the new score system?

My score is fine, but I found some parts of it to be unfair to say the least. I also found a mistake in one part and PM Scott about it.

If it meant to encourage one to be more active, it achieved the opposite in my view.

[Scott]

If it meant to encourage one to be more active, it achieved the opposite in my view.

It is meant to encourage activity in the community and to rate quality of reviews, namely in terms of how desirable a publicly posted review from a certain reviewer would be to an author on average.

Authors give us books for free and even pay to have their books reviewed. There are many different reviewers looking to review books and new members joining every day. Authors want their book reviewed in part because they think having a recommendation from a trusted reviewer on this site would help them in getting others to read their book. A review from an established member carries more weight than one from a new member.

I would like to work with feedback to adjust it to make it more fair and respected system. This is why I have shown the score and a breakdown of how it is developed even though at this point it isn't affecting anything. In the future, I will use it to supplement the levels to solve the issue of their not being enough reviews to go around.

[gali]

I understand. However if the authors rate the quality of the reviews according to the rates given or criticism of their work, it kind of defeats the purpose of writing honest reviews in my view. I myself found parts of the system unfair and it discourages me.

[Scott]

I understand. However if the authors rate the quality of the reviews according to the rates given or criticism of their work, it kind of defeats the purpose of writing honest reviews in my view. I myself found parts of the system unfair and it discourages me.

The author's do not rate the reviews. It is based on other factors. At this time, the biggest thing that would negatively impact the editor's score is if a board warning is issued for a review that does not meet the guidelines or request a rewrite. In theory, average reviews should get a rating of 5 out of 10, and almost no reviewers if any should get a score of 10 out of 10 as that would be perfection and leave no room for improvement.

I'm thinking maybe the 'editorial' rating is subjective and thus can come off as offensive. Even a rating of above average but not perfect can come off as an insult. I'm not completely sure the best way to avoid that. Originally, the rating was supposed to default to 5 unless there was something for which to give an above average rating or a penalty recorded in the database (such as a review being rejected as not being spellchecked, etc.). I'm adjusting the formula so that the rating will tend to be higher unless there are explicit issues for which the score will be penalized, the idea being that lack of problem should be taken as more of a credit.

[gali]

May I ask what factors?

[Scott]

I edited my previous post to elaborate before the most recent reply. I am also editing the reviewer scoring page to provide more information.

[gali]

I understand. However if the authors rate the quality of the reviews according to the rates given or criticism of their work, it kind of defeats the purpose of writing honest reviews in my view. I myself found parts of the system unfair and it discourages me.

The author's do not rate the reviews. It is based on other factors. At this time, the biggest thing that would negatively impact the editor's score is if a board warning is issued for a review that does not meet the guidelines or request a rewrite. In theory, average reviews should get a rating of 5 out of 10, and almost no reviewers if any should get a score of 10 out of 10 as that would be perfection and leave no room for improvement.

I'm thinking maybe the 'editorial' rating is subjective and thus can come off as offensive. Even a rating of above average but not perfect can come off as an insult. I'm not completely sure the best way to avoid that. Originally, the rating was supposed to default to 5 unless there was something for which to give an above average rating or a penalty recorded in the database (such as a review being rejected as not being spellchecked, etc.). I'm adjusting the formula so that the rating will tend to be higher unless there are explicit issues for which the score will be penalized, the idea being that lack of problem should be taken as more of a credit.

Thank you. As you have said the 'editorial' rating comes off as an insult, unless you will find a way to avoid it.

[Adonis Maratos]

well currently it seems like we're back to the old system but it says I'm a level 1 even though I was a 4 (technically five but the private blurb hasn't yet been approved) before all the changes....

-- 01 Mar 2014, 11:51 --

oh wait no reviewer score is still active.... what?

-- 01 Mar 2014, 12:03 --

Last thing of all I feel like the participation on the book of the month thing is quite unfair..... Maybe I just don't wanna read the book.... or for example with Inferno, I live in Poland where i can find many books in English, but I can't for the life of me find Inferno, seriously I have looked in every book store I know. I could buy it online but then again it's a big effort and I have other books I'd rather read

[claire mckay]

Is the old system still in place as I've dropped from 4 to a 3? Also can I check if it's ok to put a link to your review on this site on good reads? I'm trying to improve my review views but don't want to go against any rules. I know we can't put the review on but then advice on how to boost views is to share link on social networking sites. Does that include an app like Good Reads?

I like the quality of review factor actually as I would trust the guys in charge to know the standards required and substandard reviews will affect the future quantity of reviews. However I'm not so keen on the heavy weighting put on book of the month and number of views. I don't like to pester my friends to view my reviews and would only like to participate in book of the month discussions and make posts when I am genuinely interested in them. I see a lot of meaningless posts e.g.people who haven't even read the book and I think the new system might encourage even more of these random posts just to improve their score.

Just my opinion though and you'll never keep everyone happy!

[Scott]

Is the old system still in place as I've dropped from 4 to a 3? Also can I check if it's ok to put a link to your review on this site on good reads? I'm trying to improve my review views but don't want to go against any rules. I know we can't put the review on but then advice on how to boost views is to share link on social networking sites. Does that include an app like Good Reads?

Yes, that's fine.

-- 05 Mar 2014 05:15 pm --

I hear what you guys are saying about the book of the month or even post count. The thing is even if you score 0 on those two, in theory you could still have high enough of a score to reach level 5. Thus, the scoring system offers a balance between those who can have a higher level/score because they have submitted a bunch of approved, quality reviews and newer members who would otherwise be suffering in a catch-22 because they can't submit new reviews because there level is so low they don't have any opportunities but they can't increase their level because they can't submit new reviews. The new system balances in not only previous official reviews, but overall contribution to the forums. So now new members can avoid the catch-22 by writing some quick reviews of books they have already read previously. We have had books of the month for years, so one can contribute there by posting in the topic/subforum for any previous book of the month that they happen to have read. To illustrate, if someone just joined the forum this past week, and has never read any of the many books that have ever been book of the month, and thus does not make any posts in that section, would an author really want such a person assigned to be their reviewer when the book could be assigned to someone who has read those books? Sure it isn't the end all be all measure of value as a reviewer, but it matters, doesn't it?

And that's really where I'm going with the scoring system. It takes into account many different factors that all matter but none of which should alone determine one's level. You could score a 0 on any measure and still reach level 5. But all else the same, I think the author submitting their book for free is getting more of what they are looking for if it is assigned to someone who has a higher score on any one of the factors.

I really appreciate the feedback. Keep it coming.

[jechamer]

Would it be possible to include the calculations on our levels page? I have a bad habit of having my score recalculated, but then forgetting to check what was included in the calculations that gives me ideas of what to work on to increase. Then of course, I can't go back in and have it recalculate as its too soon - I don't have a problem with your time restriction and understand the reasons why. I'm just looking for the ability to see the categories that I need to work on and how I'm doing on that.

Thanks.

[Scott]

I have made it so you can see the last breakdown of the calculation without rechecking the score on the reviewer scoring page. Thank you for the suggestion.

[Winter]

I just posted a review and got a message saying that my review was sent privately to the author as constructive criticism. I'm assuming this is because I gave the book a relatively low rating. I can appreciate that, but doesn't it mean that I will get no credit for the "Popularity of reviews based on # of views and # of replies" part of the rating system for this review, as it will not be made public? If so, this doesn't seem particularly fair. It also discourages giving negative reviews.

[npandit]

I have a question about the score system too.

What is "number of disputes over private blurbs"?

And also, how many number of views constitutes a 'popular' review? I'm not sure whether this is an entirely fair objective on which to deem a reviewer reliable, because for instance, if I wrote a review on a book about the fascinating history of yam production, I'm not sure many people would click on it. I think usually people just click on the titles that they think might interest them. This element of scoring may lead to unnecessary self-promotion, like "Hey! Everyone! Please read about how much I thought this book really sucked!" and not really be in the spirit of the book club.

Not meaning to nit-pick; just wondering. The rest of it seems fine. I like the ability to calculate my score, and the new way that the review page is organized.

[jechamer]

I have made it so you can see the last breakdown of the calculation without rechecking the score on the reviewer scoring page. Thank you for the suggestion.

Thanks for updating this Scott. I just went in and looked at it.