Inside Rating Systems

There are many different approaches to rating chocolate. The most common is to assign on a number on a 1-10 or 1-100 scale. One challenge with this is that everyone who uses a numerical rating system uses a different algorithm to arrive at the final rating.This forum thread is for members to discuss how they rate chocolate.I am indebted to Casey for prompting me to start this thread.:: Clay


--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clay - http://www.thechocolatelife.com/clay/

updated by @clay: 04/19/15 08:56:07AM

This is cross posted from my blog, The Chocolate Note. There is a limit on length of forum replies, to read the entire post go here .While I respect the opinions of other writers and enjoy reading their reviews and celebrate the diversity within the field, like everyone I do have my own opinion.Although I use numbers, I do not use a weighting system in the same way chocolate is rated at 70%, as I feel that chocolate and science do not go hand in hand, except at the level where high art meets high science, in other words -- alchemy. Alchemy is what creates chocolate -- mother natures hand and the hand of an artisan together create that which we know as chocolate. Therefore, a true rating system created by a great alchemist would be far more complex than the little charts used by 70%.Perhaps neither I nor any other chocolate writer are actually up to the task. So what we are dealing with is opinions. Chocolate is a complex and mysterious elixir created by nature, and one that science readily admits that it knows nearly nothing of. Therefore I submit that chocolate is too big to fit those narrow categories in the weighting system.I do not use texture, snap, or appearance in my evaluations of chocolate. In my view these things are fun and interesting, a part of the packaging if you will, but not an essential component of what chocolate truly is. It seems to me that, while I may enjoy a good snap or sheen as much as the next person, they are almost a separate art form or category which should be considered as something different than the basic question of what is the true quality of a chocolate.I also believe that the same person can taste the same chocolate at different times, and depending on what they are really in the mood for, and other subjective factors, and come up with different ratings. This is just part of the mystery that is chocolate, and that is a person. Neither are machines. So I dont think a linear and empirical rating will quite work.The process by which I rate chocolate is still evolving. For chocolate bars that are not flavored, at this time I am giving a rating to the aroma, one to the taste experience, and another to the finish. I then take an average of the three. Right now I am playing with the idea of possibly giving each rating individually instead of taking the average of the three, but am at the moment still combining and giving one overall rating to each chocolate. For filled chocolates and flavored bars, I come up with one rating based on the overall experience.Perhaps it is simply a matter of my own personal right/left brain balance that I choose to use numbers, yet with a more subjective system. I say subjective in quotes because really, isnt it all subjective, isnt that what we are here for?Yet a word system might just be a little too subjective for my blood. I feel that whereas a very good could be so differently interpreted, an 8.6 is pretty clear. However, with respect to all kinds of minds, here is a rough translation:9-10 supreme/excellent/ superb/fantastic/heavenly/sublime8-9 very good/excellent7-8 good/very good6-7 pretty good/good5-6 okay/pretty good4-5 just there/okay/mediocre3-4 pretty bad/bad2-3 awful1-2 really disgusting0-1 inflicting injury/dangerous/poisonIf I am using numbers but not strictly using math to come up with them, then why do I use decimals? Why do I say 8.6 instead of just jumping from 8 to 9? Well, because sometimes chocolate is just better than an 8, but not really quite a 9. And I guess because my mind likes math, 8.5 is not good enough either, no, its closer to 9 than that, but not much.

Hi CaseyI enjoyed reading your thoughts on rating chocolate. It's a topic that deserves more attention. Outside of Seventypercent's site, I haven't seen any detailed and comprehensive lists. I like their system but it is somewhat cumbersome for me to use and like you, the bottom line for me is the taste experience. Their system is weighted though, and well thought out. To me, their ratings have done alot to have chocolate labeled and appreciated like other fine products.I think rating, comparing and taking notes is a great way to learn about chocolate. More useful, it has helped me discover what I prefer. I don't have a great taste memory so I started my notes as a form of a shopping list. It has taken alot of chocolate but my note taking skills have improved. Regardless of the notes on flavor, texture, etc., I like to write my short overall opinion of the bar and then assign a number rating. I use a number scale of 1-5.5- my absolute favorites4- purchase frequently3- purchase on occasion2- eat for free or buy to retest1- probably won't eat this againThe problem with my system is that with such few choices, I want to differentiate between the three's, fours, etc. and I start adding .25, .5, or .75 to the score. This usually is the result of comparing chocolates back to back. They both still belong in the 3 category but I do want to make a note that I prefer one over the other. It gets even more confusing when my taste's change and I decide to go back and start changing scores. Overall, the notes and lists are just for me and they serve the purpose of learning and creating a shopping list.I've used this system for a couple years and have tried to come up with ways to improve it. I haven't come up with a way I like better but am open to suggestions. It's interesting to factor in price of the chocolate (I only take notes on solid unflavored bars) but also a bit complicating. Price changes from store to store and fluctuates greatly with the packaging. It's hard to make a direct comparison so I haven't implemented this.

Lots of good points! I have *definitely* found that tasting the same chocolate at different times of day, in different surroundings, etc. affects the sense perceptions. Expectations also play a big part; I don't care much for the tasting notes I find on wrappers, and I usually get different results. Wybauw (in Fine Chocolates) makes the point that there is no such thing as an objective tasting. When your environment is positive, and your mood is good, you are more likely to enjoy something, and vice versa. In this sense, I think a chocolate's finish, snap, etc. are relevant, though secondary. Even package design. Intellectually they seem separate, but they definitely come into play in creating the impressions you get when you taste. One point I haven't seen brought up much (ok, I haven't exactly looked) is the thickness of a molded bar. Snapping a piece of a thin lil' Amedei bar and tasting it is a very different experience for me than..say, Dagoba? You know, one of the thicker, squatter bars.

The great subjectivity involved makes me shy away from a numerical rating system. Such a small snapshot can't convey much information, and complexity is the watchword of fine chocolate. I used to be more good/bad oriented, but as I worked with chocolate more my viewpoint changed. A chocolate I love to eat can be useless in a ganache. So is that a 5 or a 10? Some chocolate tastes like old leather, but that might be just what you want for a certain application. I prefer the flavor wheel that Felchlin uses (Chloe Doutre-Roussel has one like it in her book, as well). It gives you an idea what you're in for, and lets you decide for yourself how you feel about it. I guess I would add a section for Bland, though, to allow for the many chocolates that are.

[QUOTED]Perhaps we need to add a category for consistency of quality across a brand.[END QUOTED]Gwen: This last point is very interesting to me. I have long maintained that one of the key differences between a "good" chocolatier and a "great" chocolatier is the ability to produce consistently high quality work consistently (and no, thats not redundant). One of the reasons I write about chocolate rather than making it is that production doesn't interest me. Once I have mastered something I want to move on - not replicate it thousands of times. I have great admiration for people who know that yesterday they made 1000 of these pieces and today they have to make 1000 more and tomorrow they have to make 5000 because it's coming up on a holiday.And each piece has to be its own individual perfect little work of art.The ability to do this day in and day out, week in and week out, year in and year out separates the "merely" good from the great. I consider these people to be chocolate experts, which is one reason I don't call myself one and prefer to call myself a chocolate critic.But I have still to figure out how to capture this concept of good/great in a rating system transparently in a way that instantly makes sense.If you take a look at the system I developed prior to starting work on chocophile.com, in the end I wanted to quickly get to the answer to this question, "What sort of value is this chocolate (or chocolatier) for the price being asked?" It can be used to cover the good/great question, but it's not always obvious to people that this is part of what the approach does. A rating has three components:Category -(price range: mass market premium, gourmet, super premium, prestige)Style -(Belgian-influenced, French-influenced, American, Nouvelle American (also referred to as Modern French))Rating -(bad, poor, ordinary, good, very good, superior, extraordinary)So a chocolate might be in the gourmet price category ($25-$40/lb retail), be Belgian-influenced (lots of factors go into this component), and be an "ordinary" value. What that means is that, given the price and the style of work, there's nothing special that stands out to recommend this piece or company.It's possible to assign a value to an individual bar or piece or specific products, so sometimes rating components have more than one value. It's also possible to assign a value to a company. So, a chocolatier who achieves a superior or extraordinary rating is one that meets my "great" criteria and is shorthand for saying that all of the work falls into that category. Similarly, a chocolatier might be "very good" overall, but have some superior pieces; the reason the chocolatier is not given a "superior" rating is that the work is of inconsistent quality. Most chocolatiers do some things better than others, so this kind of rating (where the producer gets a lower rating than some of the work) is most common.There are other ways to do this that I am considering that borrow heavily from restaurant ratings, which is to give an overall rating and then list specific pieces that stand out (recommended eating) and those that don't (not recommended).I spent a lot of time on this, with the intent of providing a guide to help people find pieces that are like chocolates they already know they like (sort of like, "I like Szechuan food, is this restaurant any good?).


--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clay - http://www.thechocolatelife.com/clay/

This is a fascinating and informative discussion. I'm new to this, so I'm trying to learn all I can. In the last 18 months I've tasted and rated about 175 bars. I developed a rating scale similar to Seventypercent, but I used different weighting. Then I only let that number be a guide and I give two ratings: Class (69 or less, 70-79, 80-89, 90-100, and various flavors) and Overall. These are very subjective and for my own benefit. Also, my palate has changed a great deal over the months, so I probably need to revisit the earlier ratings. I also grade on the curve so a really great chocolate pushes the other down. That happened when I tasted Amedei Chuao. Finally, I've also developed a Value Index that attempts to compare the Overall Rating to Price (and has a factor trying to get it on a 1-100 scale.) It works pretty well for me. Oh yeah, I also categorize price as the price per 100 g, so I often have to convert the price for bars of a size other than 100g. This allows for standard comparison, though. Sometimes it's still worth it to pay top dollar, though (to splurge on Amedei Chuao, for instance).I'm looking for a way to publish my results, so I've experimented with Google pages. Here's a really rough one that I'm experimenting with. Go to http://chocolatereviews.googlepages.com/3400phinney-bread%26chocolate I'd appreciate any feedback. I understand and appreciate the critiques of assigning numbers and I know the pitfalls. I agree with most of what's been said, but this is mainly for my own use and it works for me, so I guess that's enough reason for me to keep doing it.

P.S.- How do you make a long url just appear as a hyperlink so that people just click on a word to get there without seeing the whole address?

Theo B:I suggest you look at an online service such as Zoho Creator's database tool to both gather the data and present it. Ultimately, you will find it much more interesting to do so. If you like, I can help you with it and if you need it hosted someplace I may be able to help you there, too.You can use a service like tinyurl or snipurl to generate short URLs to take the place of long ones.


--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clay - http://www.thechocolatelife.com/clay/

Casey,Your thoughts are excellent and very thought provoking. Here's a small question how did you make the hyperlink in this sentence "to read the entire post go here." so that the word "here" is a hyperlink to your blog instead of the whole URL? I've tried clicking on the "Add hyperlink" icon, but I can't make it work. Can someone please tell me what I'm doing wrong?

To add a hyperlink, you need to highlight the text that you want to attach the link to, then click on the link icon, then type in the URL of the page you want to link to.


--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clay - http://www.thechocolatelife.com/clay/

Thanks Clay.

Quote: "The problem with my system is that with such few choices, I want to differentiate between the three's, fours, etc. and I start adding .25, .5, or .75 to the score."Why not expand your ratings from 1-10? You're basically doing that anyway. In reality a 2.5 = 5 and a 4.5 = 9.Quote: "It's interesting to factor in price of the chocolate"Since 100g is a very common size for many bars I've been tracking the cost per 100g as a way for uniform comparison. In spite of the price varieties at different stores it seems to work well for me. I've also developed a "Value Index" which is a ratio of the Overall Rating I gave a bar to the price (times a weighting factor to get it on a 1-100 scale). That's helped me to figure out which chocolates are the best deal for the money. I've also been pleased with the results there too.