Preface
At times I wonder if I am the only person who has read the English–language version of Georg Bernardini's Chocolate—The Reference Standard critically because it seems like mine is the first review of any edition of TRS that is anything less than glowingly positive.
In his response to my review (which is now Part 1 of what will be a three-part series),
Georg writes:
The best chocolates and pralines in the world
What’s behind it all and what we can gladly forego
"Sorry that you don’t understand the phrase. Perhaps the translation is not correct [emphasis added], but the initial idea was not that the two phrases are in context.
"But I understand. You want to tear up the book and everything you find is good to massacre."
The fact that I don’t understand what the author and publisher actually meant is precisely my point: Difficult-to-understand translations are present on nearly every page and annoyingly confusing, detracting from TRS’s usability. The problem starts on the front cover.
Summary
"I don’t think that any of your accuses is correct. There are perhaps some delicate details which could be more clear, but in summary I don’t agree at all with your opinion."
I never expected Georg to agree with my observations (not opinions) but I did not write my review for Georg . He has said he will probably never update the book, never giving himself the opportunity to correct any of the points I bring up. So why bother?
My review was written for people who have purchased the book—or are thinking of purchasing the book—so that they can see that there are some serious concerns with a lot of what is presented as fact, along with editorial inconsistencies, and the reader might want to consider those concerns as they are reading the book.
All comments refer to the 2015 English–language edition, except where explicitly mentioned otherwise. I did not look at any prior edition of TRS to see if the issues I raise are in any other versions.
The Cultivation Countries pp27-37
Fine Flavor Cocoa Classification (p30)
Bernardini states “… ICCO classifies the cocoa and bindingly regulates [emphasis added] the percentage of cocoa which a country may export as fine flavor cocoa.”
This was not my understanding, so I reached out to someone who was at the ICCO Ad-Hoc Committee meeting in London this past September to confirm that the committee agrees upon a percentage of exports that can be considered as fine flavor as reported by the countries themselves . ICCO absolutely does not regulate exports .
A cocoa expert should be expected to know this. (As an aside, the numbers are from 2010 and at the September meeting some of the percentages were changed. Thus, the percentages reported in TRS are out of date but I don’t know if Georg could have known about them before TRS went to print.)
The Cocoa Varieties (pp27-28)
Bernardini says that there are four main varieties of cocoa: Criollo, Trinitario, Forastero, and the (Ecuadorian) Nacional sub-variety of Forastero.
We now know that not to be true. Perhaps the first most obvious place where the book was out of date before even the first edition was published , in my opinion, is that there is no mention of Juan Carlos Motamayor and team’s 2008 research paper, Geographic and Genetic Population Differentiation of the Amazonian Chocolate Tree (Theobroma cacao L) .
Motamayor’s paper mentions ten distinct varieties, and several more have been identified since then.
For Bernadini not to even mention Motamayor’s work is baffling to me (and to others to whom I mentioned this omission) in what aspires to be the reference standard.
Cocoa Aromas of Different Countries of Origin (pp32-36)
You mean, apart from the fact that the author conflates (or confuses) aroma (smell) with flavor? (Or is it a problem with the translation?)
The list omits Central America entirely : Nicaragua, Guatemala, Honduras, Costa Rica, Panama, El Salvador (Mexico was somehow transported to South America): other than Mexico (see S America) not one of these countries was found deserving of a discussion. How did an entire sub–continent go missing during the editorial review and fact–checking process?
The list omits several African growing countries: Cameroon, Nigeria, Tanzania, and Liberia.
In South America, Colombia is misspelled and has been ceded a new country: Mexico!
In Oceania, Fiji and Samoa are missing.
In the Caribbean, Haiti is not mentioned. St Lucia is not not only given more coverage than either the Dominican Republic or Grenada—both far more significant producers—but is treated fundamentally differently from every other entry in the section. Research into another question uncovered a possible reason for the length of the entry on St Lucia: The mention of a specific brand (glaringly, the only entry in this section to do so) that sources some cocoa from the island. Coppeneur (at which Bernardini worked) made (and may still make, as—lazy me—I did not try to find out) chocolate for Hotel Chocolat, the mentioned brand. I am not claiming this as any sort of proof of anything, I am just pointing out an undisclosed connection that could provide a reason this baffling editorial inconsistency.
Bernardini calls the carmelo cocoa grown on Rancho La Joya, Porcelana (p34), while labeling it extremely acidic. When discussing Venezuelan Porcelana (on the same page), he describes it as “absolutely not bitter, acidic or astringent.” One of the reasons for this is that carmelo is not genetically a pure Criollo like Porcelana. The term Porcelana refers to porcelain white, and is virtually synonymous with particular Criollos. While carmelo does have 100% white beans, it has been identified as a genetic amelonado, a fact confirmed to me by a member of the team who did the SNP analysis. One reason carmelo may be acidic is that it is not being processed properly post–harvest: It's being fermented as if it were a Criollo (which might account for excess citric acidity as well as a lactic tendency). But excess acetic acidity is almost certainly a result of poor drying practices, which Bernardini should know. I know this because I visited Rancho La Joya in December 2015 and saw their operation and asked what they were doing and tasted fresh pulp and beans.
Rancho La Joya is in the state of Tabasco whereas “the Xoconusco” is in the state of Chiapas, an important distinction to the people who live and farm there. Between them, Tabasco and Chiapas produced about 22,500MT of cocoa in the 2015 harvest according to the Minister of Agriculture for Tabasco, a figure I learned during a meeting with him during a trip there in December. It is estimated that less than 50MT of the combined total was exported as fine flavor cocoa—a fact corroborated by someone who arranges for the export of fine flavor cocoa from Mexico. This argues against the claim that the majority of cocoa grown in Mexico is mostly Criollo and Trinitario and not much Forastero. If they were growing a lot of high quality Criollos they’d be exporting a lot more of them.
Bernardini reports that the main variety of cocoa grown in Ecuador is Nacional (p33), but also Forastero. While there is mention of CCN-51 on the next page, it naïvely misrepresents the current situation with respect to production in Ecuador. During the recent ICCO ad-hoc meeting, Ecuador presented records showing that more than 25% of current exports were CCN-51. A colleague that I reached out to who works for a firm that exports large quantities of cocoa from Ecuador provided figures from INIAP that about 40% of what is currently produced in Ecuador is CCN-51, another 40% is Nacional, about 10% are Trinitarios, and the last 10% is other hybrids.
On p34, Bernardini fails to mention that a significant portion of the cocoa crop grown in Peru (and exported) is CCN-51.
Made up Terms (p34)
In his response to Part 1 of this review, Bernardini disagrees with me about the use of the terms Sanchez and Hispaniola with respect to so-called “varieties” of cacao grown in the Dominican Republic.
A quick survey on the topic with more than a dozen colleagues at the Salon du Chocolat in Paris last October and during the recent Fine Chocolate Industry Association Winter meeting suggests that he is alone in this belief.
On p27 Bernardini lays out the four main varieties. By using the word variety in his description of Sanchez and Hispaniola (and calling Hispaniola prestigious), Bernardini suggests to an unaware reader that Sanchez and Hispaniola can be used the same way as Criollo and Forastero, which is clearly not the case.
In his response he doubles down on his assertion. Bluster does not confirm the truth of something.
Idiosyncratic Translations
Why should readers be asked to parse sentences like, “On the other hand it looks after reduction of the still retained moisture and escape of undesirable flavors”(p49)?
If this were the only instance, it could be seen as a charming artifact. But there are way too many lazy translations and they quickly cross the line into being annoyingly confusing.
Questions about the quality of the translation are at the heart of one of my main criticisms of TRS. Bernardini mentions that he hired a translator. But just because he hired a translator does not mean he hired a translator whose background and skills were up to the challenge of a book like TRS. Was the translator a subject-matter expert on chocolate? Based on my less–than–superficial reading of the book I have to conclude, no.
The Concept of the Book (p17)
In his reply to the Part 1 of this review, Bernardini states:
“Sorry, I don’t think that there is any really important company overlooked. Give me a sample, please.”
Well yes, actually, there is one that jumps out very quickly: Felchlin .
Bernardini does give himself the out on this one (and points it out in his response to me). On p17, he explicitly states, “Only companies which are present in retail with their own brand are considered in this book.”
Does he mean present in retail at the time the book was published? Or at any point, ever?
Rules are meant to be broken, and Bernardini breaks this one on p281 with the inclusion of Chocovic (four pods). While Chocovic used to produce retail–branded products, they were purchased by Barry–Callebaut several years ago and no longer make chocolate bars for retail sale, a fact that Bernardini laments while including the ratings for products that can no longer be purchased. Why is Chocovic included?
He overlooks this rule even earlier, in the review of A Xoco by Anthon Berg (p144) in one of the single most confusing entries in the book—and it’s just the second entry!
A Xoco is listed as a partial bean-to-bar brand in the processing stage (also using couverture). Yet in the company portrait, he says that neither chocolates nor confectionery have been produced since 2014 .
“Even if I regard drageés as confectionery, as neither chocolate nor traditional confectionery is offered, I can no longer consider the brand in my book.”
He then goes on to list the reviews—from the first edition!—for six bars and three confectionery collections that are no longer available . A whole page is spent that could be put to better use on another brand whose products you can actually buy.
He overlooks this rule again on p405: Gnosis Chocolate has been out of active production since December 2014. (I had conversations about this with founder Vanessa Barg in December 2014 and in November 2015 here in NYC). There may be others.
“Quite assuredly, one will miss the one or the other brand in the book,” followed later on in the section by, “I had to draw a line in the case of 550 brands.”
We can wonder about that number and the selection of the brands and products, and it’s Georg’s book as he’s the author and the publisher, so he makes the rules. But we should recognize that the self–imposed limit is a limitation of the print format itself, nothing more, and the author argues against hIs own claim of being comprehensive (on the back cover).
“Every brand included in my book is portrayed—depending on how intensely it influences the market, in more or less detail. Brands of particularly good or bad products receive more attention independent of their size and significance for the chocolate market.”
In Part 1 of this review, I pointed out that Ecuadorian chocolate marketing company Hoja Verde (four pods, private label, made by Ecuatoriana, a large private–label producer in Ecuador) received four full pages of editorial whereas much more highly regarded companies—including Valrhona, Bonnat, Cluizel and Pralus to name just four—received half the love. I questioned the reason why .
I have mentioned this curious fact to many professionals in a position to know and the reaction I invariably got was, “Hoja Verde who ?”
Which returns us to the question of the selection criterion about “intensely influencing the market.” Hoja Verde influences the market intensely just how, precisely? Their products are middling—neither very good nor very bad—and it gets a four-pod rating. So, why so much love? It’s a reasonable question to ask.
I wondered if there could be any connection with the fact that Bernardini consulted to Hoja Verde. I am not offering this up as proof, just asking the question. It’s a reasonable question for a reviewer to ask.
Bernardini may say that he published guest reviewer Mark Christian's reviews unedited but that explanation falls flat given how concerned he is with the available space. He could easily have asked Mark to edit them. He asks us to trust him, there is no conflict of interest, but I have no way of confirming that. I am supposed to trust Georg and no one else ? Or the evidence of my own senses or experience as a published author? Why? On what evidence given how poorly much of the rest of the book was edited and fact checked?
“There will always be exceptions, and I cannot and do not wish to subject myself to a rigid rule [emphasis added].”
But, for The Reference Standard (and maybe the last edition its author will publish), don’t readers who are asked to depend upon the information being comprehensive and reliable deserve of the author a rigor in the selection criteria used for inclusion?
Isn’t it his job to communicate to readers clearly what the criteria behind his selections are?
The above statement illuminates why, in the author’s own words, I think TRS is an ambitious compendium, and not a good reference standard.
But in the end, it’s Bernardini’s book — his playground, his rules, and my lack of understanding about which companies were chosen and why is my concern, not his.
What I am doing is pointing out inconsistencies and questioning them, and considering whether these inconsistencies contribute to, or detract from, TRS’s claim to being the reference standard.
My opinion is that these inconsistencies result in a deeply flawed book. It cannot be taken at face value as being either comprehensive or accurate. It is a compendium that did take a lot of work, but some very important work was left undone.
Reviews and Ratings (pp142-837)
I have no intent to do an entry–by–entry fact checking of the reviews. That is an editorial task that requires many hundreds of hours of work that, quite frankly, Bernardini needs to hire editors and fact checkers to do. What I can do is point out a few “facts” that weren’t vetted properly. These may seem small, but they make my point that TRS was lazily edited and fact checked. A closer read will probably reveal hundreds of these errors.
Bernardini does mention in his response to Part 1 of this review two late changes that were caught before going to press. But catching two instances in no way proves that all of the information is up-to-date.
A Few Out of Date Facts That Were Not Caught
As has already been pointed out, Gnosis (p405) is no longer in production. Nor are A Xoco by Anton Berg and Chocovic.
Scharffen Berger no longer manufactures product in Berkeley, CA. In 2009, operations remaining in the Bay area after the 2005 sale were closed down and all manufacturing was taking place in Illinois.
TCHO no longer manufactures product in San Francisco. TCHO also purchased equipment from the Berkeley factory when Scharffen Berger closed down and moved, an interesting part of their origin story.
Stale Entries
The author makes a great deal of the number of brands and products reviewed. As he should – the book is a mammoth undertaking. But, again, is that labor everything it seems?
A close look reveals that not all of the products included in this edition were reviewed for this edition. Many were reviewed for the first edition. Anything not specifically tasted for this edition lacks tasting notes (pages!), and are thus incomplete and stale.
That would be okay (maybe) if there were just a few. But, perusing alphabetically from the beginning, the list includes:
A Xoco by Anthon Berg; Amatller; Amrani; Australian Homemade; Beschle; Bioart (partial); Bovetti; Butler’s; Cacaoyere; Café-Tasse; Charbonnel et Walker; Chococo; Chocolate Orgániko; Chocolove; Chocovic; and Coco Bruni. Sixteen companies whose reviews are implicitly labeled as stale (IMO) in the first three letters of the alphabet.
I did not have the patience to count through the entire 550 reviews, but a casual extrapolation suggests that there may be 100 or more entries— or 20% or more of the total —whose reviews and/or tasting notes were not updated for this edition.
This again speaks volumes for my thinking of the book as a compendium and not as the reference standard .
In his response to Part 1, Bernardini says, “
Again an error, Clay. Some batches are published (example: Brazen p227). … But who cares which batch was tasted? The reader? What the hell you think is usefull [sic] to him to know that I tasted 6, or 8, or 10 months ago Batch #40, Bar 32 of 44 from Brazen Bar Dominican Republic 70%??? … How many pages would I have to add if I note all the batches?”
Seriously? Let me unpack this for you:
Pointing out one instance I did not see and claiming that this proves his entire point is not proof.
If Bernardini thinks that the batch information is not useful, then why include it in any instance in the first case? ( I do agree that listing bar number figures is pretentious in most cases.)
Wines are vintaged because we know that they vary from harvest to harvest. Even mildly aware wine drinkers do not expect the 2010 vintage to taste the same as the 2011. Batch numbering is important because (especially in the case of small craft producers), the chocolate is very often not consistent from batch to batch. Everyone with a decent amount of experience tasting craft chocolate knows this. Many craft chocolate makers embrace this: It’s at the heart of my distinction between craft and industrial chocolate.
So, if someone reads a review of a small-batch craft chocolate maker based on a review, knowing the batch number signals to them that if they buy a bar from a different batch it might not taste exactly the same—or be even close.
For Georg not to understand this basic fact of chocolate demonstrates to me a pretty fundamental lack of understanding. And not to even recognize the validity of my raising the issue . Seriously, Georg? Really?
I have recent personal experience with this issue. I was in a specialty chocolate store in NYC and purchased 10 bars each of three chocolates as a part of a horizontal flight from the same origin. I was not paying attention and it did not occur to me that the store would stock a shelf with bars from different batches at the same time. When I got to the tasting (in a foreign country) I realized that I had bars from two different batches of “the same” chocolate. However, they tasted nothing alike: they were not even recognizably the same chocolate. If I had purchased one of the chocolates based on Bernardini’s reviews (not knowing the batch number), I might have been bitterly (literally) disappointed in what I bought. Thank goodness I know enough to understand why, but many consumers would not. BTW, a ll three chocolates were produced by companies that the author gives decent to outstanding ratings to in TRS and that have won awards in international competitions.
On a side note, this is a fundamental problem I have with virtually all bar ratings that get published. I mention this not in defense of Georg, but in a book that calls itself the reference standard I expected a less cavalier approach to the topic.
As I pointed out earlier, the limitations of length are an inherent limitation of the print format. To suggest that a reason not to include potentially valuable information is because of a lack of space undermines claims of being comprehensive and again begs the question of why many companies were included at all.
Ironically, if Georg had done a good job of editing the English translations from the German and fixed the awkward construction of many sentences it would have eliminated a lot of redundancy—freeing up more than enough space to include batch information in the ratings of bean-to-bar companies. On the other hand , using a construction in the Brazen review Bernardini offers up of Dark: Dominican Republic 70% (B#37) would not have changed the length of the book by a single page.
Lazy Editing in the Reviews
Some city names are not spelled in English (e.g., Belgrad [which is German, where Beograd, which is Serbian, and in English it would Belgrade]), where others are (Lucerne [Luzern], Cologne [Köln]). Consistency is important—either all English, or all in German, or all in the local tongue, but not a mix. This is what good editors do. What the publisher fails to want to understand is that these kinds of mistakes undermine the credibility of everything else in the book—for educated readers.
Some terms are just plain not translated (e.g., preiskategorie in the entry on Damian Alsop).
Apagey chocolate uses Barbarian [sic] sugar.
At least one entry lists a website URL and a Facebook URL. Why only one? Why not every company that has a Facebook page? Space? That's been addressed already.
The Rating System Used in TRS
Those of you who have been following my work on chocophile.com starting all the way back in 2001, on TheChocolateLife.com, and/or have read Discover Chocolate know that I am not a fan of numerical rating systems for chocolate.
Why is that?
In part it’s because I am partial to systems that make sense emotionally, that embrace the idea of “liking” a chocolate for reasons beyond a simple sensory evaluation. Such as, “Is this chocolate a good value for the price?” or “How fiercely do I like this chocolate?” These are concepts that are not conveyed by conventional numerical ratings.
Another part of the reason is that I struggled endlessly with where to establish the relationships between the different sensory characteristics that come into play. I asked myself (this was beginning back in 1998) “What percentage of the final score should flavor be? 25%? 20%? 40%? Aroma? Texture? Or technical aspects such as snap (hearing) and sheen (sight) which speak to how well a bar is made and attest to the skill of the maker in different ways?”
There is no standard around these weightings, making comparing ratings that assign different weights to variables or that have a different number of variables pretty much useless because an 87 in one rating is very likely calculated differently from an 87 in another rating system.
But a main part of the reason is, to this day, nearly 20 years on, I that I still ask myself: " What is the meaningful difference between an 87 rating and an 89 rating?"
What is the difference in perception created in a reader’s mind between an 89 rating and a 91 rating? Even though the 89 and the 91 are the same distance apart as the 87~89 rating?”
Why is it important? A 91 often leads to the mistaken impression that it is much better than 89 just because the leading digit is a 9 and not an 8. It's the psychological difference between getting a B+ and an A-. I have no good answers to these questions—and I suspect that they are not answerable in any satisfactory way.
Bernardini takes the inherent dysfunctionality of numerical rating systems as I point out to an illogical extreme: not only does he ask us to believe that there is a meaningful difference between fractional points (e.g., an 81.00 rating and an 81.10 rating) but that his palate is discriminating enough to reliably tell the difference .
He also asks us to believe that he can reliably distinguish the difference down to hundredths of a point! It is just not reasonable to ask anyone to accept that there is a meaningful difference between two chocolates rated 62.75 and 62.63 or that Georg has a palate that can reliably and repeatedly operate at that level of discrimination. (Examples of hundredth–point differences in ratings can be seen in the entry for Richart on p676.)
I am confident that this is actually an artifact of the math used to calculate the ratings, and not a reflection of any preternatural tasting ability on Bernardini’s part.
But it is precisely this very precision that implies a credibility that cannot exist except in the minds of unsophisticated and unquestioning readers.
Summary of Part 2
In his response to my review, Norbert Mergen-Metz effusively praises TRS, calling it a “master piece” [sic]. A dictionary definition of masterpiece is, “a work of outstanding artistry, skill, or workmanship.” There is an enormous amount of work involved in compiling, writing, editing, and publishing a book like TRS. But the amount of effort involved is not listed as a criterion for attaining masterpiece status.
There is no question, whatsoever, that Chocolate – The Reference Standard (“TRS”) represents an enormous amount of work. It is a not-close-to-being-comprehensive-compendium of many things chocolate and attempts to treat the subject holistically (literally from tree to mouth) at a scope that no other book I know of has done. For that effort the author needs to be acknowledged, and I did and do. But again, I caution that it is important not to confuse quantity with quality.
I have been approached privately by several people who have chastised me for the tone I used in my review and to say that the reviews in TRS corresponded with their own tastes and so they found them useful. But I am not concerned with TRS just for its reviews, and I acknowledge that the reviews serve a purpose and are a useful guide for some readers. O thers who have been in touch with me find the reviews less useful. YMMV – your mileage may vary.
Georg suggests that companies who feel slighted by his reviews should look at his comments as an opportunity to improve going forward (as if they should change their practices to meet Georg's standards). Georg should feel the same way about many of my comments—as constructive criticisms to address for future editions of TRS.
Except there probably won’t be any future editions. So, what he has published, warts and all, has the potential to become the reference standard purely by default .
And therein lies the danger. There is an awful lot of information in TRS that is inconsistent, incomplete, and just plain wrong. As the reference standard, these inconsistencies, partial truths, innuendo, and errors have the potential to become the truth as they are repeated on the Internet by people who don't know any better.
And that would be a very sad thing for chocolate.
I stand by my basic conclusion that the finished work product does reflect considerable laziness: laziness in editing the translation, laziness in basic fact checking, and laziness in not being internally consistent in the application of its own guidelines.
Stay tuned for Part 3 !
[Feb 2: Edited for typos, grammar, and clarity.]
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@DiscoverChoc
updated by @clay: 10/17/23 09:52:46
