I purchased my copy of the English–language version of Chocolate – The Reference Standard at full face value at the Origin Chocolate event in Amsterdam in October 2015. It was not given to me as a review copy. In reading, I noticed a favorable mention of TheChocolateLife (p875). This did not influence my review. On a side note, it was the marketing department of my publisher, Gotham Books, who decided to include the phrase “… The Ultimate Guide …” as the tag line on the cover of my book, Discover Chocolate, over my objections. Sadly, to me, “ultimate” was prophetic in one respect – it is still the only book of its kind.
Chocolate - The Reference Standard
Overall Rating: Six pods (out of six) for sheer scale. One pod (out of six) for objectivity and reliability of information.
Processing stage: Unedited or lightly edited translation (Unconfirmed).
Price category: €€€€€
As Rick and Mike Mast have been so publicly reminded over the past few weeks, if you are going to make superlative claims for your product, you had better deliver on those claims. This is a lesson that Georg Bernardini, author of Chocolate – The Reference Standard (“TRS”), may well be forced to learn.
By deliberately renaming this edition of his book The Reference Standard (p14), the task the author set himself was not just to create a broad survey of available chocolate (4,400 individual products from 550 brands from 70 countries, according to the author: I did not count them) but also to ensure that the information presented as fact is, in fact, factually accurate. In other words, to create a volume that actually deserves to be held up as not just a reference standard, but as the reference standard.
With respect to the former task of creating a survey of currently available chocolate, Bernardini set himself an almost impossible task because no matter how comprehensive the attempt is or was, there were and are bound to be many companies overlooked or given short shrift, and much of the information, especially about the products and companies would be out of date by the time the book went to press.
Nonetheless, it is the scope and expansiveness where TRS is the most satisfying. Although I have been involved in chocolate professionally since 1998 and have been writing about and reviewing and rating chocolate since 2001, there are companies in this book I had never heard of before. Not “known about but never tried,” but genuinely never heard of before. The little trill of discovery when running across a new name is cool—it some rate an entry in my travel journal as a place to visit when I can.
The information about the companies is presented in a reasonably consistent and generally approachable and understandable fashion. It’s possible to skim the book looking for a known favorite brand (entries are arranged alphabetically and there is a listing up front) or to scan the book for companies that are rated highly (five or six pods) or that Bernardini is less sanguine about (none or one pods).
It’s this grazing aspect of the consuming the book that makes it fun but the fact that there is a whole lot to consume lends an unwarranted perception of value to the book; it is when you stop grazing and actually start examining TRS closely that some very real flaws reveal themselves.
Flaws, that in my opinion, make the book dangerous and its author not someone to trust, let alone laud.
To be fair, it’s hard for me to know from where many of the flaws stem, because I am not fluent in German. But when you start reading the book it’s quickly clear that after the book was translated there was none, or only very little, editing or fact checking done by anyone whose first language is English or who is knowledgeable about cocoa and chocolate.
TRS is littered with grammatical and typographical errors, and the awkwardly convoluted structure of many sentences clearly comes from TRS’s German–language origin. The first times I came across these, they struck me as amusing. Very quickly, however, the quirky sentences became annoying because they make trying to understand what the author actually wants to convey much more difficult and many times impossible.
More troubling, to my mind, is that there are some things that are presented as fact that are ambiguous or just plain wrong. And it’s here that my lack of understanding of German (and my unwillingness to fork over another €50 plus shipping for a German-language version of TRS) comes into play.
I just don’t know how many of the errors are in the German-language original or if they crept in during the translation. The translation may be the source of some specific jarring language choices that are not in the original. Two examples, gladly forego on the cover and the overuse (to my mind) of the words tolerate and suspect. (TRS is offered up as a reference standard, so nothing should be suspect. It should be verified and fact-checked as true, or it doesn’t warrant inclusion.)
I suspect (this is a review and I am not claiming it to be a definitive reference and I am going to use the word to highlight several points) that the source of some of the factual errors in TRS are a result of the translation, but I don’t know that this is the case in any specific instance.
For example, in the section on cocoa sourcing and the Dominican Republic (p34), the text reads, “The two most frequently cultivated varieties are Sanchez and the prestigious cocoa bean Hispaniola.” Actually, Sanchez and Hispaniola are not varieties of cacao, they are terms that refer to fermented beans (Hispaniola, from the name of the island), or the lack of fermentation (Sanchez, from the name of a port). The question is, is the source of the error the translation (I think not in this case), or is it actually a fundamental misunderstanding on the author’s part? If the latter, then that calls into question everything the author claims as fact: What does he really know? What can we trust? What can we take at face value as being true?
I don’t know.
And in this specific instance I am consciously committing the same act that lies at the heart of my main criticism of TRS and the one that undermines its credibility and any claims it has to authority: I am being lazy. I could easily reach out and find someone who owns the German-language original and ask. But I did not, in order to make the very particular important point that there are many places in TRS where Bernardini has been lazy, and dangerously so because of claiming the mantle of reference standard.
An egregious example of this laziness is in the entry for Perú’s Cacaosuyo (pp 239-40). Bernardini opines that the processing stage Cacaosuyo occupies is “Bean-to-Bar (Unconfirmed)”.
The text reads, “It is not quite sure whether the company actually manufactures the chocolate itself. Too often it is rumored that they are private label products … It is hard to believe that the company controls all steps from cultivation to manufacture … For this, the communication and transparency are too meager for me [emphasis added].” And in the Summary, “A little more communication and transparency on their website because, apart from a logo, there is nothing and it would do the credibility of the company good.”
The only way I can read this is that Bernardini relied on reports of rumors and a lack of information on their website to punish the company by questioning its integrity with the Unconfirmed label. Apparently, Georg did not actually take the time or make the effort necessary to find out for sure one way or another: he perpetuates rumors with innuendo. To what purpose? What does this say about Bernardini’s integrity?
Note: I have personally visited the Cacaosuyo factory in Lima and have seen the process from the bean to finished bars. I have not visited the farms, but have spoken extensively with Samir Giha about them.
The entry for Pacari is similarly lazy and dismissive, but here’s where the deep waters of editorial decision-making become murky when a competitive entry is examined closely.
Quite rightly, Bernardini recuses himself from writing the review for Ecuadorian chocolate company and Pacari competitor, Hoja Verde (four pods, pp440-43), because he points out that he consulted to them in 2013.
Notwithstanding this distancing, Hoja Verde, which does not make its own chocolate, is given four pages of editorial where Pacari, a much more highly-respected and better-known brand internationally, a brand that consistently places highly in international competitions where Hoja Verde does not, rates the same four pods but just two pages (pp641-42) and is given the reputation–questioning (Unconfirmed) status label.
Even Valrhona, arguably one of the five most important companies in the book, rates only two pages plus a paragraph (pp792-95). Bonnat gets two pages (pp220-22) and six pods; Cluizel, a shade over two pages (pp289-291) and the same four-pod rating as Hoja Verde; Domori two-and-a-half pages (pp339-241) and six pods. Utterly bafflingly, Felchlin rates zero pages though is mentioned in passing as one of the best, if not the best, private-label manufacturers in the world!
Given these direct observations of what did and not make the cut, I can’t help but wonder how much Bernardini’s involvement with Hoja Verde did actually factor into the hard–to–believe editorial decision to give them far more love than many far more important and deserving companies. As the publisher, responsibility lies solely in Bernardini’s hands.
Favoring Hoja Verde with so much unquestioning editorial makes no sense in a book that purports to be The Reference Standard with a focus on “the best … in the world.”
Note: I have not personally visited Pacari’s operations in Ecuador. However, I have contacted people who have visited Pacari over the course of years, who know what to look for, and whose integrity is above reproach.
There are other examples of this laziness, or suspected undisclosed bias, throughout the book. Patrice Chapon (for example) is also punished with the (Unconfirmed) label, and reading the lazy and superficial explanation leaves me wondering if there is something personal behind the review.
For me, this consistent pattern (barely–known companies being given a lot of coverage and well–known companies being overlooked entirely or having comparatively few products reviewed and rated) who are clearly not “the best in the world … [that] we would gladly forego” is a key factor that undermines both the credibility and authority of the book as there are no clear guidelines about what was included—other, I suspect, than what Bernardini could get his hands on to review.
And It makes me wonder if there are any other instances where editorial coverage was influenced for personal or business reasons. Was, for example, the Maison Boissier review influenced in any way by the full page ad for The Salon du Chocolat?
Why I Say TRS is a Dangerous Book
TRS is self–published, and hiring experienced and knowledgeable editors and fact checkers to review a book of this breadth would be a very expensive proposition. However, for a book that calls itself The Reference Standard, it is precisely at this point where the author/publisher has undermined his own efforts, let down his readers, and created a situation ripe for dangerous exploitation.
As was revealed during the unfolding Mast Brothers story, the people reporting the story took the claims the Brothers made at face value and, at least apparently, did no fact checking. This meant that no one methodically looked at and publicly challenged their claims to have (for example) created/invented/innovated the entire production pathway they used until the series of articles on DallasFood.org. The Brothers (deliberately and cynically in my mind) took advantage of the lack of knowledge of media covering them and the consuming public, and coupled with some strategic endorsements from chefs who probably should have known better, were able to advance their claim that they made the best chocolate in the world.
It is exactly this confluence—ignorance (of chocolate), gullibility (it’s such a huge book it must be valuable/good), and lack of critical questioning—that lulled media and organizations and individual that should have known better into endorsing (explicitly or by implcation) both The Reference Standard and Georg Bernardini.
This uncritical institutional acceptance only serves to give weight to the claim that the book is, in fact, deserving of its self-attribution as The Reference Standard. There are ideas and errors of omission and commission in TRS that will be perpetuated for years, and reputations called into question because Bernardini was either lazy or cheap in not editing the translation or fact checking very important facts, and possibly favoring at least one company over all others.
Despite these flaws and many others, people are citing the book as a credible and authoritative source. The fact that TRS is a print publication does a great deal to imply the credibility that to my mind it does not deserve; the book was out of date before it went to press; any errors due to mistranslation or other reason cannot be corrected or discussed. If the information were online it would be far more usable (assuming the database was searchable), though far less valuable – to Bernardini’s reputation. A point that I believe is not lost on the author (who is also the publisher).
In the end, readers of Chocolate – The Reference Standard should recognize that the ratings and reviews represent the opinions of a single person (with the exception of the troubling Hoja Verde entry). They are not gospel, the truth. They represent the opinion of one person. Your experiences tasting these chocolates will differ.
In part this is because not all products mentioned in this edition were rated specifically for this edition and may not represent the current state of the product, which may have been reformulated since being reviewed in a prior edition. Furthermore, among craft bean–to–bar chocolate makers especially, great pride is taken in the fact that their chocolate is not meant to be the same from batch to batch. Nowhere in any of the reviews of bean–to–bar chocolate I read did I notice any indication of which batch was tasted, even when that information would have been available. Thus, it is virtually guaranteed that whatever you taste it will not be what Bernardini tasted, reviewed, and rated.
And where is the reference standard value in that?
While we can marvel at the effort required to compile such a collection of entries in a very short period of time, it is also that effort in such a short time frame that undermines their reliability. We should not blind ourselves into believing that the quantity of effort involved is in any way equivalent to any qualitative aspect of that effort. I have pointed out just a few of those aspects above. There are many, many, more.
At best, TRS is a survey of a sampling of products from over 500 brands that Bernardini could get his hands on, and not, as the cover proclaims “The best chocolates and pralines in the world; What’s behind it [sic] all and what we would gladly forego”. If these are the best chocolates and pralines in the world – why would we gladly forego any of them? So, it’s important to recognize that TRS is a personal, idiosyncratic survey and sampling, one that because of its vastness is rife with errors of omission and commission, filled with factual errors, and that would be far more usable and useful if it were not distributed on dead trees.
I would like, in any comments, for members to focus on fact checking the book, not engaging in nit–picking the ratings and reviews of specific products, which, as I mention above, are completely personal. But – if there are factual errors in the book, I think everyone who owns a copy or refers to TRS as a reference standard should know about them. I can’t know everything and I don’t have the time to go through the book with a fine–toothed comb looking for them.
clay - http://www.thechocolatelife.com/clay/
updated by @clay: 01/31/16 05:18:18PM