Last week I was made aware of a web site called GuideGuide.com by Shawn Askinosie. The About page says that the site's mission is to "provide the worlds largest and most reliable source of information on the health, environmental, and social impacts of consumer products." (Note: Shawn objected to the ratings of his products and they are no longer on the GoodGuide web site, pending review I assume. The observations that follow are based on notes I took before the listings were removed.)

GoodGuide claims that they have more than 100,000 products in their database and an extensive roster of scientists who review every product (or, more likely, set the criteria for what "good" is and then computer programs parse data about the products and come up with ratings).

There is a problem, though with the ratings methodology, in my mind.

A great deal of weight for the environmental and social rankings is placed on the presence of eco-label certifications. No organic certification? Then you get bad environmental ranking. No social certification? Then you get bad society ranking. It doesn't matter what other programs a company might have in place. It doesn't matter what actual benefit a company delivers to its suppliers.

All that matters is the certifications.

If there is a chocolate company on the planet that - measured by dollar of revenue or per employee - delivers more positive impact to the communities it works with than Askinosie, I'd like to know about it. I've traveled with Shawn on bean buying trips and knew about the way he wanted to do business before he made his first bar of chocolate. We've talked regularly since we first met in mid-2005 about how he runs his business, influenced by Jack Stack's A Stake in the Outcomeand his practice of open-book accounting and profit sharing with the communities he buys from. Shawn takes personal responsibility for doing what he knows to be the right thing to do.

Let's take a chocolate company that got higher Good Guide ratings, TCHO. One of the things that has never been clear (to me - and I know the founder and major funders) about TCHO is how closely the actual manufacturing processes match the claims on their web site. I am pretty confident that they have never manufactured any significant amount of product - from the bean - in the factory on the pier in San Francisco. They can't fire up their roaster and I don't know anyone who's seen the entire line in operation. Their TchoSource program is interesting, but it's meaningless unless they can stay in business - probably one of the reasons that the funders ousted founder Timothy Childs. They've always had a small lab producing their beta samples and pre-production runs so they can claim to be manufacturing bean-to-bar on the pier, but the quantities are not commercial quantities.

But TCHO has several products that are made with both FT and organic certified beans. They get great marks on the environmental and social scale, while those that are not get much lower marks (which is to be expected).

Askinosie uses packaging that is quite environmentally friendly. TCHO uses less environmentally friendly paperboard and gold foil imprinting for their boxes. But there is no place in the Good Guide methodology for any chance to think about the marketing messages and other aspects of the companies' businesses - like packaging and ethics - affect factors like environment and society.

Jif peanut butter got higher ratings than Askinosie. Is that a "good" thing?

The Good Guide health rating for chocolate is dominated by one factor - the saturated fat in chocolate. What the scientists fail to recognize is that the triglyceride structure of the saturated fat in chocolate makes it a "healthy" fat (like olive oil and coconut oil) and not a "bad" fat like saturated animal fats. (Specifically, "the nutritional value of the food, as characterized by a standard method of nutrient assessment called the Ratio of Recommended to Restricted Nutrients (RRR)." Easy to write a program to do that, hard to take into account differentiations that make a meaningful difference.)

While there are issues with the density of calories in chocolate and the percentage of calories that come from sweeteners, the Good Guide has no place in its ratings methodology for the growing science into the very real health benefits of chocolate: they make no meaningfully useful distinction between chocolate (i.e., a bar of chocolate from a craft producer) and candy (i.e., a Snickers bars) in their ratings.

Their shopping tips for buying chocolate bars are way too general on the issues of child and slave labor, entirely misrepresent issues in traceability, and confuse the need for fertilizer with the use of pesticides.

They also place way too much emphasis on certifications saying that, "Certifications ensure the chocolate has been produced under industry leading labor and environmental conditions." I BEG YOUR PARDON? You're confusing cacao growers and chocolate makers here in a way that is not defensible.

Which, for me, these issues make the entire Good Guide methodology for all products suspect. I can tell you that I will never be using their service and will never recommend it to anyone, either.

If that weren't enough, I received a confusing press release from a magazine called Organic Monitor over the weekend. Organic Monitor is hosting the Sustainable Foods Summit in San Francisco next week (January 17-18). I won't be attending as members of the press who are not employed by the media sponsors of the event must pay to attend (though the rate is discounted.)

In taking a look at the list of presentations one thing becomes instantly clear: There are no producers making presentations. The only people talking are the companies and organizations who have made millions of dollars successfully selling eco-labeling.

The absurdity of the situation was further highlighted in the press release, in which Organic Monitor posits that consumers are becoming disillusioned and confused by what organic certification really means and what the meaningful differences are between FLO Fairtrade, FTUSA Fairtrade, Rainforest Alliance, Utz, Fair for Life, and more.

Organic Monitor suggests that mobile apps that enable consumers to use QR (quick response) codes and other techniques to look up information about products as they are making their purchases and make better, more informed, decisions.

They point to Good Guide as being an exemplar of the way the industry seems to be going.

But wait. Don't the Good Guide ratings rely on the very eco-labels Organic Monitor predicts they are going to supplant?

What's an informed chocolate consumer supposed to do?

1) Buy from small producers who work you like and that you can get to know personally or through close friends and/or people you trust.

2) Don't blindly assume that because a product is organic means that it was produced in an environmentally sustainable fashion.

3) Don't assume that "Fair" trade means that smallholder farmers are treated fairly. Even when organized, growers lack the economic leverage to ensure they receive the benefits the fair trade certification process says that they are entitled to and that they have paid for. AND, remember, all of the "Fair" trade certifiers are non-profit organizations that rely heavily on industry support. Kraft pays dearly for the use of the Fairtrade label on their bars and probably pays nearly as much in licensing fees as it does in premiums. That is a fundamental conflict of interest that people blithely ignore - to the detriment of producers.

In other words - do your homework and develop the personal relationships necessary to understand what is really going on. Certification is a huge business with the backing of huge corporations. Follow the money. Who controls the economic leverage?

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clay - http://www.thechocolatelife.com/clay/

updated by @clay: 04/10/15 09:52:34

Clay thanks for highlighting the weakness of the certification programs. My partner and I own a cacao farm in Ecuador. We have gone back and forth on if we want to pursue fair trade certification/rain forest alliance. So far that answer has been no as we haven't needed it from a marketing standpoint so the hassle is not worth the time and money. I like the idea of a certificate to show that we are good citizens, treat our workers well, areenvironmentallyfriendly, etc etc. However I feel like many of them are so fundamentally flawed as to be worthless.

-Ethan

Ignore the money. Follow the cacao. In the same way that buy local, locavore, farmers markets have become big in an attempt to move away from the certification /big agro mess and muddle, why can't artisan chocolate makers do the same? Lead the way for others: educate, stimulate, generate!!! Connect with the farmers, or the co-ops, or at least get as close to the source as you can. If you buy beans buy them from someone who has gone to the farm, touched the soil, heard the birds. Certification is just certification - how many of us really trust the certificates we have? I think it must be time for the local connections to penetrate the artisan chocolate world.This last week I've heard from two people who want to develop beans specifically for individuals or small businesses. Who are looking to form personal connections with their clients. I'm not anywhere in the league of you all out there who have the experience and the developed taste to know the different varieties and nuances of beans. And I don't know how big a part that is of your businessess. I do know the difference between a bean that's been fermented for different lengths and under different conditions, and what that does to the chocolate. And I do know a bean that has been grown in healthy, vital conditions.What is important to you all? What are you looking for? If the artisan producers start dictating to and educating - and paying - for better quality beans from small producers, they'll get them. Connection is the key. And only key.

Thanks for the astute analysis Clay. One of the main points you make is all about marketing-the big guys have the money and voice to shout, shout, shout about how great their products, certs. etc. are, and if you say something enough times unfortunately it often becomes "truth" even though it may not be. It's going to be a long slog but for those of us who are the "little guys" with direct connections to farmers, producers, and small scale chocolatiers, we know it's the only way to go. Direct connections/relationships are the key.

I'm the Chief Scientist at GoodGuide and want to respond to the methodological issues you raise about our ratings of chocolate products.

1) Role of eco-certifications
GoodGuide uses third-party certifications throughout its product and company ratings, because third-party efforts to validate performance against environmental or social standards typically represent the strongest evidence available for identifying the best products/companies in a category. In the absence of certifications, the preponderance of evidence about a product or company is comprised of whatever information a company opts to make public via its website, CSR reports or marketing. As I'm sure you would agree, company-provided information rarely discloses the negative aspects of operations or products, and can often be characterized as "greenwashing." While third-party certification services vary in what attributes they cover, the stringency of their standards, and their degree of authoritativeness, I think there is no question that they generally produce more credible evidence of "goodness" than a manufacturer's unverified self-assertions. Your alternative approach - that consumers should "develop the personal relationships necessary to understand what is really going on" with each producer - is simply not realistic for most consumers. Most purchasing decisions are made by consumers who have have limited bandwidth for identifying which products match their social or environmental preferences. GoodGuide is designed to be a service for these consumers - they rarely research the production context of their purchases, and want actionable decision support at the point of purchase. Few are going to conduct independent research, or establish personal relationships with producers.

At GoodGuide, we review the certifiers active in any given product category, determine what issues a system covers, assess the stringency of a certification and the proportion of a manufacturer's catalogue or a product's components that are certified. As a result, different certifications impact a company or product score in ways that are influenced by our overall judgement of a system. We are transparent about how we judge different certifiers (see http://www.goodguide.com/categories/255760-candy##btr for chocolate and http://www.goodguide.com/categories/255779-coffee##btr for coffee). We are open to feedback criticizing the relative weight we assign to different certifiers on different issues, and we are aware there is a lot of controversy currently about changes in the certification landscape such as those associated with Free Trade labels. GoodGuide definitely does not see itself as replacing the good work being done by third-party certifiers - we are not a standard-setting or auditing entity, we are an aggregator and integrator of the public information available on products or companies. From our perspective, the most important limitation of certifiers is that they typically address only a very small percentage of the products in a consumer product category - they can help identify the best performers, but they don't help distinguish between the majority of products that a consumer finds on a retail shelf.

2) Penalizing products that lack certification
Your comments (and separate emails from Shawn Askinosie) have raised a legitimate issue regarding whether it is appropriate to penalize a product with a lower score if it lacks certification. In most product categories, GoodGuide rewards a product that has a certification with a higher score. However, in the case of chocolates and coffee, our methodology penalizes products that lack certification in addition to rewarding products that have certifications. We provide the following explanation for this practice: "While there are some gradations between the least stringent and the most stringent certifications, all of them represent a clear step above coffees [or chocolates] that do not have any certification in an environmental or social context. As a result, products that do not have any certifications generally earn lower scores. Certifications provide credible evidence to consumers that a product has been produced in accordance with basic environmental and social standards. The absence of a certification does not mean that a product was grown under unacceptable conditions, but the consumer has no way of knowing what practices were utilized to produce the commodity they are purchasing."

This practice can result in inaccurate scores if a company operates with sustainable practices but does not seek verification of these practices by third-party certifiers. Askinosie maintains that its environmental and social performance exceed the requirements of chocolate certification systems and that the company and its suppliers have legitimate reasons for not engaging third-party certifiers. We are currently considering how to revise our system so that well-documented but self-asserted performance information can displace the default negative score we assign due to the absence of certifications.

3) Health impacts of chocolate ingredients

GoodGuide uses a standard scientific method for evaluating the nutritional value of different foods. The Ratio of Recommended to Restricted Nutrients (RRR) is a measure of nutrient density designed to apply across multiple types of food. The goal is to utilize a measure that will help people select foods both within categories and across categories, so it necessarily focuses on a defined sub-set of nutritional components. You are correct that there is recent research suggesting that not all saturated fatty acids are created equal. However, this research is in its infancy and there is little information on the percentage of each type of saturated fatty acid in different foods. In the absence of detailed saturated fat composition data across categories, we are unable to include this variable in our ratings. Note that this is not as simple an issue as your blog post implies. The existing information on fatty acid composition (from the USDA Nutrient Database) of chocolate vs. olive oil vs coconut oil indicates that these three products are quite different. Calling the saturated fat in chocolate "healthy" by comparing it to these other fats would be a stretch. Olive oil is primarily comprised of monounsaturated fat (and very little saturated fat, in fact). Coconut oil, on the other hand, is primarily comprised of saturated fat - specifically lauric acid, a medium-chain triglyceride that may not be as harmful as other, longer-chain saturated fatty acids. Chocolate/cocoa butter is also primarily comprised of saturated fatty acids - specifically palmitic and stearic acids. Although stearic acid may not be as bad as other saturated fats, palmitic acid has not been exonerated at all. Again, detailed research on saturated fatty acids is in its infancy - so much so that no major worldwide health organization has revised its position on limiting saturated fat in diets. While we recognize the distinction between different types of saturated fatty acids, we do not yet believe there is enough evidence or data to incorporate this distinction into our rating system.

Bill, thanks for coming on the boards and replying. I learned a lot from your post. I appreciate your position and I actually ran across your website probably a year ago and have used it from time to time. I think you guys do a great job with what you can reasonably do and I agree that most consumers don't have enough bandwidth to learn about each purchase they are planing. I suffer from bandwidth problems myself and yet I want people to look into their chocolate more. Hypocrisy I know.

Now that I own a smallish cacao farm I see the certifications in a different light. They are a cost in terms of time, money, resources that we just don't have. Nowhere do I get to brag about the fact we don't slash and burn or that we provide meals and housing for our workers as well as pay them a reasonable wage that is a good 30% higher than others. Or that we are developing a land owners program as a pension for our long term workers so that when they no longer can work with us they will own enough land to support their family. Anyways, I'm just venting my frustration that I'm sure many can share as many of us are small enough that certifications are a serious burden that I'm interested in pursuing but just can't at this point and still be able to treat our workers like we want to.

Sorry I took so long to reply and again i really appreciate your response on the boards. What is your take with the latest fair trade shenanigans?

Ethan

Chris:

TCHO is one of the most exasperating chocolate companies in the US today. A bit of background.I first met the founder of TCHO, Timothy Childs, three to four years before he started the company, at a previous chocolate startup called Cabaret. I met sourcing director, John Kehoe, back in 2003 in Ecuador. I met Jane Metcalfe and Louis Rossetto (co-founders of Wired) back in 1990.

Back in late 2007 I wrote the first serious coverage (from inside the chocolate industry) on the company. Their go-to-market strategy was brilliant, and they muffed it. I so much wanted them to succeed and it seemed like they couldn't get out of their own way. They were vaporware for over a year and then, when they did start production, it was not as the bean-to-bar manufacturer on the pier in SF that they claimed to be, so they misrepresented a basic brand positioning and have been slow to address it. I think their new web site does a better job of discussing what they are doing than the old one did. But, oneof the points I've made to them repeatedly since their post-beta vaporware period is that the founders of Wired don't get a free pass when it comes to communicating via the Internet.

What I have been astounded at is how bad the company has been at highlighting one of its major contributions, which is the TCHOSource program. IMO, it's one of their strongest assets (always has been) and they still don't understand how to leverage it to their best advantage. I've had discussions with several members of the company about that point in the last six months, in fact.

The thread you cite in your link was started in 2008 and it looks like my last contribution was late in 2009. As I state above, I have been in contact with the company recently about many of the issues you raise.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clay - http://www.thechocolatelife.com/clay/

Chris -

You assume that anyone who's seen the tour sees the entire manufacturing operation - and that they are sophisticated enough to know what they are seeing and can evaluate what they are being told. That's not always the case, in my experience.

The updates to the TCHO web site are recent - they post-date the date of my original post on this subject by several months. To the best of my knowledge and ability to ascertain, I act on information that is correct as of the time I post it. If things change I am happy to acknowledge that things have changed, address the changes, and then move on.I appreciate your noticing that things have changed and bringing those changes to my attention.

TCHO's opinion regarding their being bean-to-bar is not one that is universally shared by the craft/artisan chocolate community. Just because they believe that it does, doesn't make it so in the eyes of their peers. Their position was undermined because their marketing position at the beginning was bean-to-bar on the pier and that message did not evolve as their manufacturing situation evolved.

I was with Shawn on his first bean buying trips (April 2006) and was also involved, very early on, in helping him get the business started. My assistance included working on marketing communications and I can assure you, from the very beginning, that the level of openness about sourcing and other aspects of the business has been integral to the operation both in their packaging and on the web site and electronic communications. The operation has evolved in many ways, so he's doing a better job at communicating than at the very beginning, but he has always been at the vanguard, at least in the US, IMO.

From the beginning, Shawn wanted to take personal responsibility for the relationship with the growers he sourced from, not abdicate that responsibility to a third party that, in his opinion, did not deliver the value (and the values) that he was interested in providing. His Chocolate University project in Tanzania is just one aspect of the work he does that would never be recognized within an institutionalized fairtrade model. The world (of chocolate) is way too complex to reduce to formulas on a general consumer web site.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clay - http://www.thechocolatelife.com/clay/