When a Guide to Good May Not Be
Posted in: Opinion
I'm the Chief Scientist at GoodGuide and want to respond to the methodological issues you raise about our ratings of chocolate products.
1) Role of eco-certifications
GoodGuide uses third-party certifications throughout its product and company ratings, because third-party efforts to validate performance against environmental or social standards typically represent the strongest evidence available for identifying the best products/companies in a category. In the absence of certifications, the preponderance of evidence about a product or company is comprised of whatever information a company opts to make public via its website, CSR reports or marketing. As I'm sure you would agree, company-provided information rarely discloses the negative aspects of operations or products, and can often be characterized as "greenwashing." While third-party certification services vary in what attributes they cover, the stringency of their standards, and their degree of authoritativeness, I think there is no question that they generally produce more credible evidence of "goodness" than a manufacturer's unverified self-assertions. Your alternative approach - that consumers should "develop the personal relationships necessary to understand what is really going on" with each producer - is simply not realistic for most consumers. Most purchasing decisions are made by consumers who have have limited bandwidth for identifying which products match their social or environmental preferences. GoodGuide is designed to be a service for these consumers - they rarely research the production context of their purchases, and want actionable decision support at the point of purchase. Few are going to conduct independent research, or establish personal relationships with producers.
At GoodGuide, we review the certifiers active in any given product category, determine what issues a system covers, assess the stringency of a certification and the proportion of a manufacturer's catalogue or a product's components that are certified. As a result, different certifications impact a company or product score in ways that are influenced by our overall judgement of a system. We are transparent about how we judge different certifiers (see http://www.goodguide.com/categories/255760-candy##btr for chocolate and http://www.goodguide.com/categories/255779-coffee##btr for coffee). We are open to feedback criticizing the relative weight we assign to different certifiers on different issues, and we are aware there is a lot of controversy currently about changes in the certification landscape such as those associated with Free Trade labels. GoodGuide definitely does not see itself as replacing the good work being done by third-party certifiers - we are not a standard-setting or auditing entity, we are an aggregator and integrator of the public information available on products or companies. From our perspective, the most important limitation of certifiers is that they typically address only a very small percentage of the products in a consumer product category - they can help identify the best performers, but they don't help distinguish between the majority of products that a consumer finds on a retail shelf.
2) Penalizing products that lack certification
Your comments (and separate emails from Shawn Askinosie) have raised a legitimate issue regarding whether it is appropriate to penalize a product with a lower score if it lacks certification. In most product categories, GoodGuide rewards a product that has a certification with a higher score. However, in the case of chocolates and coffee, our methodology penalizes products that lack certification in addition to rewarding products that have certifications. We provide the following explanation for this practice: "While there are some gradations between the least stringent and the most stringent certifications, all of them represent a clear step above coffees [or chocolates] that do not have any certification in an environmental or social context. As a result, products that do not have any certifications generally earn lower scores. Certifications provide credible evidence to consumers that a product has been produced in accordance with basic environmental and social standards. The absence of a certification does not mean that a product was grown under unacceptable conditions, but the consumer has no way of knowing what practices were utilized to produce the commodity they are purchasing."
This practice can result in inaccurate scores if a company operates with sustainable practices but does not seek verification of these practices by third-party certifiers. Askinosie maintains that its environmental and social performance exceed the requirements of chocolate certification systems and that the company and its suppliers have legitimate reasons for not engaging third-party certifiers. We are currently considering how to revise our system so that well-documented but self-asserted performance information can displace the default negative score we assign due to the absence of certifications.
3) Health impacts of chocolate ingredients
GoodGuide uses a standard scientific method for evaluating the nutritional value of different foods. The Ratio of Recommended to Restricted Nutrients (RRR) is a measure of nutrient density designed to apply across multiple types of food. The goal is to utilize a measure that will help people select foods both within categories and across categories, so it necessarily focuses on a defined sub-set of nutritional components. You are correct that there is recent research suggesting that not all saturated fatty acids are created equal. However, this research is in its infancy and there is little information on the percentage of each type of saturated fatty acid in different foods. In the absence of detailed saturated fat composition data across categories, we are unable to include this variable in our ratings. Note that this is not as simple an issue as your blog post implies. The existing information on fatty acid composition (from the USDA Nutrient Database) of chocolate vs. olive oil vs coconut oil indicates that these three products are quite different. Calling the saturated fat in chocolate "healthy" by comparing it to these other fats would be a stretch. Olive oil is primarily comprised of monounsaturated fat (and very little saturated fat, in fact). Coconut oil, on the other hand, is primarily comprised of saturated fat - specifically lauric acid, a medium-chain triglyceride that may not be as harmful as other, longer-chain saturated fatty acids. Chocolate/cocoa butter is also primarily comprised of saturated fatty acids - specifically palmitic and stearic acids. Although stearic acid may not be as bad as other saturated fats, palmitic acid has not been exonerated at all. Again, detailed research on saturated fatty acids is in its infancy - so much so that no major worldwide health organization has revised its position on limiting saturated fat in diets. While we recognize the distinction between different types of saturated fatty acids, we do not yet believe there is enough evidence or data to incorporate this distinction into our rating system.