A recent Letter to the Editor of The Standard encouraged me to step back for a moment and consider if the last column I wrote about butter had errors in it. I decided it didn’t – it just expressed an opposing point of view.
This brought to mind considerations I have offered in previous columns, about the difficulty in deciding which information sources to trust. As an example, for years I studied pesticides and their impacts on people and wildlife and their legacy in the environment. I constantly faced challenges from focus groups (either pro or con pesticides) which used carefully selected data to their advantage to ‘prove’ pesticides were good or bad depending on their agenda. Data is what you make it and cunningly analyzing it to fit your argument and crafting your words can definitely influence the ‘facts’ in significant ways!
One of the challenges in trying to assess the validity of a source is to determine if their protocol is scientifically supportable and if they had any bias in their techniques and reporting. One means to confirm this is to do a peer review, which is a mechanism whereby experts in the field study the report and determine if it is both scientifically and statistically valid and if the conclusions derived were reasonable and most importantly reproducible using normal scientific techniques.
The authors of the Expert Working Group on Feed Supplements report, cited by Ms. Taylor-Sevier in her letter, state “the information provided in this report is strictly descriptive and cannot be submitted to a proper statistical analysis, due to the nature of the sampling procedure” [e.g. only 40 butter samples]. This in itself doesn’t negate their conclusions but does indicate more study is needed. Literature reviews, which were the primary tool they used in reaching their conclusions, have immense value, as each one was (hopefully) peer reviewed prior to this analysis.
How you present your conclusion is equally important. In this case, they concluded “there is no data to confirm that there has been a change in the consistency of butter over time”. These are interesting words as they don’t stipulate the butter hasn’t changed – they just say, the available data doesn’t prove that. They go on to say “As a result of this lack of data, it is not possible to test for a causal relationship and therefore draw conclusions, on the link between the use of palm-derived supplements … and the consistency of butter …”. This actually ignores the original question as to whether butter has changed and replaces it with a new question – if it did change, what caused that change? It’s paramount, when conducting and reporting on studies, the fundamental questions behind the study are not lost.
One thing, which is rarely included in these studies, is the genesis of the question. Is awareness a factor which needs to be considered? For example, the authors state they consulted with processors who determined they had not made changes to the butter manufacturing processes in recent times but acknowledge multiple processing factors can directly influence the characteristics of the butter. But what does this mean? Well, a home baker might notice the butter is harder, but a bread manufacturer who is busy making millions of loaves of bread might not. So the perspective of the original reporter should at least be explored.
Setting science aside for a moment, one challenge I face as a columnist is editorial limits. My columns are supposed to be about 700 words.
But how do I broach a subject as complicated as this and try to tell the reader what is important? Admittedly, sometimes not enough information can be presented to support the argument. However, the reader is always encouraged to look for other sources to try to find out more. Often a newspaper article is simply a stimulus to further research on the part of the reader.
Thank-you to Ms. Taylor-Sevier for expressing her thoughts, as this makes us all better and more confident in our thinking. Meanwhile my butter is still hard!
Geoff Carpentier is a published author, expedition guide and environmental consultant. Visit Geoff on-line on LinkedIn, Facebook and Instagram.