April 17, 2017

Avoid the P-Value Pitfall

Author: Jay Harrison

Is your pet food research based on an oversimplified interpretation of the data analysis? That was the question I explored in my recent Petfood Forum 2017 presentation, “The true meaning of the p-value in pet food research,” to attendees at the show in Kansas City.

A LITTLE BACKGROUND

In the pet food industry, business decisions about product advancement and claims about pet and consumer behavior are based on comparative assessments using data. However, data must be carefully analyzed to tell an accurate story. Often, this analysis hinges on the p-value, a statistic that indicates whether a hypothetical situation seems reasonable after collecting and analyzing the data.

In pet food research, when comparing two rations with a statistical test:


A large p-value means the experiment did not provide compelling evidence that the two rations were different in preference in the pet population.

A small p-value means we would be unlikely to observe such a large difference between the two rations if, in fact, they are equally preferred in the pet population. In this way, a small p-value provides evidence supporting the idea that the two rations are different.

The historically accepted .05 “cutoff” means p-values less than .05 are considered statistically significant. This cutoff is based on tradition and was originally influenced by computational convenience before computers became widely available.

P-VALUE CONFUSION

The p-value is complicated. Confusion—and even an incorrect conclusion—can arise when the p-value is oversimplified. For example, one common claim is that a p-value that falls above the .05 cutoff indicates the two rations were the same in preference. In fact, it merely indicates there wasn’t enough evidence in the data to conclude the rations were different.

The situation is akin to a courtroom trial in which the defendant is found not guilty. That verdict doesn’t necessarily mean the defendant didn’t commit the crime; instead, it means the evidence didn’t adequately convince the jury the defendant is guilty.

In short, the p-value doesn’t tell the whole story. Consider this example: Animals in a pet food trial have high variation, with some showing high preference for one ration and some preferring the other ration. This would result in a large p-value, indicating no significant difference between rations. However, rather than dismiss the results, it would be wise to further investigate whether there was an identifiable characteristic potentially responsible for the preferences—such as senior animals preferring one ration and younger animals preferring the other. Finding that key could result in a strategy for marketing each ration to different consumer segments, despite no statistically significant difference between rations. The result could be maximized research efficiency and new opportunities to serve customers more effectively.

MAKING THE P-VALUE MEANINGFUL

To avoid this p-value pitfall, include supplementary data that can add context, such as:

  • Mean, sample size, standard deviation
  • Graphs
  • Confidence intervals

In addition, avoid creating oversimplified explanations for a general audience. It’s not necessary to give the mathematical definition; instead, state that the p-value is simply a number calculated from the pet food research data to help indicate whether one ration is preferred to the other in the pet population.

For more insights into p-value, including analogies and examples that will resonate with scientists and non-scientists alike, see “The true meaning of the p-value in pet food research” in the Downloads area at afbinternational.com or email me at jharrison@afbinternational.com.

Tags: standard deviation, sample size, Petfood Forum 2017, pet preference, Pet Food Research Data, pet food research, Pet Food Product Advancement, Pet Food P-Value, pet food industry, pet behavior,


White Papers

FREE WHITE PAPER

The true meaning of the p-value in pet food research

AFB principal biostatistician Jay Harrison dispels misconceptions about the p-value—a probability calculation used in pet food research to determine whether one ration is preferred to the other—in this presentation shared at Petfood Forum 2017.

Download

Comments






Please enter the word you see in the image below:

{snippet_privacy_policy}