Neglect Negative Data No More

4 comments

Monday, February 9, 2015

Most scientists are passionate about their jobs. We’re driven by a desire to understand how the world works and ways we can improve it. We live for the major ‘a-ha’ moment, that discovery that will have a major impact our field. In the case of biomedical research, these advances are of personal interest to the public and a primary source of health information offered by the media.

Breakthroughs are the exception, not the rule

But there is more to know than the public is aware of. What we scientists know is that working in a lab is not a series of daily achievements and exciting, groundbreaking results. Great discoveries usually come after a series of frustrating failed efforts. It’s a long, hard, and often an emotionally draining road. “Eureka” moments are the exceptions and not the norm. Frequently experiments do not generate the expected outcome, results do not support the model, or the data contradict the initial hypothesis. In other words, you have negative results. The data are there; they are just different than expected.

Negative data piling up on your desk likely hides a good amount of useful information. Why waste all the hard work by forgetting about it?

Coming to terms with negative data

Many of us feel a sense of failure when an experiment doesn’t yield the expected outcome. As scientists, we tend to neglect the importance of  negative results. These results are rarely exciting or interesting unless they provide information to contradict a previously established model or theory. We often think publishing negative data is bad for our careers. If we want to get them published, we usually have to go to a journal with a much lower impact factor than we would have hoped when starting the project.

Working on negative data can become a low or nonexistent priority. Why waste time organizing and publishing them? There are so many other things to do (e.g., write a review, attend a scientific conference, work on a grant renewal, and keep up with the lab work).

For many of us, especially students and postdocs who have a limited time in a lab and want to advance quickly in their careers, obtaining negative data is an alarm to move on to a different project and never look back. As a consequence, negative results end up in a file that is rarely opened again or not at all.

Reevaluate the value of negative data

I believe we need to reconsider how we view negative data. It can still be informative. Did you really put in all that  hard work into your experiment only to then forget about the outcome?

New research opportunities may arise for determining why the results did not match the hypothesis. Moreover, making these data publically available supports your colleagues in the field, who can save energy, time, and resources on an assay or project that has previously been shown not to work.

Why publish negative data?

Publishing negative data could mitigate stigma associated with so-called failed experiments. The hard work could still be appreciated. You would be able to show that the experiments were done in the way they were originally proposed.
Publishing such data also would help with transparency (an important consideration for Federally-funded or conducted research).

Many scientific journal already publish negative results (Bob O'Hara, Nature, 471, 448–449, 2011). If you don’t want to put together a paper entirely on the negative data, don’t just forget them in a drawer forever, but try to make them part of another publication, included as at least one figure along with the rest of results.

Negative findings are part of the scientific enterprise

You might not be happy when an experiment isn’t  working; but that doesn’t mean you can’t love the research process. Remember, there is still a great learning opportunity with negative data, for finding answers to interesting questions.

Not every piece of negative data should be considered for publication. But, we should take the negative data out of the drawers and spend a little bit of time organizing them. You might find the data will be informative for others in the field and the overall scientific community.

Category: Science

Leave a Comment

4 Comments

Sivakumaran
February 9, 2015, 10:59 am

I agree with every line in this article. I have seen many young budding researchers get disappointed with the so called 'negative data' or data that they were not expecting. I don't think we, as researchers, should be disappointed. Rather, as the author of this article says, we should think about it and figure out the science behind the 'negative data' - the same science which is behind a 'positive result' in a given experiment. That is what science is all about. If the result we obtained was not what we expected, it is not necessarily because there was something wrong in the experimental approach or technique. It could very well be there is something beyond what we thought would happen, something beyond what we thought about how science works. Scientific research outcome is not a defined entity. Research is to re-search. I have always believed that data which did not produce a result we expected will always have something else in store - something we might have overlooked or not thought about. During my PhD years, I remember having personal experience with such 'negative data' but we in fact figured out what was happening and ended up making finding out of it (which actually led us to few more publications). It should be imperative of every scientific research institution and PI to make sure students, postdocs, trainees don't get disappointed with 'negative data' but rather take a moment to think about it and try to figure out what might be happening. It might very well throw up nothing but it is worth spending some time on.

labrat
February 9, 2015, 2:17 pm

Liked the piece. It makes me think beyond rebranding negative data - two points (unrelated to each other). 1. Help others not to repeat the same negative data unknowingly. This is different from an attempt to reproduce negative data. By disclosing the negative data (and more importantly how it was done), others can avoid being trapped in the same route. Crucial in terms of saving NIH dollars. 2. Mental flexibility of PI. There is another rather important piece missing in the writing. I think it's the PI and those in the management positions. Negative data are labeled so not always because they contradict with the literature, but often they contradict with those above the scientists at the bench. When the PI can adapt to a new set of contradicting data and form a new hypothesis accordingly, it's no longer 'negative.' I think it becomes fellows' responsibility to convince the PI with a new hypothesis (that is if there is.)

sonali sengupta
February 10, 2015, 7:17 am

agree..every bit of data provides clues...ignoring any bit of information is like shutting of an avenue for further research...kudos to the author of this article.....

Steve Durfee
February 10, 2015, 2:15 pm

The Michaelson-Morley experiment resulted in negative data, and it's one of the most famous and informative experiments in the physics of the time. http://en.m.wikipedia.org/wiki/Michelson–Morley_experiment