Responsible Conduct of Research (RCR) Event Reflection

For my Responsible Conduct of Research Event, I attended the Research Integrity Roundtable titled “Reproducibility Crisis: What Can We Do?” on January 6, 2023. This event took place on Zoom and was organized by The Advancing Scientific Integrity, Services and Training (ASIST) program at the Duke Office for Scientific Integrity (DOSI). Having taken academic classes and research training courses that touched on this crisis, I was already somewhat familiar with the topic going into the event. Nonetheless, the participants in the roundtable offered a wealth of new backgrounds and perspectives that broaden my views on the topic.

For example, as a statistics major, my previous discussions on replicability and reproducibility have involved the purposeful mishandling or misuse of data, often with the goal of biasing the data towards a certain outcome. In these cases, there is clear misconduct on the part of the researchers. However, in the roundtable, many of the participants were medical professionals or biologists without a background in statistics. They brought up the fact that statistical methods have become increasingly complex over time and that they often do not fully understand the methods being used. In fact, many of the researchers have a dedicated statistician on their team or outsource their analysis to another statistician at Duke instead of doing it themselves.

Thus, although these researchers might believe that the data has been misrepresented, they do not know enough statistics to confidently question the methods used in the paper. This issue is especially exacerbated in younger co-authors (for example, undergraduate and graduate students), who may have less confidence to question the methods used in a paper. They may assume that the older researcher knows best and that it’s simply a method they haven’t encountered before. At the same time, perceived misconduct in younger researchers has a disproportionately large impact on their careers as they are less established. Thus, the participants in the roundtable brought up the fact that some of the reproducibility crisis may not be due to purposeful misconduct, but rather misunderstanding of statistics and lack of confidence to speak up.

In these cases, the participants highlighted the importance of peer reviewers to catch this misconduct before it gets published. They acknowledged that, because peer reviewers are typically anonymous, they should be able to point out inconsistencies without fearing backlash. At the same time, other participants noted that, for the same reason, they do not have a stake in the published research. One doctor in particular stated that peer reviewers may see a well-established researcher and, whether purposefully or not, give a biased review that allows faulty research to be published. Thus, this doctor wanted to see more accountability for peer reviewers in stopping the reproducibility crisis.

These reflections were especially impactful for me as they reinforced the amount of responsibility I have in the research process. My project is a healthcare-based project in collaboration with two clinicians who typically outsource their statistical analyses (as many of the participants in the roundtable did). It is important for me to remember that, while they can provide their expertise in the form of domain knowledge, I have the bulk of the responsibility to make justifiable and statistically sound decisions. Furthermore, I must be cautious in my interpretations to make sure that I am not misrepresenting the conclusions of the study. Because there is no established peer review process and the clinicians are not as familiar with the data and methodology, it is important for me to be clear concerning any limitations of the analysis, both in this independent study and in future research projects.