The 1st Unspoken Rule for Success in Science

After a decade of earning my Ph.D. in psychology, I wrote a list of several scientific articles that nobody should read or cite anymore. The list only included my own articles. Now that 16 years have passed since I earned my Ph.D., it is time to write a long overdue article on what I learned about conducting psychological science that is not spoken in public. Rules that are only shared in private conversations at dinner. Rules that put less socially connected people at a disadvantage (especially early career researchers). This list is not meant to be comprehensive. If the response is relatively positive, I will share additional unspoken rules in subsequent posts.

I need to give you a warning: You are going to judge me, perhaps harshly. You are going to experience a bit of cynicism about science. Everything below is true. I am writing this because I wish someone shared this information with me when I was a graduate student and young professor. Nobody did. I refuse to keep them to myself any longer.

For anything negative, I try to denigrate myself (not too difficult). Be aware that I could give a wide range of examples about other researchers, many of whom are widely respected and bestowed with eminent titles and awards from leading scientific organizations. To me, the cost of “outing” anyone is not worth the benefits. I am sure readers have their own stories and know many of the researchers I am thinking of. What is most important is that you gain insight into what happens behind the scenes.

Rule #1: There are secret backdoor routes to getting scientific articles published.

Here is a little-known fact. My two first-author articles published in Psychological Science were both peer reviewed and rejected. One after a revised and resubmitted manuscript failed to satiate the anonymous reviewers.

• Kashdan, T.B., Ferssizidis, P., Collins, R.L., & Muraven, M. (2010). Emotion differentiation as resilience against excessive alcohol use: An ecological momentary assessment in underage social drinkers. Psychological Science, 21, 1341-1347.
• Kashdan, T.B., & Steger, M. (2006). Expanding the topography of social anxiety: An experience sampling assessment of positive emotions and events, and emotion suppression. Psychological Science, 17, 120-128.

My co-authors and I disagreed with the rationale provided for the rejection decisions. We appealed. We wrote lengthy responses to the appeals with new data analyses and results, and plenty of detailed conceptual explanations filled with scientific references. The action editors accepted our appeal, sent the papers back out to reviewers, and both papers were eventually accepted.

I bring this up because people do not often realize this is an option. Sometimes you get subpar, even unfair reviews. For the majority of my publications, the action editor did not offer an independent review. They took a box score vote and accepted the reviewer comments and decisions without any critical analysis. I offer an immense amount of respect for the few times editors pointed out problems with one or more of the reviewer comments and overruled them. It takes courage. The field of science needs more courage.

It is important for you to be empowered. You are not always at the mercy of an inadequate review process. If you think you’ve been wronged, I strongly encourage you to run this by several colleagues who are willing and able to disagree with you. Give them an opportunity to offer an objective opinion with an opener such as this: “I just received a few reviews of my manuscript. I was wondering if you would be willing to read our paper and let us know which reviewer comments you agree with most and which you disagree with (if any)?” If your peers agree with you, sit on the paper for 48 hours. If you are still incensed, begin writing a detailed letter to the action editor. Treat the letter no differently than a revise-resubmit. Be detailed. Respond to everything that is important. Quote the exact words of the reviewers. Try to find ways you can offer objective, instead of subjective, data to respond to disagreements. Above all else, be incredibly kind and considerate. It might work. It also might not be worth the time and effort.

You need to know that appeals are commonplace. I have been an Associate Editor of the Journal of Personality and Social Psychology, Journal of Personality, and Journal of Positive Psychology. Close to 10 research teams appealed my decisions. One appeal was written by a very (very) senior researcher in the field (president of psychological organizations, distinguished professor title, etc.). Their email to me was filled with ad hominem comments and passive-aggressive threats. I was worried about losing my position. I was worried I did the wrong thing by rejecting their paper. I was worried about what their wrath would do to my reputation and future opportunities. So while I did not agree with their flimsy arguments (no data, just indignation), I changed my decision. I still regret it to this day. Their paper should not have been published. They did not respond to the key criticisms raised by reviewers that I agreed with. I caved. Since that day, I wondered: How many peer-reviewed publications by prestigious, senior authors are in journals because of a similar process? Which publications have truly passed the quality control process of peer review?

Psychology Essential Reads

Is a Texas Bitcoin Miner Making People Sick?

What Is a Scientific Model?

I ask these questions because there is another back-door route to getting work published in peer-reviewed articles: Invited submissions to journals. My most widely cited article is based on an invitation to be part of a special issue in Clinical Psychology Review.

• Kashdan, T.B., & Rottenberg, J. (2010). Psychological flexibility as a fundamental aspect of health. Clinical Psychology Review, 30, 865-878.

It went through a peer-review process but if I am going to be honest, I never doubted that the paper was going to get in. I am unaware of a single article invited to the Special Issue on Positive Clinical Psychology that was rejected. Multiple articles were authored or co-authored by the Guest Editors. You could argue that the invited authors are leading scientists in their area. But I could name 10 alternative scientists who could have been written any of the articles in the issue.

Make the right friends. Be in the right lineage. Go to the right university. And you too could get a fast pass to published peer-reviewed journal articles. Again, it makes you wonder whether peer-reviewed scientific review is the gold standard that it is promoted to be.

I train my students to read articles and to not fall for proxies for quality such as the journal outlet, the impact factor of journals, or the prestige of authors (whether by their years in the field, number of citations to their work, h-factor, home institution, twitter followers, TED talk numbers, or anything else). Every scientific article must be evaluated on merit — the exact research methodology, statistical analyses, and data that are provided and used to draw conclusions.

This is the first in a series providing an insider's view of academia. I have tenure. I have a sufficient number of years in the field that I can absorb some of the backlash that might stem from being candid about what I have seen that is questionable. My hope is to improve the field by exposing what's wrong with it so we can discuss ways to improve the field and implement them. Just because norms exist does not mean they are helpful or functional. In a flattened social hierarchy, where everyone can access science from Google Scholar, and contact any scholar by email or social media, it is time to mobilize and improve psychology. Because in a world where people are having a hard time distinguishing between real and fake information, psychology needs to be a trustworthy repository of accurate, replicable knowledge about the human condition.

Send any questions, comments, or ideas about this post or future unspoken rules to me by finding me on Twitter: @toddkashdan