Please Commit More Blatant Academic Fraud (2021)

Double-blind review is a mirage that does not hold up. While I was in academia I reviewed a paper that turned out to be a blatant case of plagiarism. It was a clear Level 1 copy according to the IEEE plagiarism levels (Uncredited Verbatim Copy of more than 50% of a single paper). I submitted all of these findings with the original paper and what parts were copied (essentially all of it) as my review.

A few days later I got an email from the author (some professor) who wanted to discuss this with me, claiming that the paper was written by some of his students who were not credited as authors. They were unexperienced, made a mistake, yaddah yaddah yaddah. I forwarded the mail to the editors and never heard from this case again. I don't expect that anything happened, the corrective actions for a level-1 violation are pretty harsh and would have been hard to miss.

The fact that this person was able to obtain my name and contact info shattered any trust I had in the "blind" part of the double-blind review process.

The other two reviewers had recommended to accept the paper without revisions, by the way.

I am a co-author on a paper I never asked for, but my supervisor insisted, because the petty idea upgrading his desperate try to the point of "considerable at all" came from me. It was a chair which normally prouded itself of only publishing in the most highly regarded journals of its field (internally graded A, B, C). They had a few years without viable paper. Desperate to publish. From my POV this was a D. The paper is worthless crap, hastily put together within two weeks. It should have been obvious to the reviewers.

I feel ashamed that my name is on it. I wish I could retract it.

So, yes please: make it hard to impossible for paper mills and kill the whole publish or perish approach.

"For the first time, researchers reading conference proceedings will be forced to wonder: does this work truly merit my attention? Or is its publication simply the result of fraud? [...] But the mere possibility that any given paper was published through fraud forces people to engage more skeptically with all published work."

Well... spending a few weeks reproducing a shiny conference paper that simply doesn't work and is easily beaten by any classical baseline will do that to you in the first few months of your PhD imo. I've become so skeptic over the years that I assume almost all papers to be lies until proven otherwise.

"This surfaces the fundamental tension between good science and career progression buried deep at the heart of academia. Most researchers are to some extent “career researchers”, motivated by the power and prestige that rewards those who excel in the academic system, rather than idealistic pursuit of scientific truth."

For the first years of my PhD I simply refused to parttake in the subtle kinds of frauud listed in the second paragraph of the post. As a result, I barely had any publications worth mentioning. Mostly papers shared with others, where I couldn't stop the paper from happening by the time I realized that there is too little substance for me to be comfortable with it.

As a result, my publication history looks sad and my carreer looks nothing like I wished it would.

Now, a few years later, I've become much better at research and can now get my papers to the point where I'm comfortable submitting them with a straight face. I've also came to terms with overselling something that does have substance, just not as much as I wish it had.

I had two experiences at the polar opposites of the spectrum - one research team I worked on had very high standards and was comfortable being patient for material that had value. The other involved an approach that obviously stood no chance to be useful to anyone.

Some differences:

- The first one was in a space with more low hanging fruit

- The first one was after large effect sizes, not the kind where you can massage the statistical model

- The second one was a topic with far higher public interest

- The second one was primarily an analytic project, whereas the first one was primarily experimental

I feel like bad science lives in the middle of a spectrum - you have young fields/subfields with boring but impressive experimental breakthroughs, and on the other end you have highly political questions that have been argued to death without resolution. Bad science is about borrowing some of the strategies used in politics because all the important experiments have already been done.

> Proclaiming that your work is a “promising first step” in your introduction, despite being fully aware that nobody will ever build on it.

Science produces discrete units which can be used in different ways, if not in their exact form from a preceding research. I am not sure it’s reasonable to say that existing ideas, even if not cited, are not inspirational (to the researchers themselves). Peer-review isn’t perfect, but I think that all accepted papers have something academically or scientifically relevant, even if there’s no guarantee that the paper will generate hundreds of subsequent citations. I think improving your subsequent work is more important, which includes mentioning why you think some previous work may not be as relevant anymore. This last step is often missing from many research papers.

I think the author is right that it doesn’t quite make sense to publish anything you know isn’t quite correct. But I can think of several papers in different fields which someone may think are “not quite correct”, but the goal of such papers, I think, is to demonstrate the power of low probability scenarios, or edge cases. Edge cases are important because they break expected behavior, and are often the root cause of system fragility, system evolution, or poor generalization in other systems.

> Most researchers are to some extent “career researchers”, motivated by the power and prestige that rewards those who excel in the academic system, rather than idealistic pursuit of scientific truth.

This is the funny part. There is little to no power and prestige to be had in the academic system. To a first approximation no one outside academia cares.

I was just working as a staff programmer and taking grad courses with my tuition benefit, and found myself getting caught up in the mentality of needing a PhD to really be successful and valuable. Then got a job in industry making far more money and realized how academia is a small self contained world with status hierarchies irrelevant outside that small world.

> Submitting a paper to a conference because it’s got a decent shot at acceptance and you don’t want the time you spent on it go to waste, even though you’ve since realized that the core ideas aren’t quite correct.

I don't see a problem with this? If papers are the vehicle for conference entries why shouldn't authors submit it just because it's wrong? Conferences are for discussion. So go there and discuss it... "My paper says XYZ, but since I wrote it I realised ABC" - sounds like a good talk to me?

(Naivety check: I am not an academic)

> And we must ensure that explicit suggestions to modify one’s science in the service of one’s career – “you need to do X to be published”, “you need to publish Y to graduate”, “you need to avoid criticizing Z to get hired” – carry social penalties as severe as a suggestion of plagiarism or fraud.

One of the pernicious things in this area is that, even as we teach young researchers how to avoid making mistakes and engage sceptically with the work of others and that scientific fraud is a nontrivial issue, we also tell them how to commit fraud themselves and that their competition is doing it.

"Watch out for P-hacking, that's where the researcher uses a form of analysis that has a small chance of a false positive, and analyses loads of subsets of your dataset until a false positive arises and just publishes that one"

"Watch out for over-fitting to benchmarks, like a car taking the speed crown by sacrificing the ability to corner"

"Watch out for incomplete descriptions of test setups, like testing on a 'continent-scale map' but not mentioning how detailed a map it was"

"Watch out for citations where the cited paper doesn't say what is claimed, some people will copy-and-paste citations without reading the source paper"

"Watch out for papers using complicated notation, fancy equations and jargon to make you feel this looks like a 'proper' paper"

"Watch out for deceptive choice of accurate numbers, like a study with a 25% completion rate including the drop-outs in the number of participants"

"Watch out for simulations with inaccurate noise models, if the noise is gaussian in the simulation but a random walk in reality, great simulated results won't transfer to reality"

I've made no suggestion at all that you should modify your science or commit fraud - but I've also just trained you in how to do it.

> * A group of colluding authors writes and submits papers to the conference.

> * The colluders share, amongst themselves, the titles of each other's papers, violating the tenet of blind reviewing and creating a significant undisclosed conflict of interest.

> * The colluders hide conflicts of interest, then bid to review these papers, sometimes from duplicate accounts, in an attempt to be assigned to these papers as reviewers.

Is it that common that conference reviewers also submit papers to the conference? Wouldn't that alone already be a conflict of interest? (After all, you then have an interest in dismissing as many papers as possible to increase the likelihood of your own paper being accepted). And how do you create "duplicate accounts"? The conferences I have submitted to, and reviewed for, all had an invitation-like process for potential reviewers.

Hmmm, I wonder if you could turn this into a sport and have like one paper per year per group of total BS, and shame on the reviewers/conference/journal if they don't catch it, and kudos to the submitters the more blatant it is.

Come to think of it, is there a "Journal of Academic Fraud"?

I agree with the analysis completely, but the solution is depressing. I keep thinking that publications on arxiv might be a better source of knowledge given the motivation for publishing there is not a contribution to career progression. Keyword search over arxiv papers? But perhaps we should bring back the idea of anonymous publishing:-0

I don't have top-tier publications, and I haven't gotten any awards. I've seen people get awards for bullshit and farming prestigous publications. I only have one citation for work that is 100% mine. But every time I read something like this, I feel proud that I don't bullshit, and at least try to do real science. I truly believe in all my work and every sentence I write.

That being said, the insane emphasis on venue is what's pushing me out of academia. I can't compete with people like this.