Who exactly needs to get approval from an institutional review board (IRB)?
164 comments
·February 13, 2025Kevin_S
mmooss
> IRB does always give my studies "exempt" status, but it still has to be reviewed.
How could they do it more efficiently?:
How could they know it's exempt without reviewing it? Should they take researchers' at face value? That seems to undermine the reason we have IRBs, which was unscrupulous researchers. Should we assume they are willing to torture people but not to mislead the IRB?
Why shouldn't data be private? How hard is it?
> a Kinesiology dissertation study where they were asked to do extremely strenuous physical activity. He fainted(!) at one point from the activity. And it seemed to them that there were relatively few safeguards from that happening.
Informed consent is essential to an IRB; your friend would have read and signed something detailing the activities and risks. Strenuous excercise to the point of exhaustion is a part of sports performance research, at least. As long as you inform people, they have the power to opt-out (not prisoners, 6 year olds, etc.), and no undue risk.
bpodgursky
> How could they do it more efficiently?:
Think about how taxes work. The IRS doesn't check every person. You make rules, you perform random audits.
IRBs could easily do the same thing. Set some rules like (simplified) "No IRB necessary if your research doesn't physically touch a person". Researchers will read the rules and skip an IRB if they're in an exempt category. Then just audit at a high enough rate they won't lie.
dr_dshiv
There is a huge opportunity in simplifying and automating IRB review.
Here in the Netherlands (we call it HREC, human research ethics committee) the process takes months and months to get permission to talk to people.
Like, “hey I’ve designed some cool new interaction design, I want to get permission to interview people and do human-centered design iterations.” Yes, you will need to spend dozens of hours on forms and wait months.
It cannot be ethical to put these many barriers on talking to humans. And, if you want to do something like an educational game for kids? Or support for the elderly? Or for your students? Well, those are vulnerable populations, so add a few more months.
Education research is extremely difficult here (in the USA it is exempt). How to conduct research on the use of AI in a class you are teaching? You can’t get retroactive consent.
The system is not reasonable.
mmooss
Nobody dies if someone's taxes are wrong. Also, IRBs are not usually about lies, but about scientists not understanding the risks and solutions.
jampekka
We are not. Here's a great essay about the subject:
https://journals.sagepub.com/doi/abs/10.1177/174701611100700...
The author also has a blog about IRBs: http://www.institutionalreviewblog.com/?m=1
inetknght
> I do accounting research. My human-subjects research involves surveying practitioners about their jobs, interviewing them about their experiences, and conducting very simple experiments online that ask them to make decisions. There is virtually 0 real risk to any participants of my studies.
Cool, I guess. I wouldn't want to participate if my answers were visible to my employer because honest answers could put my employment at risk.
> As an academic researcher, it's most frustrating in that it feels like IRB scrutiny doesn't seem to align with risk.
As a worker^H^H^H^H^H^Hhuman I find it most frustrating when academics don't understand what things I consider to be risks.
If my answers to your research are leaked, can they be tracked back to me? Onedrive is terrible, just look at Microsoft's repeated flagrant disregard for security. Dropbox is awful too, just look at how easy it is to accidentally a whole folder. Why would you store your data in such places instead of on a research computer with locked down access? That screams to me of a researcher who doesn't understand the value (and risk) of the data they hold. Or perhaps of a researcher who doesn't value that risk appropriately. Perhaps also researcher who doesn't coordinate with their IT department, or an IT department who's equally or worse as competent.
p_j_w
>Onedrive is fine, but dropbox is not for some reason
As far as I know, it's possible to get OneDrive even HIIPA compliant, but I don't think so for Dropbox.
eszed
Dropbox claims that it can achieve HIPAA-compliance, and I personally know two (large and serious) organizations who use it as such.
https://help.dropbox.com/security/hipaa-hitech-overview
In keeping with the theme of the OP, I don't know if that's been tested in court, soooo... Take that all for what you will.
derbOac
The answer is no, you don't need permission, but you do incur a legal and ethical risk whenever you do research, and as such, the more you can do to strengthen the legal and ethical arguments in your defense, the better.
IRBs are simply a way to say "I had an independent group of ethical and legal authority review my research and they thought it was ok".
billjive
That is right, and further - I think about IRBs like I do non-profit boards. One of the key tasks of a board is to determine what risk the organization is willing to bear. I think the IRBs at these institutions are 1) following federal laws but 2) making a judgement on the risk the institution is willing to sign up for. For example, for many clinical trials are distributed to multiple "sites" where the trial is actually administered. Many of these sites are universities. What the University of Iowa is willing to bear is different than the University of Michigan and the IRBs reflect that.
mionhe
With medical device development, the IRB represents the institution taking liability for harm to patients. They're there to make sure they aren't about to kill someone or harm them and get the institution sued.
So they take it pretty seriously. More so in countries where it's easy to sue, though.
rights_reminder
It's hard to take the article seriously since
1) It doesn't even reference the relevant law
2) The relevant law is so simple and clear
"The Secretary shall by regulation require that each entity which applies for a grant, contract, or cooperative agreement under this chapter for any project or program which involves the conduct of biomedical or behavioral research involving human subjects submit in or with its application for such grant, contract, or cooperative agreement assurances satisfactory to the Secretary that it has established (in accordance with regulations which the Secretary shall prescribe) a board (to be known as an “Institutional Review Board”) to review biomedical and behavioral research involving human subjects conducted at or supported by such entity in order to protect the rights of the human subjects of such research."[1]
To put it succinctly this only relates to any entity which applies for a grant, contract, coop. agreement under chapter 6A of 42 USC.
This in no way relates to private research. It's easy to get hung up on regulation spaghetti code, but regulations cannot exceed the scope of the statutes, which in turn cannot exceed the authority of Congress.
Edit: add source [1] https://www.law.cornell.edu/uscode/text/42/289
karaterobot
> To put it succinctly this only relates to any entity which applies for a grant, contract, coop. agreement under chapter 6A of 42 USC.
But the whole point of the article is that there are other cases when you need IRB approval besides this one. That's one of the myths he's busting.
rights_reminder
The point that I was making was that there can be no federal law which requires an institutional review board for private research, which was something the author alluded to in the paragraph after the flow chart:
> The government can’t possibly claim to regulate what me and my roommates eat at home! That would be stupid!” Yes, it would be stupid. But who says the world makes sense? Now, would you actually be prosecuted for violating those rules?
Author says "those rules" without actually citing anything.
There is no scenario in which the Secretary of HHS can create rules which would do what the author was suggesting. It is not even something Congress could do, which is why they haven't created a statute which goes beyond their authority in requiring these boards for anything other than research receiving federal funding. If the Secretary were to create such a rule, it would obviously exceed his authority.
cal1co
An article like this is interesting from a research (meta-research?) perspective, but I think the lack of relevant case law (nobody seems to have ever been prosecuted) really undermines the usefulness.
It's easy for people (especially the kinds of people who gravitate towards engineering) to get all worked up about the necessary implications of the regulations as-written. But it's also important to remember that being facings consequences would require
A) The intentional action of either a state prosecutor or an injured private party B) The involvement of a human being as a judge to interpret the facts and applicable rules
When you look at it from that angle, it's obvious why organizations like META just do the research, imho.
Xcelerate
> It's easy for people (especially the kinds of people who gravitate towards engineering) to get all worked up about the necessary implications of the regulations as-written
I was once trying to figure out taxes owed on some RSUs granted in California that vested in North Carolina (and some other edge case that I can’t remember). Hoh boy. What a rabbit hole. I didn’t think the CPA that I hired to do my taxes handled it correctly, so I started digging into the tax laws and regulations. Couldn’t find anything that covered my specific case. So I started digging into court cases involving disputes over RSU taxes owed in North Carolina and found conflicting outcomes.
I brought what I had found back to the CPA who basically said something along the lines of: there is no absolute “rule” for what to do in this case like you’re looking for. We make a good faith effort to mirror what was done in similar prior situations, write a number down, and then forget about it unless you hear from the IRS.
Did not make me feel very confident about the legal system.
floxy
>...In the real world, people usually attempt to solve problems by forming hypotheses and then testing them against the facts as they know them. When the facts confirm the hypotheses, they are accepted as true, although subject to re-evaluation as new evidence is discovered. This is a successful method of reasoning about scientific and other empirical matters because the physical world has a definite, unique structure. It works because the laws of nature are consistent. In the real world, it is entirely appropriate to assume that once you have confirmed your hypothesis, all other hypotheses inconsistent with it are incorrect.
> In the legal world, however, this assumption does not hold. This is because unlike the laws of nature, political laws are not consistent. The law human beings create to regulate their conduct is made up of incompatible, contradictory rules and principles; and, as anyone who has studied a little logic can demonstrate, any conclusion can be validly derived from a set of contradictory premises. This means that a logically sound argument can be found for any legal conclusion.
The Myth of the Rule of Law:
https://drive.google.com/file/d/1I-JhqpU3_0r_HL06hP-5DABhEtG...
birktj
Thanks for the link! I have been thinking a lot about topics related to this lately and this looks like an interesting read. Skimming through it I might not agree with all the ideas of the author I very much welcome new ideas about how to organize our society and our legal system. I have especially been thinking about how the legal system is mostly fiction and what makes it work is not so much what the laws say and what the punishments are for breaking them, but rather our shared belief in the system. And I think this is problematic because it takes away our agency to imagine other ways of doing things, we think our current legal system is the only possible system because it has to be. Thoughts of anything else would cause a collapse.
I have also been thinking about how basing morality on following laws is a bad idea. You should not be afraid of breaking laws or rules if they are stupid. All of modern workers rights, women's rights, etc are based on brave people of the past that were not afraid of breaking stupid laws of their time, why should today be any different.
noodlesUK
I think this illustrates an issue in common law systems in general. Contrary to what we’re taught in school about how actions are generally legal by default, and only illegal if a law has been passed to prohibit them, in reality, actions exist in a superposition of legal and illegal until someone challenges that position and a court finds out (subject to any appeals). You can only evaluate in practice whether something is legal based on copying others in similar situations. The further you get from the known outcomes the fuzzier this all becomes.
This tends to be something people encounter a lot around tax, complex business structures, and various international dealings where you’re more likely to end up in an edge case that hasn’t been well explored.
miki123211
And it's worth emphasizing that this is a good thing.
Civil law systems (which are used in the non-English-speaking parts of Europe for example) don't have this property. In such systems, court precedents don't really exist, so if something isn't clearly defined by law, and there's no extremely ubiquitous legal interpretation that everybody seems to follow, you don't know whether it's legal or not.
For example, the Polish tax authority lets you ask for a tax interpretation for cases that aren't legally obvious, something that I believe the US would have handled via the court precedent system. It's not unusual for two people in identical situations to receive opposite rulings, and Those rulings are not binding on the authority that issued them.
In other words, you don't know how to handle your tax situation, you ask your tax authority, they tell you "hey, we're the tax authority, we're telling you to do x", you do x, they audit you, they say "this isn't clearly defined in law, we now believe you should have done y instead, here's your orange jumpsuit."
dekhn
It took me a while to accept that the law is not a consistent system with hierarchical top-down rules that can be used to determine legality in an unambiguous way.
If you're a software engineer and you're knee-deep in analyzing court cases to find out how to file some tax correctly, you're definitely overthinking it. The CPA was exactly correct.
Xcelerate
Yeah but now I just switch my perspective from thinking of it as a formal deductive system to thinking of it as an optimization problem.
aqueueaqueue
If you move from Germany to Brazil you might also hit edge cases.
Yes that is 2 different countries. But it is similar. States have their own laws.
You can't expect the law across 3 jurisdictions to work out every permutation and legislate for it. There are 52 (I guess more... lots of territories) * 51 permutations of moves so 2652 moves that are possible.
In addition you are talking RSUs and not some vanilla thing like claiming an expense.
appleorchard46
This is addressed at a few different points in the article, I found this bit particularly insightful:
> So what happens in practice is politicians write a vague law. Bureaucrats turn that law into very detailed (but often still vague) specific rules. Those rules might or might not be “legal”, but nobody want to risk fighting them in court. If the regulations are particularly ridiculous or likely to be overturned if challenged, prosecutors may quietly stop bringing cases. But the regulations still sit there on the books. And people still usually pay attention to them, because why risk it?
billjive
This specific quote from the article made me think of lawmaking - the more detailed the law the harder it is to pass (someone can always fine something to nitpick). Therefore, laws are best at establishing goals, principles, guide rails, etc. Nothing is perfect but the way in which laws are implemented SHOULD remain a separate part of the law passage.
Separate rant: I live on the west coast and am annoyed when I vote because of the number of ballot measures. None of them are sufficiently detailed and fall victim to vague language that's open to interpretation. I'd rather "hire" law makers to do the hard work of details law creation.
nradov
Nah. There's no valid reason for lawmakers to delegate so much authority to bureaucrats. We should eviscerate the administrative state and if legislators want to prohibit something then they ought to specifically write it down. If that means it becomes harder and slower to pass new laws then that's fine. It should be a careful and deliberative process.
memhole
Some of that vagueness is intentional and not always for nefarious reasons either. When getting my the FCRA cert, there definitely a few requirements that were obviously very open ended so the gov could either pursue a potential unforeseen circumstance or not stifle business. The vagaries in laws has always been interesting to me. Unfortunately, I don't remember any definitive examples since I took the test years ago now.
cal1co
I think the article did sufficient to talk about 'what would happen if one were prosecuted,' given that it's really hard to predict.
What I thought was missing was an acknowledgement of the circumstances that would cause a _prosecutor_ or _other litigant_ to decide to _bring an action to court._
Prosecutors are political, have limited resources, and are ultimately more or less accountable to electoral forces. Their prerogative is (almost) never going after every single person who does an action that is technically illegal. Factors like harm, the reputation of one's actions in their community, and their alignment with the political establishment are going to be considerations.
As written, I could see the article maybe deterring a particularly rules-abiding individual from running a potato-eating weight loss experiment with their roommates. I've seen friends and relatives get all wound up worrying about whether something is technically illegal in some marginal way.
Understanding the human factors in the pipeline between a potentially unlawful action and a consequence for it is really important. All the institutions that get anything done are keenly aware of them.
appleorchard46
It is briefly mentioned in the notes at the end that nobody has seemingly ever been prosecuted for this, and the laws are primarily aimed at institutions doing shady stuff rather than individuals wanting to feed their friends potatoes. But yes, the focus definitely isn't on that side of things.
I don't think it's fair to call that a deficiency, though; both the letter of the law and the practical applications are complex and interesting topics in their own right(s?), and either could fill a lengthy article as we see here. Both are welcome! One doesn't preclude the other. I would be interested in reading an article covering what you describe too.
nradov
Violations of federal regulations (rules) are usually handled as administrative or civil matters. Prosecution is very rare and only possible for a limited set of rule violations. If you're contemplating doing something in a regulatory gray area then you can ask the competent agency for a formal opinion letter stating whether that thing would or wouldn't be a violation. They are required to issue those letters although it can be a slow process. Once you get an official approval letter that generally shields you against any sort of enforcement action.
mmooss
> nobody seems to have ever been prosecuted
But they have been sued, sometimes for extraordinary amounts, as have their employers, corporate sponsors, etc. And they have lost funding for their research and the trust of everyone above (and of future IRBs). These are powerful disincentives.
nxobject
Conversely, a social and “in practice” reason why government-funded institutions _do_ use IRBs is to allay fiscal hawks and criticism about state funding being misused.
ahi
Also, a great deal of the research organizations like META "just do" wouldn't pass muster with any IRB worth the name.
tonetegeatinst
This is just as confusing and explosives and energetics research.
Explosives are regulated by the ATF, but can also fall under regulations of the DOT and the EPA. And if your doing any pulse power z pinch or electronic detanator experiment or research, then that's also regulated.
Keep in mind this also has state regulations and ordinance to worry about and its almost like they are trying to get you to accidentally make some mistake that can technically be a felony. And they can visit and do inspections during your hours of operations.
Potato cannons, model rockets, and innovation and experiments in your backyard seem to be impossible these days.
Do I think safety is important, sure, but I'd argue that the paperwork of regulation has stifled the common man who want to follow the laws.
Also see: export control and gpu's
oneplane
You'd think someone would notice how this process is suboptimal and either come up with a way to make the process better, or at least a way to have a single point of contact with knowledge and experience (and reputation) to deal with this for you, i.e. on a more local governmental level.
As a random citizen, you'd perhaps get away with an online self-service form, as a small company or startup you might need a form and an interview, but the entire "talk to 4 different agencies at different levels of abstraction" seems like a waste of time for everyone when it can be done in one go (provided the thing you wanted to do is common enough that it doesn't require some extreme documentation and safeguards etc).
If 100000 people were to want to try out a potato canon, it should be sufficient to do something at a local level where you select "I wanna play with a potato canon for a bit" in some drop-down, and it might ask you if you intend to shoot at people or not, and if you have enough room to experiment with this without hurting anyone (including yourself). If yes, problem solved, no need to fill out a form, if unsure or no, fill out a form since you probably are doing something novel, or something you might not want to do. If this then means some other agency at some other level needs to get notified, sure, automate that, who cares. If it turns out you filled out your form and specified radio active potatoes, perhaps one of those other agencies might want to have a chat first to see if everything is okay.
But the default of "talk to an arbitrary number of agencies in every case" is not great.
nxobject
> I'd argue that the paperwork of regulation has stifled the common man who want to follow the laws.
I’d argue that it’s not the regulations themselves, but people who want to avoid liability, whether ethical or social. (The existence of institutional review is a good argument against, say, a fiscal hawk looking at a state university’s research output to identify the existence of fraud or abuse.)
tptacek
The article suggests that simply agreeing with friends to experiment with diets would trigger the IRB requirement, and cites a NY state law requiring affiliation with an IRB-enabled institution to do human research. But the "definitions" section of that New York law says:
2. "Human research" means any medical experiments, research, or scientific or psychological investigation, which utilizes human subjects and which involves physical or psychological intervention by the researcher upon the body of the subject and which is not required for the purposes of obtaining information for the diagnosis, prevention, or treatment of disease or the assessment of medical condition for the direct benefit of the subject. Human research shall not, however, be construed to mean the conduct of biological studies exclusively utilizing tissue or fluids after their removal or withdrawal from a human subject in the course of standard medical practice, or to include epidemiological investigations.
lmm
Are you claiming that agreeing with friends to experiment with diets does not fall under that definition? If so, which part do you think it doesn't meet? If you read "physical or psychological intervention by the researcher upon the body of the subject" in such a way that a friend telling another friend to follow a given diet doesn't qualify then most medical research would be exempt too - researchers generally give subjects medication and tell them to take it rather than physically pushing it down their throat.
awo495jwal34i5
According to the Office of Compliance at the University of Texas at Austin, my graduate advisor didn't need to get IRB approval to perform experiments on human test subjects that risked severe bodily harm. I reported him, they ignored the complaint, and then gave him tenure.
They also didn't care about the part where he falsified his results or lied to his sponsors about how he was spending their money.
eesmith
This topic came up 11 years ago after Facebook did human subjects research by tweaking a ranking algorithm.
https://petrieflom.law.harvard.edu/2014/06/29/how-an-irb-cou...
asdff
This is basically why some researchers gravitate towards certain model systems in terms of abstracting the problem. It is just easier in terms of approval the simpler things get and you can basically grasp at the same sort of questions since basic cell biology is so conserved. You can do things like grow up a generation of transgenic yeast that glows green under UV light and then pour bleach over it and throw it in a carboy for disposal over the week without anyone needing approve anything but the form for the carboy to get incinerated. You try and write up a proposal for transgenic mice it might take months since the IRB needs planets to align for meetings to be scheduled. And you are paying orders of magnitude more for orders of magnitude fewer individuals in your sample that will take orders of magnitude longer to get data, than an organism that can measure reproduction rates on the log scale over hours and cost pennies on the billions.
trhway
>"HEY! YOU CAN’T DO THAT! THAT’S HUMAN SUBJECTS RESEARCH! YOU NEED TO GET APPROVAL FROM AN INSTITUTIONAL REVIEW BOARD!”
reminded - The EcoHealth Alliance application successfully granted by NIH for those Wuhan coronavirus gain-of-function experiments did have the "Human Subjects Included" and "IRB" checkboxes checked :)
asimpleusecase
I worked in environmental chemistry - commercial lab. We did not do tests on humans nor animals but we were constrained by federal law and commercial liability . If the test you were doing was covered by federal law appeared in the CFR ( code of federal regulations) you needed to make if you said you were doing a test that was detailed in law that you followed that procedure in a documented way. If you changed it in a significant way you were engaging in a felony. Then you had some Tests that were not covered by federal regulations- so how do you protect yourself from lawsuits ? ( the fact you could do a test that would cost $100 but the client was making a million dollar decision with the results meant there could be a lot of liability risk)so in those cases where there was no clear regulation you would opt for “industry best practice “ using processes, equipment, reagents that were commonly used for that test by others in the Industry. If you did that then your insurance provider would defend yourself. If you had good documentation you could prove you did the right thing the right way and win the lawsuit. Based on that background I suspect the biggest reason for review boards is liability protection.
caycep
I feel like the OP is overthinking it. The simple answer is you submit an approval for an IRB waiver, and if your IRB isn't staffed with nitwits, they usually grant it. (i.e. most retrospective studies including w/ chart review, or "minimal harm" studies i.e. w/ asking questions etc, w/o any drugs/surgeries/blood draws etc, or any risk of disseminating private health information usually get the waiver)
scraptor
This assumes that I already have "my" IRB. Most people who want publish an informal online survey do not.
nxobject
And, conversely, when people _do_ weaponize institutional review requirements to police research, you have a much more pressing and bigger problem: the existence of people who would be happy to find any mechanism to police research.
thereisnospork
I would argue that the weaponization is an intrinsic part of the nature of review boards. Not just for ethics, but accross pretty much all categories.
It's been pretty universal in my experience that people will use the power given to them to push their agenda, which is generally only somewhat aligned with the nameplate mission.
E.g. a vegan or vegetarian on an ERB has an obvious, if soft, conflict of interests that will come out in how they treat requests involving animals, or whose results might lead to future testing on animals.
caycep
But...people and people-communications make it work, i.e "Technical problems are easy, people are hard." This is a truism of life. I think this comment in the marca linux kernel brouhaha thread is relevant.
(and most IRB squabbles that involve non-invasion/low risk research never really get as bad as that, at least in my experience, knock on wood)
mmooss
Do you have experiences with IRBs? IME, most structures and people want to get along and at worst, gain favors and reduce enemies. People are short-sighted sociopaths, usually.
caycep
In which case conversations should be had about individual board member behavior. It is not a reason to not to have safety oversight.
cm2012
IRBs are one of the most wasteful and hurtful institutions in the developing world today. In order to prevent a few possible ethical lapses, they smother hundreds of good potential research.
See Scott Alexander's works on IRB review.
Viliam1234
> See Scott Alexander's works on IRB review.
Links:
https://slatestarcodex.com/2017/08/29/my-irb-nightmare/
https://slatestarcodex.com/2017/08/31/highlights-from-the-co...
mmooss
> In order to prevent a few possible ethical lapses, they smother hundreds of good potential research.
Doesn't that describe all safety and security measures? Airport security wastes millions of hours in order to prevent a few possible problems.
> ethical lapses
That's quite a euphemism for death and bodily injury. I'm sure you wouldn't mind it yourself.
There is plenty of evidence of what happens without IRBs - just look what happened before IRBs.
cm2012
Do yourself a favor and read this link: https://slatestarcodex.com/2017/08/29/my-irb-nightmare/
And yes, airport security is also a colossal waste
dekhn
IRBs dont prevent ethical lapses; they reduce the liability of the contractee/grantee.
As an academic researcher, it's most frustrating in that it feels like IRB scrutiny doesn't seem to align with risk.
I do accounting research. My human-subjects research involves surveying practitioners about their jobs, interviewing them about their experiences, and conducting very simple experiments online that ask them to make decisions. There is virtually 0 real risk to any participants of my studies. IRB does always give my studies "exempt" status, but it still has to be reviewed. And they will pester me about different things like where will store our data (Onedrive is fine, but dropbox is not for some reason). This process will typically take a couple weeks of back and forth.
Yet I have a friend who participated in a Kinesiology dissertation study where they were asked to do extremely strenuous physical activity. He fainted(!) at one point from the activity. And it seemed to them that there were relatively few safeguards from that happening. Now, I'm sure that study did have IRB approval, but it really got me thinking... are we really scrutinizing studies optimally?