Highlights From The Comments On My IRB Nightmare

Many people took My IRB Nightmare as an opportunity to share their own IRB stories. From an emergency medicine doctor, via my inbox:

Thanks for the great post about IRBs. I lived the same absurd nightmare in 2015-2016, as an attending, and it’s amazing how your experience matches my own, despite my being in Canada.

One of our residents had an idea for an extremely simple physiological study of COPD exacerbations, where she’d basically look at the patient and monitor his RR, saturation, exhaled CO2 temporal changes during initial treatment. Just as you were, I was really naive back in 2015, and expected we wouldn’t even a consent form, since she didn’t even have to *talk* to the patients, much less perform any intervention. Boy I was wrong ! The IRB, of course, insisted on a two-page consent form discussing risks and benefits of the intervention, and many other forms. I had to help her file over 300 pages (!) of various forms. Just as in your case, we had to abandon the study when, two years after the first contact with the IRB, they suggested hilarious “adjustments” to the study protocol “in order to mitigate possible risks”.

From baj2235 on the subreddit:

Currently working in a brand new lab, so one would think I’d have a lot to do. Instead, thus far my job has consisted of sitting in an empty room coming up with increasingly unlikely hypotheses that will probably never be tested because our IRB hasn’t approve our NOU (Notice of use) forms. For those who don’t know, NOUs are essentially 15 page forms that say “We study these this, and we promise to be super responsible while studying it.” We have 4 currently awaiting approval, submitted in May. The reason they aren’t approved yet? The IRB hasn’t met since June, and likely won’t meet again this month because of frickin’ Harvey. Which in essence means the fine American taxpayer has essentially been paying me to sit in a room and twiddle my thumbs for the past 3-months because I can’t even grow E. coli without a frickin’ NOU.

From Garrett in the comments:

Oh, dear! I’ve actually been through this. I work in tech, but volunteer in EMS. As a part of wanting to advance the profession of EMS I figured I’d take on a small study. It would be a retrospective study about how well paramedics could recognize diabetic ketoacidosis (DKA) and Hyperosmolar hyperglycemic state (HHS) in comparison to ER doctors. […]

I had to do the “I am not a Nazi” training as well. In order to pass that, I had to be able to recite the FDA form number used as a part of new implantable medical device investigations. I wasn’t looking at a new device. I wasn’t looking at an old device. I was going to look at pairs of medical records and go “who correctly identified the problem?” […]

It’s now ~5 years after IRB and because of all of the headaches of getting the data to someone who isn’t faculty or a doctor, and who doesn’t have a $100k+ grant, I still don’t have my data. I need to send another email. I’m sure we can get an IRB extension with a few more trees sacrificed.

From Katie on Facebook:

I used to work at an fMRI research center and also had to take the Don’t Be a Nazi course!

My favorite story about the annoying IRB regulations is how they insisted on an HCG (pregnancy) test for our volunteers, despite the fact that MRI has no known adverse effect on pregnancy. So, fine, extra caution against an unknown but possible risk, sure.

But they insisted on a *blood test* done days in advance instead of five minute urine dip stick test that *actual doctors offices* would use. You know what doesn’t have risks? Peeing in a cup. And what does have risks of fainting, infection, collapsing a vein, etc? A blood draw.

Of course, we had an extra consent form for them to sign, about the risks of the blood draw the IRB was helpfully insisting on.

From Hirsin on Hacker News:

My freshman year of college I proposed a study to our hospitals IRB to strap small lasers to three week old infants in an effort to measure concentrations of a chemical in their blood. The most frustrating part was not the arcane insistence on ink and bolded study names, but the hardline insistence that it was impossible (illegal) to test the device before getting IRB approval – even on ourselves. Meaning that without any calibration or testing, our initial study would likely come back with poor results or be a dud, but we couldn’t find out until we filled out all the paperwork.

What is our country coming to when you can’t even attach lasers to babies anymore?

Some of the other stories were kind of cute. Dahud in the comments:

I’ve had exactly one interaction with an IRB – in 6th grade. My science fair project involved studying the health risks of Communion as performed in the Episcopal church. (For those unfamiliar, a priest lifts a silver chalice of port wine to your lips, you take a small sip, and the priest wipes the site with a linen cloth and rotates the chalice.)

Thing was, the science fair was being held by a Baptist University. The IRB was really not fond of the whole wine thing. They wanted me to use grape juice instead, in the Baptist fashion. I, as a minor, shouldn’t be allowed anywhere near the corrupting influence of the communion wine that I had partaken of last Sunday.

Of course, the use of communion wine was essential to the study, so we reached a compromise. I would thoroughly document all the sample collection and preparation procedures, and let someone of age carry out the experiment while I waited in the hall.

And of course James Miller is still James Miller:

Several forms I have to sign to do things at my college ask if what I will be doing will expose anyone to radiation. Although I’m an economist, this has caused me to think of experiments I could do with radiation such as secretly exposing a large number of students to radiation and seeing, years later, if it influences their income.

Along with these, a lot of other people were broadly sympathetic but thought that if I knew how to play the system a little better, or was somewhere a little more research-focused, things might have gone better for me. Virgil in the comments:

FWIW, I’m a graduate student in the Social Sciences. Our IRBs have the same rules on paper, but we get around it by using generic versions of applications with the critical info swapped out, or just ignoring them altogether. Though we don’t have to face audits, so…I’ve found that usually if you make one or two glaring errors in the application on purpose, the IRB will be happy to inform you of those and approve it when you correct them. They just want to feel powerful / like they’re making a difference, so if you oblige them they will usually let you through with no further hassle.

From Eternaltraveler in the comments:

Most of the bureaucracy you experienced is institutional and not regulatory. I have done research both in an institutional setting (turn around time at UC Berkeley=5 months to obtain ethics approval and countless hours sucking up to self important bureaucrats who think it’s their sacred duty to grind potentially life saving research to a halt over trivia they themselves know is meaningless), and as an entrepreneur and PI at a biotech startup (turn around time for outsourced IRB=5 days with reasonable and informed questions related to participants well being), where we also do quite a bit more than ask questions. FYI the kind of research I did at UC Berkeley that took 5 months for approval has absolutely no regulatory requirements outside of it.

And from PM_ME_YOUR_FRAME on the subreddit (who I might hunt down and beg to be my research advisor if I ever do anything like this again):

Amateur. What you do is you sweet talk the clinicians into using their medical judgement to adopt the form as part of their routine clinical practice and get them to include it as part of the patient’s medical records. Later… you approach the IRB for a retrospective chart review study and get blessed with waived consent. Bonus: very likely to also get expedited review.

And this really thorough comment from friendlygrantadmit:

I’m not an expert in IRB (although that’s kind of my point–getting to that), but I think your headaches were largely institutional rather than dictated by government fiat. Let me explain…

I used to be the grant administrator for a regional university while my husband was a postdoc at the large research university 20 miles away. Aside from fiscal stuff, I was the grants office, and the grants office was me. However, there was an IRB of longstanding duration, so I never had to do much other than connect faculty whose work might involve human subjects with the IRB Chair. I think I was technically a non-voting member or something, but no one expected me to attend meetings.

This was in the process of changing when I left the university because my husband’s postdoc ended and we moved. It was a subject that generated much bitterness among the small cadre of faculty involved. Because I was on my way out, I never made it my business to worry about nascent IRB woes. My understanding was that they had difficulty getting people to serve on the IRB because it was an unpaid position, but as the university expanded, they were going to need more and different types of expertise represented on the IRB. I can’t be more specific than that without basically naming the university, at which I was very happy and with which I have no quarrel. I never heard any horror stories about our IRB, and I would have been the first point person to hear the them, so I presume it was fairly easy to work with.

Anyway, the IRB auditing stuff you outline is just insane. The institutional regulations pertaining to the audits were probably what generated the mind-numbing and arcane complexity of your institution’s IRB. Add in finicky personalities and you have a recipe for endless hassle as described.

So here’s the other thing to bear in mind: almost everyone in research administration is self-trained. I think there are a few programs (probably mostly online), but it’s the sort of field that people stumble into from related fields. You learn on the job and via newsletters, conferences, and listservs. You also listen to your share of mind-numbing government webinars. But almost everyone–usually including the federal program officers, who are usually experts in their field but who aren’t necessarily experts in their own particular bureaucracy–is just winging it.

Most research admins are willing to admit the “winging it” factor among themselves. For obvious reasons, however, you want the faculty and/or researchers with whom you interact to respect your professional judgment. This was never a problem at my institution, which is probably one reason I still have a high opinion of it and its administration, but I heard plenty (PLENTY) of stories of bigshot faculty pulling rank to have the rules and regulations bent or broken in their favor because GRANT MONEY, usually with success. So of course you’re not going to confess that you don’t really have a clue what you’re doing; you’re just puzzling over these regulations like so many tea leaves and trying to make a reasonable judgment based on your status as a reasonably well-educated and fair-minded human being.

What this means in practice is almost zero uniformity in the field. Your IRB from hell story wasn’t even remotely shocking to me. Other commenters’ IRB from just-fine-ville stories are also far from shocking. Since so few people really understand what the regulations mean or how to interpret them, let alone how to protect against government bogeymen yelling at you failing to follow them, there is a wild profusion of institutional approaches to research administration, and this includes huge variations in concern for the more fine-grained regulatory details. It is really hard to find someone to lead a grants or research administration office who has expertise in all the varied fields of compliance now required. It’s hard to find someone with the expertise in any of the particular fields, to be honest.

There is one area in which this is not so much true, and that is financial regulations. Why? Well, for one thing, they’re not all that tricky–I could read and interpret them with far greater confidence than many other regs, despite having a humanities background. The other reason is that despite their comparative transparency, they were very, very widely flouted until the government started auditing large research institutions around 15-ish years ago.

I have a short story related to that, too–basically, when my husband started grad school, we would frequently go out to dinner with his lab group and advisor. The whole tab, including my dinner and that of any other SOs and all alcoholic beverages (which can’t be paid for with grant funds aside from narrow research-related exceptions), would be charged to whichever research grant because it was a working meal. I found it mildly surprising, but I certainly wasn’t going to argue.

Then the university got audited and fined millions of dollars for violations such as these and Found Religion vis-à-vis grant expenditures.

With regards to your story, I’m guessing that part of the reason the IRB is such a big deal is that human subjects research is the main type of research, so they are really, really worried about their exposure to any IRB lapses. However, it sounds like they are fairly provincial in that they aren’t connected to what more major research institutions are doing or how they handle these issues, which is always a mistake. Even if you don’t think some other institution’s approach is going to work for you, it’s good to know about as many different approaches as you can to know that you’re not some insane outlier as your IRB seems to be. As others have noted, it also sounds like that IRB has become the fiefdom of some fairly difficult personalities.

I already know how extensive, thorough, and helpful training pertaining to IRB regs is, which is not very. I remain deeply curious about the qualifications and training of your obviously well-intentioned “auditor.” My guess is she inherited her procedures from someone else and is carefully following whatever checklist was laid down so as not to expose herself to accusations of sloppiness or lack of thoroughness … but that is only a guess.

Even though I hate hearing stories like yours–there is obviously no excuse for essentially trying to thwart any and all human subjects research the way your IRB did–I am sympathetic to the need for some regulations, and not just because of Nazis and the Tuskeegee Syphilis experiments. I’m sympathetic because lack of oversight basically gives big name researchers carte blanche to ignore regulations they find inconvenient because the institutional preference, barring opposing headwinds, will always be to keep researchers happy.

Some people thought I was being too flippant, or leaving out parts of the story. Many of them mentioned that the focus on Nazis overshadowed some genuinely horrific all-American research misconduct like the Tuskegee Syphilis Experiment. They emphasized that my personal experience doesn’t overrule all of the really important reasons IRBs exist. For example, tedwick from the subreddit:

So, I wrote out all of the ways in which Scott’s terrible IRB experience was at least in part self-imposed, and how a lot of the post was about stuff that’s pretty straightforward, but it was kind of a snarky comment. Not unlike his post, but you know, whatever. Long story short, I’ve done similar work (arranged a really simple survey looking at dietary behaviors in kids, another IRB-protected group) and had to interface with the IRB frequently. Yep, it can be annoying at times. But the reason they ask people like Scott whether they’re going to try anything funny with prisoners is because sometimes people like Scott are trying something funny with prisoners. Just because Scott swears that he’s not Mengele doesn’t mean that he’s not going to do something dumb a priori. As his experience with expedited review might indicate, sitting down with an IRB officer for maybe 30 minutes would have cleared up a lot of things on both sides.

Is there room for IRB reform? Sure! Let’s make the easy stuff easy, and let’s make sure IRB intervention is on actual substance. I’m with him on this. However, a lot of the stuff Scott is complaining about doesn’t fall into that category (e.g. “why do all the researchers have to be on the IRB!?”). I get that the post was probably cathartic for Scott to write, but there are plenty of great researchers who are able to navigate this stuff without all the drama. “Bureaucracy Bad” is a fine rallying cry and all that, but most of the stuff Scott is complaining about is not all that hard and there for a reason.

And kyleboddy from the comments:

Nazism isn’t the reason IRBs exist. Far worse. American unethical experimentation is, and omitting it is a huge error. Massive and bureaucratic oversight exists because American scientists would stop at nothing to advance the field of science.

The Tuskegee Syphilis Experiment is the landmark case on why ethical training and IRB approval is required. You should know this. This was 100% covered in your ethical training.

I get why IRB approval sucks. My Informed Consent forms get banged all the time. But we’re talking about consent here, often with disadvantaged populations. It pays to be careful.

Last, most researchers who need speed and expedited review go through private IRB organizations now because the bureaucracy of medical/university systems is too much to handle. Our private IRB that we engage with sends back our forms within a week and their fees are reasonable. Their board meets twice per week, not once per month. The market has solved at least this particular issue.

EDIT: Private IRBs do not care about nonsensical stuff like the Principal Investigator having an advanced degree or being someone high of stature. (For example, I am a college dropout and have had multiple IRB studies approved.) Only bureaucratic, publicly-attached ones do. That’s a very reasonable complaint.

A lot of these are good points. And some of what I wrote was definitely unfair snark – I understand they’ve got to ask you whether you plan on removing anyone’s organs; if they don’t ask, how will they know? And maybe linking to Schneider’s book about eliminating the IRB system was a mistake – I just meant to show there was an existing conversation about this. I definitely didn’t mean to trivialize Tuskegee, to say that I am a radical Scheiderian, to act like my single experience damns all IRBs forever, or to claim that IRBs can’t possibly have a useful role to play. I haven’t even begun to wade into the debate between the critics and proponents of the system. The point I wanted to make was that whether or not IRBs are useful for high-risk studies, they’ve crept annoyingly far into low-risk studies – to the detriment of everyone.

Nobody expects any harm from asking your co-worker “How are you this morning?” in conversation. But if I were to turn this into a study – “Diurnal Variability In Well-Being Among Office Workers” – I would need to hire a whole team of managers just to get through the risk paperwork and the consent paperwork and the weekly reports and the team meetings. I can give a patient twice the standard dose of a dangerous medication without justifying myself to anyone. I can confine a patient involuntarily for weeks and face only the most perfunctory legal oversight. But if I want to ask them “How are you this morning?” and make a study out of it, I need to block off my calendar for the next ten years to do the relevant paperwork.

I feel like I’m protesting a police state, and people are responding “Well, you don’t want total anarchy with murder being legal, do you?” No, I don’t. I think there’s a wide range of possibilities between “police state” and “anarchy”. In the same way, I think there’s a wide range of possibilities between “science is totally unregulated” and “scientists have to complete a mountain of paperwork before they can ask someone how their day is going”.

I dare you to tell me we’re at a happy medium right now. Go on, I dare you.

I regret to say this is only getting worse. New NIH policies are increasingly trying to reclassify basic science as “clinical trials”, requiring more paperwork and oversight. For example, under the new regulations, brain scan research – the type where they ask you to think about something while you’re in an fMRI to see which parts of your brain light up – would be a “clinical trial” since it measures “a health-related biomedical or behavioral outcome”. This could require these studies to meet the same high standards as studies giving experimental drugs or new gene therapies. The Science Magazine article quotes a cognitive neuroscientist:

The agency’s widening definition of clinical trials could sweep up a broad array of basic science studies, resulting in wasted resources and public confusion. “The massive amount of dysfunction and paperwork that will result from this decision boggles the mind” and will hobble basic research.

A bunch of researchers from top universities have written a petition trying to delay the changes (if you’ve got an academic affiliation, you might want to check and consider signing yourself). But it’s anyone’s guess whether they’ll succeed. If not, good luck to any brain researcher who doesn’t want to go through everything I did. They’ll need it.