When my students and I started our review of the science on COVID-19 and masks, I had no idea how much of a response it would trigger. Usually when I publish a research paper, it sails silently into the sea of science. For this report, I have been overwhelmed with inquiries, comments, and suggestions. I am really grateful that so many people care about this issue, and it confirms my observations that many sincere individuals want to get to the bottom of this issue. I'm also grateful that most of the response has been respectful and constructive, even when people disagree. Several questions keep coming up, so I decided to post this list of FAQs. Please take a look and then feel free to leave a comment if you aren't satisfied or if you have additional suggestions.
1. How did you select the papers you included?
We generated an initial list of studies using Google Scholar and Web of Science. We focused this initial search on studies specifically dealing with COVID-19 and masks. As we read those papers, we added pertinent studies they cited to our list. Additionally, we considered about a dozen studies that concerned individuals sent us. These studies were mostly about the safety of masks.
Our goal was to provide a non-technical summary for those interested in this topic. Because the medical literature on COVID-19 is so large, we cite many studies that are reviews, summaries, and commentaries on the state of the literature. While we also read many specific primary research articles, we favored citations of reviews and summaries to encourage readers to interact with the peer-reviewed holistic assessment of the state of scientific understanding. These synthesis papers are often more reliable and robust than any individual study upon which they draw. They also avoid the "single-study syndrome," where one study is interpreted without context from other work. Science is a joint venture and only reliable when the result is repeatable and falsifiable.
2. How did you ensure you weren’t getting a biased sample of studies?
We considered all the scientific studies on masks and COVID-19 that we could find, including about a dozen sent to us by individuals who do not support masking. We continue to consider all studies that are sent, so if we missed a particularly pertinent piece on COVID-19 and masks, please send it along (but please read FAQ #4 before sending unrelated studies). Likewise, if you see an error or point of confusion in the report, please let us know and we will update if we find there is an error. This has happened several times already, mainly with typos, but also with two important mistakes: 1. In the viral transmission section, we initially wrote “COVID-19 viruses” when the study in question was actually about coronaviruses generally, and 2. Our initial description of the most recent WHO recommendations were confusing. At the top of the report, we mention the “last updated on” date, so you can know when the last change was made.
3. Why do you cite some non-scientific sources such as news articles and government reports?
We cited a few of these articles for context to provide an idea of what the larger conversation is outside of science. In one section (the deep dive on mask safety), we cite some of these news reports because they have quotes from legitimate medical experts. We mention that we are citing media sources in that section to make sure people know what we are basing our interpretation on.
4. What about this or that study that was done on another disease?
There is a huge amount of research on other diseases, including several on the effectiveness of public masking in stopping those diseases (though fewer than we expected). We cite some of this research in our report to provide context. For example, it’s helpful to know that public masking was widely considered ineffective before the COVID-19 pandemic based on studies done with other diseases in very different contexts. This explains why many medical experts and agencies recommend against mass masking initially—that was the state of the available science in December 2019 and January 2020.
For our report, we focus heavily on the COVID-19-specific studies for one very important reason: COVID-19 is the disease causing this pandemic. Findings from previous outbreaks and community studies are helpful as a starting place, especially in the absence of more directly pertinent evidence. However, the pathology of each disease and the societal dynamics of each outbreak are different. Naively applying findings about influenza and masks in normal times to COVID-19 during a global pandemic is problematic, to say the least. As we evaluated the evidence from multiple studies, we gave more weight to those that dealt specifically with the current outbreak, while still considering the other studies for context.
One specific study is worth mentioning here, because of how often it is brought up in discussions of masks: MacIntyre and others 2015, "Facemasks for the prevention of infection in healthcare and community settings." This study is often brought up as evidence that masks (especially cloth masks) don't work, and that they might actually cause harm (increase risk of infection). On first read of the abstract, it is easy to get this impression. However, this is a completely incorrect interpretation of this research for several reasons. First, the study is on different diseases in different conditions (see last two paragraphs). Second, the study tested the effectiveness of masks at protecting the wearer (in this case healthcare workers), rather than source control, which is the primary purpose of masks in the COVID-19 pandemic. Third, the study did not have a "no mask" control, so they have no way of comparing cloth mask with no mask. The authors make this clear in the "Strengths and limitations" section of their paper, which appears adjacent to the abstract when you download the PDF. Importantly, the same research group did a study the following year on the effectiveness of masks at reducing infection when worn by sick individuals (MacIntyre and others 2016). They found that masks likely reduced infection, though they acknowledge that they didn't have enough participants to strongly conclude anything. Since the COVID-19 pandemic, this research group has done a systematic review of the effectiveness of masks (MacIntyre and others 2020), including their 2015 study. They conclude:
There are thousands of papers on COVID-19, which is a scientific achievement on its own. I salute the researchers and organizations that have, often at great expense, changed their research programs to address this important challenge. However, this means that we cannot cite every study on COVID. See the previous FAQs for info on how we decided what to focus on and what to cite.
6. Why do you claim there are controlled experiments when there haven’t been any randomized controlled trials of public masking?
One of the most common arguments made against masking is that there aren’t any randomized controlled trials (RCTs) of public masking and COVID-19. While it is true that there are not currently COVID-specific RCTs, a very recent systematic review by Macintyre and Chughtai of public masking studies (including many RCTs) concluded that masks are effective at slowing the spread of many respiratory diseases. The question of COVID-specific RCTs deserves some context.
We generated an initial list of studies using Google Scholar and Web of Science. We focused this initial search on studies specifically dealing with COVID-19 and masks. As we read those papers, we added pertinent studies they cited to our list. Additionally, we considered about a dozen studies that concerned individuals sent us. These studies were mostly about the safety of masks.
Our goal was to provide a non-technical summary for those interested in this topic. Because the medical literature on COVID-19 is so large, we cite many studies that are reviews, summaries, and commentaries on the state of the literature. While we also read many specific primary research articles, we favored citations of reviews and summaries to encourage readers to interact with the peer-reviewed holistic assessment of the state of scientific understanding. These synthesis papers are often more reliable and robust than any individual study upon which they draw. They also avoid the "single-study syndrome," where one study is interpreted without context from other work. Science is a joint venture and only reliable when the result is repeatable and falsifiable.
2. How did you ensure you weren’t getting a biased sample of studies?
We considered all the scientific studies on masks and COVID-19 that we could find, including about a dozen sent to us by individuals who do not support masking. We continue to consider all studies that are sent, so if we missed a particularly pertinent piece on COVID-19 and masks, please send it along (but please read FAQ #4 before sending unrelated studies). Likewise, if you see an error or point of confusion in the report, please let us know and we will update if we find there is an error. This has happened several times already, mainly with typos, but also with two important mistakes: 1. In the viral transmission section, we initially wrote “COVID-19 viruses” when the study in question was actually about coronaviruses generally, and 2. Our initial description of the most recent WHO recommendations were confusing. At the top of the report, we mention the “last updated on” date, so you can know when the last change was made.
3. Why do you cite some non-scientific sources such as news articles and government reports?
We cited a few of these articles for context to provide an idea of what the larger conversation is outside of science. In one section (the deep dive on mask safety), we cite some of these news reports because they have quotes from legitimate medical experts. We mention that we are citing media sources in that section to make sure people know what we are basing our interpretation on.
4. What about this or that study that was done on another disease?
There is a huge amount of research on other diseases, including several on the effectiveness of public masking in stopping those diseases (though fewer than we expected). We cite some of this research in our report to provide context. For example, it’s helpful to know that public masking was widely considered ineffective before the COVID-19 pandemic based on studies done with other diseases in very different contexts. This explains why many medical experts and agencies recommend against mass masking initially—that was the state of the available science in December 2019 and January 2020.
For our report, we focus heavily on the COVID-19-specific studies for one very important reason: COVID-19 is the disease causing this pandemic. Findings from previous outbreaks and community studies are helpful as a starting place, especially in the absence of more directly pertinent evidence. However, the pathology of each disease and the societal dynamics of each outbreak are different. Naively applying findings about influenza and masks in normal times to COVID-19 during a global pandemic is problematic, to say the least. As we evaluated the evidence from multiple studies, we gave more weight to those that dealt specifically with the current outbreak, while still considering the other studies for context.
One specific study is worth mentioning here, because of how often it is brought up in discussions of masks: MacIntyre and others 2015, "Facemasks for the prevention of infection in healthcare and community settings." This study is often brought up as evidence that masks (especially cloth masks) don't work, and that they might actually cause harm (increase risk of infection). On first read of the abstract, it is easy to get this impression. However, this is a completely incorrect interpretation of this research for several reasons. First, the study is on different diseases in different conditions (see last two paragraphs). Second, the study tested the effectiveness of masks at protecting the wearer (in this case healthcare workers), rather than source control, which is the primary purpose of masks in the COVID-19 pandemic. Third, the study did not have a "no mask" control, so they have no way of comparing cloth mask with no mask. The authors make this clear in the "Strengths and limitations" section of their paper, which appears adjacent to the abstract when you download the PDF. Importantly, the same research group did a study the following year on the effectiveness of masks at reducing infection when worn by sick individuals (MacIntyre and others 2016). They found that masks likely reduced infection, though they acknowledge that they didn't have enough participants to strongly conclude anything. Since the COVID-19 pandemic, this research group has done a systematic review of the effectiveness of masks (MacIntyre and others 2020), including their 2015 study. They conclude:
The study suggests that community mask use by well people could be beneficial, particularly for COVID-19, where transmission may be pre-symptomatic. The studies of masks as source control also suggest a benefit, and may be important during the COVID-19 pandemic in universal community face mask use as well as in health care settings.5. Why don’t you cite this or that specific study?
There are thousands of papers on COVID-19, which is a scientific achievement on its own. I salute the researchers and organizations that have, often at great expense, changed their research programs to address this important challenge. However, this means that we cannot cite every study on COVID. See the previous FAQs for info on how we decided what to focus on and what to cite.
6. Why do you claim there are controlled experiments when there haven’t been any randomized controlled trials of public masking?
One of the most common arguments made against masking is that there aren’t any randomized controlled trials (RCTs) of public masking and COVID-19. While it is true that there are not currently COVID-specific RCTs, a very recent systematic review by Macintyre and Chughtai of public masking studies (including many RCTs) concluded that masks are effective at slowing the spread of many respiratory diseases. The question of COVID-specific RCTs deserves some context.
RCTs are powerful tools that can—when done correctly—quantify the effectiveness of a drug or other intervention. They are often described as the gold standard for establishing cause and effect and are usually required before high-risk changes in medical practice are implemented (for example introducing a new medication). RCTs also take time and ethically require that the trial doesn’t cause undue risk or harm to the participants. In the context of a global pandemic, it is not surprising that there are not yet RCTs on COVID-19 and public masking. Indeed, concern over the lack of RCTs sparked a fruitful discussion in the medical community (mainly in March and April), as it should have, about whether to change recommendations without RCTs. Check out this paper by Greenhalgh and colleagues from April for a snapshot of what was being discussed at that point.
As this discussion played out in the medical community, other types of evidence started becoming available, including:
Given the mounting evidence, most medical experts and public health entities updated their recommendations early this year. Given the high likelihood of benefit and the extremely low risk of the intervention (there is abundant evidence that mask wearing is exceedingly safe for healthy individuals), recommending public masking was determined to be the only ethical course of action. While most papers we read continue to call for RCTs in their conclusions or discussion, they also recognized that such results could still be months or years out. Some papers have also pointed out that given the strength of the evidence, an RCT on public masking may no longer be ethical because asking people to not wear masks in areas of COVID-19 would knowingly endanger participants.
Because our report is meant for a general audience, we classified studies as controlled or observational. We used controlled to refer to any study with a manipulation (for example, implementation of healthcare worker and then patient masking in the Massachusetts study, or lab experiments on mask filtration of droplets and viral particles). We used observational to refer to studies that compare groups or outcomes without a targeted intervention. We recognize this division is imperfect, but after discussion, we decided it was a clearer distinction than observational and manipulative, the more common scientific terms.
In summary, we are not aware of any RCTs on COVID-19 and masking. However, there are many RCTs included in the best systematic review to date on public masking, which concludes that masks are effective in public settings (Macintyre and Chughtai). Claiming that there is no evidence for masks combating COVID-19 because there is no RCT for this specific disease is a fundamental misunderstanding of the scientific process. For urgent issues of great societal interest where lives or livelihoods are at stake, a full assessment of the best available information is needed. Additionally, the burden of proof should be considered in context with the real-world risks of a false positive or false negative conclusion. For the statistically inclined, this is a question of the alpha value (decision criterion) and p value (probability). There is a useful discussion going on in science generally on how to improve interpretation of scientific findings, particularly when they intersect with health and environment. I thought this editorial in Nature gave a nice, brief introduction and check out this article in Biology Letters if you want more substance. In any case, no serious health officials are advocating that we only consider RCT studies in the current pandemic, hence the change in recommendations from the CDC and WHO on masks. Thankfully, both of these agencies are following the science and adapting their recommendations as more COVID-specific data becomes available.
As this discussion played out in the medical community, other types of evidence started becoming available, including:
- Longitudinal studies quantifying infection rates and death rates after implementation of masking in different settings
- Comparative studies measuring outcomes in different regions and countries
- Case studies of specific events where the virus was or was not transmitted
- Large-scale studies on the pathology (transmission dynamics, contagiousness, symptoms, etc.) of COVID-19
- Laboratory studies quantifying the effectiveness of masks in filtering, muffling, and containing particles and droplets
Given the mounting evidence, most medical experts and public health entities updated their recommendations early this year. Given the high likelihood of benefit and the extremely low risk of the intervention (there is abundant evidence that mask wearing is exceedingly safe for healthy individuals), recommending public masking was determined to be the only ethical course of action. While most papers we read continue to call for RCTs in their conclusions or discussion, they also recognized that such results could still be months or years out. Some papers have also pointed out that given the strength of the evidence, an RCT on public masking may no longer be ethical because asking people to not wear masks in areas of COVID-19 would knowingly endanger participants.
Because our report is meant for a general audience, we classified studies as controlled or observational. We used controlled to refer to any study with a manipulation (for example, implementation of healthcare worker and then patient masking in the Massachusetts study, or lab experiments on mask filtration of droplets and viral particles). We used observational to refer to studies that compare groups or outcomes without a targeted intervention. We recognize this division is imperfect, but after discussion, we decided it was a clearer distinction than observational and manipulative, the more common scientific terms.
In summary, we are not aware of any RCTs on COVID-19 and masking. However, there are many RCTs included in the best systematic review to date on public masking, which concludes that masks are effective in public settings (Macintyre and Chughtai). Claiming that there is no evidence for masks combating COVID-19 because there is no RCT for this specific disease is a fundamental misunderstanding of the scientific process. For urgent issues of great societal interest where lives or livelihoods are at stake, a full assessment of the best available information is needed. Additionally, the burden of proof should be considered in context with the real-world risks of a false positive or false negative conclusion. For the statistically inclined, this is a question of the alpha value (decision criterion) and p value (probability). There is a useful discussion going on in science generally on how to improve interpretation of scientific findings, particularly when they intersect with health and environment. I thought this editorial in Nature gave a nice, brief introduction and check out this article in Biology Letters if you want more substance. In any case, no serious health officials are advocating that we only consider RCT studies in the current pandemic, hence the change in recommendations from the CDC and WHO on masks. Thankfully, both of these agencies are following the science and adapting their recommendations as more COVID-specific data becomes available.
7. Were you trying to prove something with this study? Aren't you biased?
Our only purpose in performing this research was to make more of the science available to the public. Here is the email I sent my lab group on July 14th, when I initiated the project:
Dear lab group,
Just reaching out to see if any of you might be interested in contributing to a one or two page report on efficacy and risks of wearing masks? Maybe it's just my social media feed, which is dominated by moms and dads, but this appears to be a hot topic right now. As there is very little trust in currently available summaries, I think a succinct and carefully nonpoliticial mini-review could do a lot of good. I don't know if it would be publishable in a peer-reviewed journal (depends on how much we want to put into it), but we could easily self publish on the lab website and do a press release.To the question of bias, yes, like all humans, we are. Our beliefs and values influence how we see the world around us. We discussed this as a group of authors and worked hard to provide a fair summary of the literature. Staying neutral was helped by the variety of political views and backgrounds among the co-authors. I take scientific integrity extremely seriously, plus I believe that I have a God-given responsibility to be honest. I'm sure there are some gaps and room for improvement, but I have done my best to accurately represent the scientific literature.
For context, personally, I went into the study somewhat mask agnostic. I thought that there might be benefits, but I also expected there to be substantial tradeoffs and side effects. I was surprised at the clarity of the results—masks are one of the fastest and safest bridges back to normal.
Especially during these politically charged times, all of us need to be extra vigilant and responsible about what we share and say. Whether it is this report or anything else, please read the article before reposting and interpreting it on social media. It’s a matter of basic honesty and integrity. Plus, what we say and share has serious consequences in the real world.
8. Are you an agent of the NWO?
I can't tell if you are joking, but no. I had to look up what NWO even stood for.
 |
| Taking a break after releasing the report. Grateful for no cell service on the Nebo Loop. |
Are you sure the death rates you have posted on your summary are up to date. They do not seem to match what I am currently reading. Your quote, "Thankfully, most people who have COVID-19 only experience flu-like symptoms such as fever, cough, difficulty breathing, and fatigue. However, COVID-19 is much more contagious than the flu, and it has a much higher death rate: 0.3 to 5.7% based on the most reliable estimates 22–26. "
ReplyDeleteThanks for the question, Frank. We used the range of the five cited studies (citations 22 through 26), which include recent and very recent estimates. From the epidemiologists I've heard from, we probably won't have a precise estimate of death rate until months from now because of how the virus is often asymptomatic. Additionally, the death rate appears to be highly variable in different regions, even after accounting for differences in testing prevalence and methods. This likely has to do with confounding factors including viral doses (amount of exposure), healthcare conditions, and environmental conditions such as air pollution.
DeleteWith schools looking at mask policies, I'm curious if any of these studies focused around school settings or effectiveness of masks on children?
ReplyDeleteWe didn't come across any school-specific mask studies, but check out citations 74-76 for a good synthesis of what is known about COVID-19 and children generally (pasted below for convenience). On a personal note, as a parent of 4, I am nervous about the prospect of so many children getting together with positive cases so high in the state. For the teachers, administrators, and students, we need to get rates down in the state. Distancing, hand washing, self-quarantining, and masks are all more effective when a large proportion of the population apply them. We are in this together.
Delete74. Munro, A. P. S. & Faust, S. N. Children are not COVID-19 super spreaders: time to go back to school. Archives of Disease in Childhood 105, 618–619 (2020).
75. Munro, A. P. S. & Faust, S. N. Addendum to: Children are not COVID-19 super spreaders: time to go back to school. Archives of Disease in Childhood (2020) doi:10.1136/archdischild-2020-319908.
76. Puntis, J. W. Is it really time to go back to school? Archives of Disease in Childhood (2020) doi:10.1136/archdischild-2020-319911.
careful thinking, careful responses. thank you ben
ReplyDeleteI am just curious why you have some different conclusions than this guy. It looks like he has reviewed several studies, too.
ReplyDeletehttps://www.rcreader.com/commentary/masks-dont-work-covid-a-review-of-science-relevant-to-covide-19-social-policy
Thank you!
Thanks for the question. We have received links to that article from several readers. There are three main reasons for the different conclusions. First, the author doesn't cite any COVID-specific studies. The studies that are cited are based on other diseases. Second, the cited studies investigate the question "do masks work to protect healthcare workers from external infections"? The question for effectiveness of masks in the current pandemic is "does public masking limit the transmission of COVID-19"? These are fundamentally different questions that require different experimental deisgns. Third, the author arbitrarily only considers one type of evidence (randomized control trials), completely ignoring the strong evidence from comparative, longitudinal, and laboratory studies. See FAQs 4 and 6 for context.
DeleteWe actually contacted the publisher of this study more than a week ago. They claim: "We pledge to publish all letters, guest commentaries, or studies refuting [Rancourt's] general premise that this mask-wearing culture and shaming could be more harmful than helpful. Please send your feedback to info@rcreader.com." Unfortunately they never responded or published our letter.
Unfortunately for you and other researchers, because of how politicized this has become, almost any research done by anyone since March 1 is considered "biased" in some way. I appreciate your work, but have also noticed now the mainstream media is not really repeating your conclusions, which are summed up pretty well by Unknown (1 & 2) below. Sadly, we are in a time where people are going to believe what they want to believe. But, keep trying!!
DeleteI think the science behind the covid pandemic is supportive of these three conclusion about face masks:
ReplyDelete1) if you're sick and have covid; wearing a face mask will stop the spread of 90 percent of the virus you are spreading.
2) if you wear a face mask it will not protect you from getting the covid virus
3) the spread of covid19 by asymptomatic individuals is rare.
Does your study agree with these three conclusion? or do you disagree?
Yes for #1. Probably for #2. Incorrect for #3. There is lots of evidence of asymptomatic spread (many estimates are around 50% or more of new cases). See the main report for the details.
DeleteThank you Benabbo, I am not a scientist just a decision maker, who tries to avoid decisions made on perception and emotion, and trying to understand the debate.
ReplyDeleteI want to respectfully follow up on your response to #3....lots of evidence??? I read footnote 1 and 39. I think you mean lots of evidence for pre-symptomatic transmission, not to be confused with asymptomatic...an individuals who never experience symptoms. WHO currently states we don't know, previously stated it was rare.
Scientific data estimates that 40 -50% of the cases are asymptomatic. Also, scientific data, that most children are asymptomatic and that there are no documented cases worldwide that a child has given the covid19 to an adult, which would support #3 "the spread of covid19 by asymptomatic individuals is rare." With that explanation do you still believe #3 to be incorrect?
If you do accept #3, it would follow that the science has not changed, that is a good thing. And, that science still supports both opinions, which are not science based just opinion. The first, being the general public should wear masks, the current opinion; versus the general public will not benefit much if at all by wearing a mask, the previous opinion in January and February.
I think that is the true scientific data behind wearing or not wearing masks....there is a lot a opinion involved, but only the science should determine which is correct.
Hi Unknown,
DeleteYou are right about the distinction between asymptomatic and pre-symptomatic. Usually, when talking with a nontechnical audience, I try to say "people without symptoms" (including both pre- and asymptomatic), which is what I meant in my last reply. There are reports of true asymptomatic spread, but they appear to account for a pretty small minority of cases. However, the case for pre-syptomatic spread is well established and should be added to the list of "supported conclusions." A lot has come out in the last two weeks and we are going to update the report this next week. Thanks for your comments and questions.
Your link to the compiled PDFs of each cited study in the online folder is not working. We were hoping to share some of the pdfs as we get questions. Can you let us know when it is working again? Thanks!
ReplyDeleteHi Unknown. I just checked in my browsers (Chrome desktop and mobile and Safari mobile) and it is working. Here is the direct link from the study: https://byu.box.com/s/k5xsy2wxs7njpmpye6cgi07jgwgd15j2. It should take you to a "Box" folder, which has six subfolders. Please let me know if it isn't working for you.
DeleteThank you! Firewall issue on our end.
ReplyDeleteThe last update of this review was over a year ago. Are there any plans to update this study? I found it very useful last year ubut as masks are still a hot-button issue, I would love some more up-to-date information. If your lab is not planning to update this, can you direct me to another source with reliable information on the current state of mask research?
ReplyDelete