COVID Immunity Part II
Commentary: Sheila Fallon-Friedlander, MD
Is antibody testing the answer? And do we know how to do it in a valid fashion? Accurate antibody testing should allow us to determine who has been infected, and how the virus has spread. It should enable better contact identification and testing, and help us determine how to “re-open” the country
Unfortunately there is much we still won’t know: Is immunity (positive antibody titers) truly protective? Do you need a robust response to be clinically protected?
Is severity of illness and/or treatment associated with robustness of immune response? And given the multitude of mostly non-FDA approved and variably accurate test kits out there, can we trust results? We have a long way to go, but data is accumulating that may be of help.
COVID-19 Antibody Seroprevalence in Santa Clara County, California
https://www.medrxiv.org/content/10.1101/2020.04.14.20062463v1
In this prospective study, investigators at Stanford evaluated the antibody seroprevalence of Covid-19 in 3,330 randomly selected individuals in Santa Clara County, California Participants were recruited using a Facebook message “Participate in our COVID-19 Community Study” affiliated with the Stanford Dept of Medicine. It requested healthy volunteers, and was designed to broadly canvas the county, and better target under-represented groups utilizing zip-codes, with the goal of achieving good geographic and demographic representation.
Over a 2 day period (April 3 & 4), investigators performed finger stick antibody testing on volunteers using a lateral flow immunoassay kit (Premier Biotech, Hangzhou Biotest IgM & IgG). A “drive-by” technique was used, and adult (and sometimes 1 child as well!) samples were collected while they remained in their cars. Results from 3,330 participants were evaluated and adjusted in several in several ways, accounting for 1) zip code, race & ethnicity, 2)test manufacturer’s data re test validity 3) a sample of 30 positive and 30 negative controls at Stanford and 4) a combination of the latter 2 results.
The unadjusted prevalence of antibodies in the Santa Clara population was 1.5%. (95SCI 1.11-1.97%), and the weighted population prevalence was 2.8%. When adjusted for the other considerations listed above (2,3, and 4), the prevalence increased to a range from 2.49% to 4.16% (See attached table for Confidence Intervals.) This was 50-80 fold higher than the predicted number as measured by confirmed cases at that time period. It was suggested that this information could be used to calibrate epidemic and mortality projections.
Commentary: This study utilized creative methods to attempt to measure the prevalence of disease in one California county. Using social media to attract people, representing different zip codes areas as a surrogate marker to attempt fair representation, and then utilizing “drive-by” intervention to safely retrieve samples was an ingenious approach.
However , the study has come under significant scrutiny, predominantly by statisticians for the following : 1) sampling error. Though there were three separate collection sites, all participants had to have a means to get in a car, and therefore likely underrepresented lower socio-economic groups and would likely not include those who were unwilling to leave their homes due to high COVID-19 anxiety. Those who were sick recently and perhaps wanted to confirm COVID status would be overrepresented. 2) Validity of the tests; incidence of “common cold” Covid ELISA tests in Europe has been reported at 3%, so it would be crucial to confirm that neutralization tests ruled this “common cold confounder” out. 3) Bad Confidence Intervals! : The risk of a small proportion of false-positives in a low-prevalence study becomes more meaningful and could result in large over-estimate. (2 out of 371 know negatives in manufacturesrs data re the test they were using).
It would have been nice to have concurrent PCR data to identify acute infection on all of those studied; however the delicate RNA assayed in such studies would likely not have survived well in this study design, where samples had to be transported from the collection centers back to a lab. In addition, the scarcity of PCR tests needed for clinical purposes did not allow for research use of these badly needed qPCR tests.
The authors have addressed the issue of Confidence Intervals, test sensitivity and specificity, and ran laboratory ELISA assays on 47 samples where sufficient residual sample allowed. 30/47 samples remained positive utilizing this different assay. Regarding the other 17, there was concern that the small residual sample and manner in which the sample was transported and stored may have lead to false negatives. All 17 of these “discordant result” patients were brought back and retested using ELISA, these results are pending.
Online information not yet in preprint form:
April 17, 2020 Boston Globe;
Jonathan Saltzman
MGH conducted a study of 200 Chelsea, Massachusetts residents who donated a finger-prick drop of blood at Bellingham Square on April 14 & 15. All participants appeared healthy but approximately half reported that they had suffered from at least one symptom of Covid-19 in the previous month. Anyone who tested positive for Covid-19 by nasal swab PCR was excluded. The diagnostic test used (Biomedomics, Morrisville, N.C.) is not yet FDA approved.
64 residents tested positive for antibodies, representing 32% of participants. Chelsea is a small town near Boston and approximately 65% of its population is Latino. Many of its residents work in health-related and hospitality fields, and it has the state’s highest rate of confirmed COVID-19 cases.
Covid-19 antibody seroprevalence in LA County, California.
News.usc.edu
On 4/20 the LA County Dept of Public Health in collaboration with USC released preliminary results of a study they conducted on Covid-19 seroprevalence in LA County. They identified this as a “first round” of results from ongoing studies. In another on-line statement, they mention that they planned to assess 1,000 individuals, but it is not clear from the announcement if this initial result included that many individuals.
Similar to the Santa Clara study, they utilized “drive-by” antibody testing at 6 sites on April 10 & 11. In order to select candidates for evaluation, they used a proprietary database maintained by LRW Group, a market research firm; they stated their test group was representative of the County population. A rapid antibody test was performed, whose accuracy was assessed at a lab at Stanford university (no clarity on how this was carried out).
They found that 4.1% of the county’s adult population tested positive for Covid-19 antibodies. Statistical margin of error would estimate that 2.8-5.6% of the population has antibodies. They state that this is 28 to 55 times higher than the number of confirmed cases identified at the same time in LA County.
Well, I was hoping these two reports would at least be in preprint form by now, but since that it is not the case, thought it useful to review what we have from on-line sources thus far. Several findings are striking:
- Remarkable concordance between LA county and Santa Clara County results. Though both studies have been attacked for statistical flaws (see previous review of Santa Clara preprint), it is reassuring that they are getting the same range of seroprevalence.
- Much higher seroprevalence in Chelsea, Massachusetts than either LA or Santa Clara. Chelsea had already been identified as having the highest number of confirmed Covid-19 cases in the state. Though the study may have represented a somewhat random group of people hanging out in Bellingham Square, it did not represent a random sampling of the county or state. Therefore the higher prevalence given this is the Boston “epicenter” makes sense.
- Concern re: variability introduced by different testing kits. It would be outside the range of plausible to attribute the strikingly higher prevalence noted in the Chelsea study to test kit result variability. However, there are clearly varying degrees of sensitivity and specificity amongst the many commercialized non-FDA approved ELISA studies now being marketed and utilized. (Okba NMA et al ;Emerging Infectious Diseases doi: 10.3201/eid2607.200841) We must keep such discrepancies in mind when evaluating data. The strikingly different rates of the Chelsea versus the West Coast (LA & Santa Clara) studies more likely represent….
- The beneficial effects of social distancing. Early in the COVID crisis, California aggressively instituted social isolation policies. In contrast, that did not happen in the Boston area.
- The importance of testing. All authors of all of these studies have emphasized the importance of testing (both PCR & Seroprevalence studies) to better carry out case identification and contact tracing. Such information could better inform the best manner in which to “re-open” our society. Though we cannot be sure if immunity is protective, it is the “best guess” information we can quickly gather to inform some very important decisions that state governors and local authorities must make regarding relaxation of isolation policies.
