The University of Arizona offers a slick dashboard that's updated daily and cleanly shows the number of positive tests, easy-to-understand graphs of case numbers, and — crucially — the percentage of total university tests that are coming back positive. The transparency is obvious in that we can all know just how poorly the university has been doing in terms of associated COVID-19 cases over the last month. University President Robert Robbins cited the specific percent positive number as a key metric he's looking at.
ASU, on the other hand, has a janky text post that it updates twice a week. What information it does release has slowly increased over time. Initially, the university said it would not release any information on case numbers associated with the institution. Then it relented under pressure and began releasing, confusingly, only the number of "active" cases associated with the university. Under further pressure, the university has revealed that it has seen more than 1,800 associated cases since the beginning of August. Still, ASU refused to disclose what percentage of tests were coming back positive, saying it wanted to wait until it had completed a statistically significant sample through its random testing program.
As of a few weeks ago, it's relented again, releasing the percent positive numbers.... but only for its random testing program.
What this means, essentially, is that it's probably excluding the majority of cases where people have symptoms from the count.
Here's how you might think the random testing program works: ASU would randomly select a student or faculty member to be tested. If a person is currently infected with COVID-19, it would count as an automatic positive. Then the total number of positives would be divided by the number of people selected for testing. But this is not the case. ASU excludes from random testing anybody who has ever tested positive for COVID-19, so that any residual amount of the virus in their system doesn't cause a false-positive result.
This would be fine if the results from the random testing program were then averaged with the results from people who sought out testing, to give an overall percent positive number. In fact, it should be done that way to ensure a student is not double-counted as a positive from both the random testing and from when they sought out a test, according to Joshua LaBaer, executive director of ASU's Biodesign Institute. People who sought out testing would be caught in a different "bucket" that would then go into the overall number, he said before ASU released the data on its random testing program.
ASU, instead, created a testing sample that excludes those with symptoms, then tried to use that to represent the prevalence of COVID-19 in the broader ASU community. One study found that 40 percent of COVID-19 cases are asymptomatic. Even considering that as young people the majority of ASU students are less likely to have symptoms, this methodology misses a huge portion of cases.
Chris Fiscus, an ASU spokesperson, defended the methodology as more accurate.
"Reporting the percent positive from people who choose to be tested is not a representative of the population," he said in an email. "Such numbers are only meaningful if you know exactly which populations were measured, and why (for example, if you test primarily people who test because they feel sick, you will get a very high percent positive, but this does not represent the population – it reflects only those that choose to come to the health center). Such numbers are often quite variable and can be misleading... Our approach is more scientifically accurate, consistent and informative. We do not have any additional updates to the data process planned at this time."
Fiscus said he would provide further information but never did.
Even if the random testing program is a better metric for measuring overall spread, by refusing to release the total percent positive — a key metric used by the state, county and many universities — ASU prevents comparisons of how widespread COVID-19 is there compared to the larger community. In fact, without the proper context, a person could be forgiven for believing ASU has had a lower incidence of COVID-19 than it actually did.
To put it another way, the percentage of tests returning positive of ASU's random testing may accurately go up and down with the spread of the virus, but because the number they're using is calculated differently than everyone else's, the university is being graded on its own scale.
No matter which statistic you release, the percent positive metric still needs to be put in context. Johns Hopkins University expert Dr. Amesh Adalja said that the general value of the percent positive measure is that it lets you know how prevalent COVID-19 is so you can adjust behavior accordingly, but it also gives a sense of the effectiveness of testing regimens.
"I think it makes sense to release at any time," he said.
What doesn't make sense, according to Will Humble, executive director of the Arizona Public Health Association, is for ASU to release only the random testing metric.
They should publish the total percentage of ASU tests coming back positive, he said, adding, "they have the data."
It's hard not to wonder about ASU's motivation for not releasing the overall percent positive — the statistic that includes both random testing and people seeking out testing. The university predicted a surge in cases would come as students returned for fall, even as it initially planned to release no information on associated cases. Facing a spike that was larger than expected as students returned en masse, the university made the decision to weather the storm and stay open, even as case numbers rose and similar universities shut down. Criticism mounted. Administrators said that, in spite of reports of widespread cases on campus and unsafe behaviors, the numbers were in part due to the testing regime.
President Michael Crow's gamble seems to have paid off — the number of active cases has continued to drop and sits at less than 200 currently — but would the public, or legislators, have allowed ASU to stay the course if they knew what the actual percent of tests coming back positive was?
In a press conference in early September, Neal Woodbury, interim executive vice president and chief science and technology officer for ASU’s Knowledge Enterprise, mentioned off-hand that between August 27 and September 2 over 7,000 students were tested, for a "raw" positivity rate of around 9 percent, while the rate among randomly selected students — the metric ASU now releases — was closer to 3 percent. Countywide, the overall positivity rate of people tested for the week ending August 30 was 4 percent. Generally, 5 percent or lower is a sign that COVID-19 is under control.
This lack of transparency comes at a cost for ASU's famous rivalry with its Tucson-based sister school. U of A also has a random testing program, but it factors the results from that program into its overall percent positive which it releases each day. If you do want to look solely at random testing as a metric, its dashboard allows that too.
"We Rate COVID Dashboards" a group of mostly Yale-based public health wonks, gives the University of Arizona a B+ rating for its dashboard. Based on the group's publicly available rating scheme, ASU's dashboard would only be a — very generous — B-, in part because it lacks that same transparency.
B+ for University of Arizona! It shares testing positivity rate, test result time, and separates employee and student data. Including more info on their testing strategy, local Covid trends, and isolation/quarantine stats would bump them to an A! https://t.co/yzb4rdPtu5 pic.twitter.com/LZ1qoQFdGM
— We Rate Covid Dashboards (@CovidDashboards) September 29, 2020
