News You Can Use

Three Years Trending: Curating COVID-19 Article Lists

Learn more about a process that may be useful for other topics, too.

BY CHRIS BELTER, NICHD

Three years ago, I started sending lists of research articles about COVID-19 that were trending on social media to the NIH COVID19RESEARCH listserv. In this article, I reflect on how that started, how it has gone, and where it might be going.

The idea for these lists came in early 2020. As a librarian at the time, I was focused on helping colleagues keep up with the emerging research on COVID-19. By early April, I was generating daily lists of all new COVID-19 articles for colleagues to manually screen for importance. The lists started at around 500 articles per day but grew to over 1,000 per day within just two weeks. No one could keep up at that pace, so I started looking for ways of filtering the lists to highlight articles that were likely to be important for our HHS colleagues.

Altmetric attention scores seemed to be a promising way to do that. Altmetric scores measure online activity associated with articles, such as how often they are posted about in social media, saved to reference management software, or mentioned in news stories. I was investigating altmetrics for other purposes and noticed that articles about COVID-19 had especially high scores, so I started using altmetric scores to filter my COVID-19 lists. By late April 2020, I had developed the method I continue to use today.

Each day, I used altmetric.com to obtain the 50 articles with the highest altmetric attention scores over the previous two days and then filtered the list to only retain articles about COVID-19. The resulting “trending articles” list became the list I sent, and still send, to the COVID19RESEARCH listserv.

The method worked because most of the articles with the highest altmetric scores each day were about COVID-19. From May 2020 through December 2021, around 30 of the top 50 articles each day were about COVID-19, and some days it was over 40. But starting in early 2022, the number of COVID-19 articles in the top 50 began to decline and has continued to decline since. This is partly because the total number of new COVID-19 articles is declining, but it is mostly because the articles that are published do not attract as much attention as they used to. People are moving on.

The articles included in these lists tended to be on topics that affect people’s lives. Vaccines, treatments, transmission, new variants, and long COVID all attracted significant attention. Most of the major discoveries on COVID-19 made the lists.

Of course, altmetric attention is not always positive. Many of the articles with the highest scores were also the most controversial. Topics such as the origin of COVID-19, masking, lockdowns, hydroxychloroquine, and ivermectin appeared often in these lists, and many top articles were later retracted. It seemed important, however, to keep these articles in the lists to ensure readers knew what was being discussed.

Despite these drawbacks, the lists have inspired readers to ask about trending article lists on other topics. Such lists can be generated in the same way as the COVID-19 list by changing certain parameters.

For example, I have developed lists of articles on topics of interest to the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) that were in the top 500 for attention score over the prior week and lists of articles supported by NICHD in the top 500 over the prior month. To generate these lists, I pull the top 500 articles for the designated timeframe and apply the relevant topic filter to the resulting list. Expanding the number of top articles retrieved is especially important because few topics have articles in the top 50 in any timeframe.

More detailed instructions and sample code for generating such lists are available in my trending articles GitHub repository.

The trending COVID-19 lists will end at some point. As pandemic response measures wind down and as altmetric attention shifts to other topics, the need for and method of generating these lists are both declining. Until then, I will keep sending the lists and engaging with the NIH COVID-19 Scientific Interest Group.

Chris Belter is the lead analyst and evaluator for the Division of Extramural Research at NICHD, where he analyzes NICHD’s research portfolios to inform strategic decisions and coordinates analytical activities across the Division. Prior to joining NICHD, he was a bibliometrics informationist at the NIH Library. In his spare time, he enjoys hiking, travel, and cooking Italian food.