DHS undersecretary: Log4j problem is not over, may take ‘a decade or longer’

Concern around Log4j is far from over, according to the chairs of the Cyber Safety Review Board, which recently released a wide-ranging report on the bug’s origins.

Rob Silvers, the undersecretary for policy at the U.S. Department of Homeland Security and board co-chair, spoke at the Black Hat conference on Thursday about "the largest mass scale cyber response in history" after the vulnerability was discovered in December 2021. 

While Silvers and co-chair Heather Adkins, Google's vice president of security engineering, lauded the industry's efforts to address the issue, both acknowledged that it will be years before Log4j is found and addressed in all its forms. The vulnerability in the open-source software opened up hundreds of millions of devices to exploitation.

“Log4j is not over. This was not a historic look back and now we're in the clear,” Silvers said. “The board found that it is likely that organizations are going to be dealing with continued Log4j exposure for years to come, maybe a decade or longer.”

The vulnerability in the widely-used Log4j Java library was discovered by an engineer working for Alibaba in early December and reported to the Apache Software Foundation. 

The controversy around the bug became the focus of the inaugural report in July from the DHS's Cyber Safety Review Board — which found that, despite efforts by organizations across the federal and private sectors to protect their networks, Log4j had become an “endemic vulnerability” — meaning unpatched versions of the omnipresent software library will remain in systems for the foreseeable future.

Silvers said the 15-person board — which spent five months on the report — filled a gap in the cybersecurity ecosystem, given its expertise from both the public and private sector.

He noted that the board offered confidentiality protections to the more than 80 organizations that spoke about Log4j and anonymized the information provided.

Adkins said the board spoke directly with Apache Software Foundation, cybersecurity vendors who had watched the ecosystem respond in real-time, and even representatives from the Chinese government.

Chinese government response

One of the most controversial parts of the report was the confirmation that the Chinese government had not exploited the vulnerability before Log4j was widely disclosed. Because it was originally discovered by an Alibaba employee, there were fears that it had been handed over to the government for malicious use. 

Silvers said data from several companies confirmed that it was reported first to Apache, before the Chinese government was notified, quelling rumors that it was exploited by Chinese advance persistent threat groups before organizations had a chance to patch it. 

The board worked with Cloudflare and found that there was no evidence of Chinese exploitation before the existence of the vulnerability was made public, according to Silvers.

In his remarks, Silvers highlighted the unique disclosure process surrounding Log4j. Regulations in China mandate that the government be informed first of vulnerabilities, and reports emerged earlier this year that the government sanctioned Alibaba for reporting Log4j to Apache first.

Silvers said the board had asked the Chinese government directly about this, and while officials had been open and honest about most things related to Log4j, they declined to comment about whether Alibaba faced penalties.

“We think that this was a good vulnerability disclosure process and it was troubling to us that there would be some kind of punishment,” Silvers said. “The board expressed concern that [the regulations] could give the Chinese government early access to very serious problems before patches could be issued to network defenders.” 

More open source support

Adkins said Log4j was emblematic of a larger issue with widely-used open source software, which is often created and run by small teams of volunteers. 

“The tension that came out in the report was that the benefit of the open source community was them having the freedom to run software the way they want to run software,” Adkins said. “But we also learned that there are millions if not billions of people relying upon them getting it right.”

Adkins floated several potential corrective measures, which included finding more resources for the volunteer community and offering developer security training as well as security audits.

The board looked into whether someone could have caught Log4j before it entered into production. They determined that a very specific kind of expertise would have been required during a code audit to flag the Log4j problem. 

“These communities just don't have the resources available to them at the time that they write code,” she said. 

She added that there needs to be an incentive structure to improve security, such as scorecards, which would rate open source projects. Projects with tighter security would get better scores.  

CISA’s innovative GitHub repository

One of the other findings from the report that stood out to Silvers was the critical role the Cybersecurity and Infrastructure Security Agency (CISA) played in sharing information and helping organizations address Log4j.

When a flood of information emerged on Twitter about the bug, CISA helped sort through the most important knowledge and collate resources for organizations. 

Silvers said CISA also did something unprecedented among U.S. agencies and global cybersecurity bodies: it worked with security researchers to build a GitHub repository cataloging all software products that were known to have Log4j. Part of what made Log4j so concerning was its ubiquity, embedded in a variety of products with many organizations unsure if it was used in their tools or not.

“It was so hard to know what software products have Log4j, so if you're a CISO [chief information security officer] and you can't figure it out, you can just go to CISA and this GitHub repo and you can say, okay, I know I have these software packages,” Silvers said. 

“The board could not find another example, from the U.S. government or any other government, of something similar... This showed that CISA has this role of being kind of like a lighthouse."

Silvers added that the report’s findings confirmed the board’s viability, and explained that they are now working on finding permanent staffing and refining its procedures.

He told the audience that the board — modeled after the National Transportation Safety Board, which investigates plane and helicopter crashes — will now be “an enduring institution in the cybersecurity ecosystem.”

While Silvers would not say what the board will dig into next, he said they are “tuning it up for future reviews, which will surely come.”