Thousands of companies using Ray framework exposed to cyberattacks, researchers say

Researchers are warning that hackers are actively exploiting a disputed vulnerability in a popular open-source AI framework known as Ray.

This tool is commonly used to develop and deploy large-scale Python applications, particularly for tasks like machine learning, scientific computing and data processing.

According to Ray’s developer, Anyscale, the framework is used by major tech companies such as Uber, Amazon and OpenAI.

Researchers at Israel-based cybersecurity firm Oligo Security found that thousands of publicly exposed Ray servers worldwide were compromised due to the vulnerability tracked as CVE-2023-48022, and dubbed by the company as ShadowRay.

This flaw allows attackers to take control of companies' computing power and leak sensitive data. Impacted organizations came from many industries, according to the report, including medical companies, video analytics firms and elite educational institutions. Some of the affected devices were compromised for at least seven months.

The hackers leaked a trove of sensitive information via the compromised servers, researchers said.

For example, some of the credentials required to access a database were exposed, allowing attackers to silently download complete databases. On some machines, attackers could modify the database or encrypt it with ransomware.

Other leaked information reportedly included password hashes, Stripe tokens that attackers could use to drain payment accounts by signing their transactions on the live platform, and Slack tokens that could allow attackers to read an impacted organization’s Slack messages or send arbitrary messages to certain channels on the platform.

Shadow vulnerability

Discovered back in 2023, CVE-2023-48022 was not initially considered a serious risk and was not promptly fixed. 

According to the National Vulnerability Database, the flaw allows a remote attacker to execute arbitrary code via the job submission API — an interface provided by a framework to submit computational tasks or jobs for execution.

Anyscale has stated that the report on this flaw is irrelevant because Ray "is not intended for use outside of a strictly controlled network environment." In fact, the company said that it is not a bug but rather a deliberate design decision.

Due to the controversy surrounding whether it constituted a vulnerability, ShadowRay did not appear in several databases. Oligo Security refers to it as a “shadow vulnerability” since security teams around the world “had no idea that they could be at risk.”

Researchers said that the ShadowRay exploit is “the first known instance of AI workloads actively being exploited in the wild through vulnerabilities in modern AI infrastructure.” 

A typical AI environment contains “a wealth of sensitive information,” making it a lucrative target for hackers. In addition, AI models typically run on expensive, high-powered machines, which makes the computing power they use a prime target for attackers.

“The AI infrastructure can be a single point of failure for AI-driven companies — and a hidden treasure for attackers,” researchers said.

Cryptomining campaign

During analysis, Oligo discovered hundreds of compromised GPU clusters, each containing numerous nodes connected over the network. Attackers exploited some of the GPUs on these nodes for cryptocurrency mining.

The total value of compromised machines and compute power is estimated at nearly a billion dollars, based on clusters observed by Oligo Security in recent weeks. 

During cryptomining campaigns using ShadowRay for infiltration, hackers installed cryptocurrency miners such as XMRig Miner, NBMiner and the Java-based Zephyr miner. The majority of reported GPU clusters were already compromised with crypto-miners, and were allegedly targeted by the same attackers, researchers said.

Although the specific hacker group behind the campaign hasn’t been identified, researchers believe that the threat actors are likely part of a well-established hacking group.