Microsoft: ML infrastructure under atack from cryptomining gangs again

Microsoft said it detected a new malicious campaign that is hijacking Azure infrastructure typically used for machine learning operations in order to deploy cryptocurrency mining workloads.

The attacks target Kubeflow, an open-source project that allows scientists to run machine learning (ML) heavy-duty computing on top of Kubernetes clusters.

Yossi Weizman, a security researcher for Microsoft's Azure Security Center, said the attacks appear to be a continuation of a campaign that he first discovered last year in April.

While that campaign hit its peak in June last year and subsided after Microsoft's warning, new attacks were detected last month, in May 2021.

In these new attacks, Microsoft said attackers appear to have gained access to a large number of Kubeflow dashboards from where they deployed new work pipelines.

These pipelines ran a version of Google's TensorFlow open-source machine learning library but modified to mine cryptocurrency.

"The burst of deployments on the various clusters was simultaneous," Weizman said in a new report on Tuesday.

"This indicates that the attackers scanned those clusters in advance and maintained a list of potential targets, which were later attacked on the same time," he added.

Weizman believes the attackers used TensorFlow images to hide attacks because such workloads are common in Kubeflow dashboards and would be hard to spot as malicious at a superficial glance by the account owners.

However, Weizman said the attackers named all their malicious Kubeflow pods the same, following a "sequential-pipeline-{random pattern}" scheme, which can now allow account owners to inspect accounts for possible compromises.

Any malicious pods should be shut down and account credentials changed.

Weizman said the campaign is still active, and new Kubernetes clusters that run Kubeflow are still getting compromised.