Protect AI and Hugging Face have announced a significant milestone in their six-month partnership to enhance machine learning (ML) model security through Guardian's scanning technology. Since October 2024, Protect AI has expanded Guardian's detection capabilities, improving existing threat detection and launching four new modules. The collaboration has resulted in the scanning of over 4.47 million unique model versions on the Hugging Face Hub, with 352,000 unsafe or suspicious issues identified across 51,700 models.

What Happened

In October 2024, Protect AI and Hugging Face partnered to enhance ML model security through Guardian's scanning technology. This partnership was a natural fit, as Hugging Face aims to democratize the use of open-source AI while prioritizing safety and security. Protect AI is building the guardrails to make open-source models safe for all users.

Since then, Protect AI has significantly expanded Guardian's detection capabilities, improving existing threat detection and launching four new modules: PAIT-ARV-100, PAIT-JOBLIB-101, PAIT-TF-200, and PAIT-LMAFL-300. These updates enable Guardian to cover more model file formats and detect additional sophisticated obfuscation techniques, including the high-severity CVE-2025-1550 vulnerability in Keras.

Through enhanced detection tooling, Hugging Face users receive critical security information via inline alerts on the platform and gain access to comprehensive vulnerability reports on Insights DB. Clearly labeled findings are available on each model page, empowering users to make informed decisions about which models to integrate into their projects.

Background and Context

The partnership between Protect AI and Hugging Face is part of a broader effort to address the growing concern of ML model security. As more organizations adopt AI and ML technologies, the risk of malicious models being used in production environments increases. The Hugging Face Hub hosts over 2.2 million models with roughly 15 million downloads per day, making it a prime target for attackers.

A recent study by JFrog found that 28,000+ models on Hugging Face are currently marked suspicious, and 100+ confirmed malicious models were found in a single research sweep. The majority of these malicious models target PyTorch's pickle serialization format, which can execute arbitrary OS commands when loaded.

Hugging Face flags unsafe models but does not block downloads, leaving users vulnerable to potential attacks. Additionally, the platform's trust signals, such as the green "safe" badge and download count, do not correspond to any meaningful security guarantee.

Why It Matters to the Industry

The collaboration between Protect AI and Hugging Face highlights the importance of prioritizing ML model security in the adult industry. With the increasing adoption of AI and ML technologies, the risk of malicious models being used in production environments grows exponentially.

The use of open-source models from public repositories like Hugging Face can be particularly problematic, as these models often lack formal admission processes or security guarantees. The lack of scanning and vetting procedures leaves organizations vulnerable to potential attacks, which can have severe consequences for both the business and its users.

What Comes Next

The partnership between Protect AI and Hugging Face is a significant step towards addressing the growing concern of ML model security. As the industry continues to adopt AI and ML technologies, it is essential that organizations prioritize model security and implement robust scanning and vetting procedures.

Protect AI's Guardian technology provides a critical layer of protection against malicious models, and its integration with Hugging Face has resulted in significant improvements in detection capabilities. The collaboration serves as a model for other organizations to follow, emphasizing the importance of prioritizing ML model security in the adult industry.

Key Facts

  • Protect AI and Hugging Face have partnered to enhance ML model security through Guardian's scanning technology.
  • Since October 2024, Protect AI has expanded Guardian's detection capabilities, improving existing threat detection and launching four new modules.
  • Over 4.47 million unique model versions on the Hugging Face Hub have been scanned, with 352,000 unsafe or suspicious issues identified across 51,700 models.
  • Hugging Face flags unsafe models but does not block downloads, leaving users vulnerable to potential attacks.
  • The majority of malicious models target PyTorch's pickle serialization format, which can execute arbitrary OS commands when loaded.