The integration of Intel's Gaudi hardware support into Text Generation Inference (TGI), a production-ready serving solution for Large Language Models (LLMs), has been announced by Hugging Face. This move brings the power of Intel's specialized AI accelerators to TGI's high-performance inference stack, enabling more deployment options for the open-source AI community.

The integration supports Intel's full line of Gaudi hardware, including Gaudi1, Gaudi2, and Gaudi3, which are available on various cloud platforms such as AWS EC2 DL1 instances, Intel Tiber AI Cloud, IBM Cloud, and from OEMs like Dell, HP, and Supermicro. This move is significant for the adult industry, where LLMs are increasingly being used for tasks such as content generation, moderation, and customer support.

What Happened

The integration of Gaudi hardware support into TGI was announced on March 28, 2025, by Hugging Face. The company has fully integrated Gaudi support into TGI's main codebase in PR #3091, which previously maintained a separate fork for Gaudi devices at tgi-gaudi. This move allows users to access the latest features and models without having to maintain a custom repository.

The integration supports various key benefits, including hardware diversity, cost efficiency, production-readiness, model support, and advanced features such as multi-card inference (sharding), vision-language models, and FP8 precision. The Gaudi backend for TGI provides several advantages over traditional GPUs, including more deployment options, compelling price-performance, and robustness.

Background and Context

The Intel Gaudi 2 AI accelerator has been recognized as the only benchmarked alternative to Nvidia H100 for generative AI performance. According to a recent article by Intel, the Gaudi 2 accelerator provides strong performance-per-dollar and is available via AWS, the Intel Developer Cloud, Supermicro, and WiWynn. The accelerator supports both deep learning training and inference for AI models like LLMs.

Designveloper's blog post on "12 vLLM Alternatives for Efficient and Scalable LLM Inference" highlights various alternatives to vLLM, including SGLang, TensorRT-LLM, Hugging Face Text Generation Inference (TGI), OpenLLM, DeepSpeed, Ollama, and others. These alternatives offer different features, use cases, and pricing models, providing users with a range of options for LLM inference.

Why It Matters to the Industry

The integration of Gaudi hardware support into TGI is significant for the adult industry because it provides more deployment options for LLMs. The Gaudi backend for TGI offers several advantages over traditional GPUs, including cost efficiency, production-readiness, and advanced features such as multi-card inference (sharding) and vision-language models.

The use of LLMs in the adult industry is increasing, with applications in content generation, moderation, and customer support. The integration of Gaudi hardware support into TGI enables developers to deploy LLMs more efficiently and effectively, which can lead to improved performance, reduced costs, and enhanced user experiences.

What Comes Next

Hugging Face has announced that it will continue to expand its model lineup with cutting-edge additions, including DeepSeek-r1/v3, QWen-VL, and more powerful models. The company invites the community to try out TGI on Gaudi hardware and provide feedback, which can be done by checking out the contribution guidelines or opening an issue on GitHub.

Key Facts

  • The integration of Intel's Gaudi hardware support into TGI has been announced by Hugging Face.
  • The integration supports Intel's full line of Gaudi hardware, including Gaudi1, Gaudi2, and Gaudi3.
  • The Gaudi backend for TGI provides several advantages over traditional GPUs, including cost efficiency, production-readiness, and advanced features.
  • The use of LLMs in the adult industry is increasing, with applications in content generation, moderation, and customer support.
  • Hugging Face will continue to expand its model lineup with cutting-edge additions, including DeepSeek-r1/v3, QWen-VL, and more powerful models.