Key points
- Improved global AI response speeds by 10-20%, including 20%+ faster AI interactions across Southeast Asia
- Replaced a costly self-built AI dispatch platform, dedicated international connectivity, and overseas infrastructure with Zenlayer AI Gateway
- Established a unified AI gateway that simplifies future adoption of multiple AI models through a single platform
About the customer
Industry: Artificial intelligence of things (AIoT)
Needs: AI inference acceleration, AI traffic routing optimization, multi-modal AI rollout support
Zenlayer services used: Zenlayer AI Gateway, Global Private Backbone
The customer is a leading global AIoT service provider that provides cloud-based development services for brands, OEMs, and developers across North America, Europe, Southeast Asia, and other major markets. Their platform supports hundreds of millions of connected smart devices that increasingly rely on AI for voice interaction, automation, and real-time decision making.
As AI workloads expanded across the devices and markets they served, the customer needed to maintain consistent AI response times for users around the world.
The challenge: managing inconsistent global AI performance + growing infrastructure complexity
The customer initially accessed OpenAI’s GPT models directly, but response times varied significantly across regions because users and inference endpoints were geographically dispersed, surfacing latency issues across their global AIoT platform.
To improve performance, they invested heavily in their own AI dispatch infrastructure, developing a custom traffic routing platform, purchasing International Ethernet Private Lines (IEPL), and leasing overseas data center resources to shorten network paths between users and AI models.
Although this lifted performance along some routing paths, maintaining the platform became increasingly costly and operationally complex as AI traffic continued to grow. They also wanted an architecture that could support additional AI models without building and maintaining separate integrations for every provider.
The AIoT service provider needed a simpler, more scalable approach to accelerate AI traffic globally while reducing infrastructure complexity and preparing for a multi-model AI rollout worldwide.
Accelerating global AI inference
Leveraging Zenlayer AI Gateway, the customer unified global AI traffic management with intelligent traffic routing, private backbone acceleration, and enterprise-grade security through a single managed service.
Instead of relying only on public networks, the gateway directs inference traffic across our global private backbone. Intelligent routing automatically evaluates user location and network conditions to send requests along the most efficient path to the best inference endpoint.
Through Zenlayer AI Gateway, the AIoT service provider achieved:
- 10% faster average response times for GPT-4o
- 15% faster average response times for GPT-3.5 Turbo and other lightweight models
- More than 20% faster AI interactions across Southeast Asia, where our leading regional infrastructure provided particularly strong performance gains
These improvements helped the platform deliver faster AI interactions across their global base of connected smart devices.
Simplifying global AI infrastructure
With Zenlayer AI Gateway managing their global AI traffic, the customer retired much of the dedicated infrastructure supporting their custom dispatch platform, including costly international circuits and overseas data center resources.
This helped reduce the AIoT service provider’s operational complexity while giving them a more flexible way to scale AI traffic management as demand changes. Our economies of scale also helped reduce token pricing, further improving their platform’s overall cost efficiency.
Supporting secure, multi-model AI deployment
Zenlayer AI Gateway operates in passthrough mode, forwarding and accelerating AI requests without retaining customer prompts or model responses. Our gateway’s passthrough architecture combined with ISO 27001 certification supports the company’s global security and compliance requirements across North America, Europe, Southeast Asia, and other international markets.
Instead of building separate integrations for each AI provider, the AIoT service provider now has a streamlined path to introduce additional models such as Gemini, Claude, and future foundation models through a unified gateway.
Looking ahead
With Zenlayer AI Gateway streamlining their global AI traffic routing, the AIoT service provider now has a more flexible way to expand their multi-model AI rollout and introduce additional models without building separate integrations for each provider. We’ll continue helping them scale their platform across global markets with simpler AI infrastructure and routing as it grows.
Simplify global AI deployment with Zenlayer AI Gateway
We leveraged Zenlayer AI Gateway, intelligent routing, and our global private backbone to help a global AIoT platform accelerate AI responses while simplifying their infrastructure and preparing for future multi-model expansion.
Whether you’re optimizing AI performance worldwide or building a multi-model AI strategy, we can help you reduce infrastructure complexity while delivering a faster, more consistent user experience. To get started, talk to a Zenlayer solution expert today.
For the fastest service, check out zenConsole, our self-service platform that lets you deploy around the world in minutes.