OpenAI and Broadcom Launch First Custom AI Inference Chip "Jalapeo": Costs Halved, Compute Self-Reliance Accelerates
Introduction
As large-model applications flourish, high inference costs have become a core bottleneck limiting AI at scale. In April 2025, OpenAI and semiconductor giant Broadcom jointly announced their first custom AI inference chip, "Jalapeo." Designed specifically for large-model inference, the chip aims to cut inference costs by about 50% while significantly improving compute efficiency, marking a key step in the AI industry from reliance on general-purpose GPUs toward self-reliance through dedicated chips. This move is expected not only to reshape the economics of AI infrastructure, but also to accelerate a profound shift in the global compute landscape.
Main Text
1. Dedicated Chips: The Key to Solving the Large-Model Inference Cost Problem
Today, large-model inference mainly relies on general-purpose GPUs from vendors such as Nvidia. While GPUs perform well in training, they involve a lot of redundant computation in inference, leading to poor energy efficiency. Industry estimates show that in large-model inference, only about 20% to 30% of GPU compute is effectively used, while the rest sits idle or is wasted. This inefficiency directly drives up operating costs, especially for cloud providers and app developers that call large models frequently. In many cases, inference expenses account for more than 60% of total costs.
The OpenAI-Broadcom collaboration targets exactly this pain point. Jalapeo uses a highly customized architecture, with hardware-level optimization for core operations in Transformer models such as matrix multiplication and attention mechanisms. By trimming unnecessary compute units and adding dedicated data-flow channels and on-chip memory bandwidth, Jalapeo achieves 2x to 3x higher inference throughput at the same power level, reducing per-query inference costs by about 50%. In practical terms, a task that once cost $100 now costs just $50, greatly lowering the barrier to AI adoption.
2. Jalapeo's Technical Highlights and Design Philosophy
Jalapeo is more than a hardware innovation; it is a model of co-design between software and hardware. Its core technical features include:
- Sparse computing engine: Because activation sparsity is common in large models, Jalapeo integrates a dedicated sparse accelerator that can skip zero-value operations and improve compute efficiency several-fold.
- High-bandwidth on-chip memory: Using the near-memory computing concept, it tightly couples high-bandwidth SRAM with compute units, reducing data-movement latency and making it especially suitable for long-sequence context processing in large models.
- Open instruction set and ecosystem compatibility: To reduce migration costs for developers, Jalapeo supports low-level operator interfaces for mainstream AI frameworks such as PyTorch and TensorFlow, and provides an optimized inference runtime library.
In design philosophy, OpenAI emphasizes "vertical integration from algorithms to hardware." As one of the world's most advanced large-model developers, OpenAI brings deep understanding of model-inference behavior into chip design, while Broadcom leverages its strong expertise in networking chips and ASICs to turn those requirements into a high-yield mass-production solution. This "application-defined chip" model could become the mainstream path for future AI chip development.
3. Compute Self-Reliance: Strategic Importance and Industry Impact
The launch of Jalapeo is not only a technical breakthrough but also a strategic move. Today, global AI compute depends heavily on a small number of suppliers, and supply risk plus pricing-power imbalance have long troubled the industry. As a leader in large models, OpenAI's launch of its own inference chip signals a move away from dependence on a single compute stack and toward compute self-reliance.
From an industry perspective, Jalapeo will trigger several chain reactions:
- Lowering the AI adoption barrier: Halving inference costs means small and medium-sized businesses, and even individual developers, can deploy large-model applications with much smaller budgets, pushing AI from a luxury to a daily tool.
- Accelerating compute decentralization: The high efficiency of dedicated chips makes edge inference possible. In the future, AI inference tasks may move from cloud data centers down to enterprise local servers or even end devices, forming a distributed compute network.
- Creating a new chip ecosystem: Jalapeo's success will encourage more AI companies to follow suit. Future AI chip competition is likely to move from a single-center monopoly to a multi-polar landscape.
Of course, challenges remain. Jalapeo is currently mainly aimed at inference, while training still depends on general-purpose GPUs. Its ecosystem also needs time to mature, and the level of developer-community support will be a key variable in determining whether it can scale.
Conclusion
The Jalapeo chip jointly released by OpenAI and Broadcom sets halving large-model inference cost as its core goal. Through a dedicated architecture and software-hardware co-design, it not only offers a new paradigm for the commercial viability of AI applications, but also marks a substantial acceleration in the strategy of compute self-reliance. In the wave of deeper convergence between AI and semiconductors, Jalapeo may become a milestone: it proves a vertical innovation path from algorithm to chip and points to an AI compute era that is more efficient, open, and diverse. Looking ahead, as more ecosystem partners join and iterative versions of the chip emerge, we have reason to believe that AI will no longer be the privilege of a few giants, but a productivity engine for all society.
