Why are lightweight models popular overseas?

  • The Together AI expert interviewed by Nomura observed that Chinese open-weight models lead the overseas open-weight model market they have observed, thanks to their combination of capability and cost.
  • DeepSeek 4.1 Flash is one of the most popular models on the Together AI platform, even surpassing some flagship open-weight models in popularity: it meets everyday task needs while its inference cost is significantly lower than that of large flagship models.
  • The expert believes Chinese model developers will advance frontier model capabilities while translating technological gains into more cost-efficient models for large-scale commercial use.

Why might agents increase inference demand?

  • The expert observed that overseas model use is shifting from traditional conversations to agent workflows. The latter often involve longer contexts and multiple rounds of inference, consuming significantly more tokens per task and potentially sustaining inference demand.
  • Model capabilities, inference speed and total cost of use remain developers’ main considerations when selecting a model. Large enterprises also consider data privacy, data center location and regulatory compliance.

How does Together AI serve overseas demand?

  • Together AI independently hosts open-weight models and serves developers and smaller customers through a service charged by token usage. Its local deployments overseas also allow enterprises with data residency and compliance requirements to use these models.
  • Dedicated inference reserves GPU compute capacity for large enterprises and model developers and customizes the deployment environment. The expert says this business accounts for a significantly larger share of Together AI’s inference business.
  • Some Chinese model developers partner with Together AI, prioritizing their internal compute capacity for training and development. Together AI provides infrastructure and model services, while developers retain customer relationships and API revenue. Nomura views this as a relatively asset-light way to expand inference capacity.
  • Together AI also delivers GPU clusters and data center infrastructure. The expert believes this may be its largest source of revenue, but its gross margin is lower than that of inference services.

What determines an advantage in inference price competition?

  • As API price competition intensifies, inference economics depend not only on GPU costs but also on how many tokens can be processed and how much revenue can be generated per GPU hour. Higher utilization and throughput help maintain cost competitiveness and margins.
  • The expert says DeepSeek has strong inference engineering capabilities. Its publicly released DeepInfer framework can improve resource utilization and throughput, but system optimization and infrastructure integration in production environments are complex, making its actual efficiency difficult for other providers to replicate. Together AI is attempting to adapt similar technology to its own GPUs and production environment.

What different constraints apply to cross-border deployment?

  • The expert believes the immediate commercial risks of independent third-party hosting of publicly available model weights are relatively low, but applicable licensing and regulatory requirements must still be observed.
  • Dedicated inference involves a more direct commercial relationship with Chinese model developers and may face greater regulatory scrutiny. It is also more susceptible to changes in technology export controls or cross-border compute restrictions. As of the interview, the expert said existing partnerships had not been materially disrupted by recent regulatory developments.