Model Distillation: Why Chinese AI Is So Good and Cheap

DeepSeek’s R1 model had an unusual debut. The Chinese lab released it on January 20, 2025. Its API cost was $2.19 per million output tokens, while OpenAI’s o1 cost $60 for the same volume.

Within a week, Nvidia lost nearly $600 billion in market value, and that too in a single trading day. Eighteen months later, Qwen, GLM, and Kimi joined DeepSeek at the top of the open-weight leaderboards.

So, how does a company deliver near-frontier AI at a fraction of frontier prices, and what happens to your data when you use it? Part of the first answer is model distillation, a technique that transfers capability from a large teacher model to a smaller, cheaper student model. The second answer depends almost entirely on how you access the model.

This article explains how distillation built the Chinese AI wave, where your data goes when you use it, and how businesses can capture the savings without the risk.

Model distillation is a training technique in which a smaller student model learns from the outputs or behavior of a larger teacher model. The student studies what the teacher produces rather than repeating the teacher’s entire training process. The result is a smaller model that performs well on targeted tasks at a fraction of the cost.

A classroom analogy makes the idea concrete. An experienced teacher solves 100,000 hard problems and writes out worked explanations for each one. A student then studies the solved set and picks up the skill without repeating the teacher’s decades of learning.

Researchers call the broader technique knowledge distillation, which the AI researcher Geoffrey Hinton and his co-authors formalized in a 2015 paper. The classic version transferred probability outputs from a large neural network to a small one. Modern LLM distillation usually trains the student on plain text the teacher wrote, which looks like supervised fine-tuning on synthetic data. The teacher-student framework stays the same either way.

Model distillation works as a pipeline with seven steps, and most production versions follow the same shape.

1. Pick a teacher: The lab selects a model capable enough to produce examples worth learning from.
2. Build a task dataset: Prompts come from real workloads or from synthetic generation.
3. Generate teacher outputs: The teacher answers every prompt in the dataset.
4. Filter the examples: The lab removes weak or wrong outputs before training.
5. Train the student: The smaller model learns to reproduce the teacher’s answers on the curated set.
6. Evaluate: The lab compares the student against the teacher and against independent benchmarks.
7. Iterate: Areas where the student underperforms get more teacher data in the next round.

Distillation makes AI cheaper because a frontier model is twice as expensive. Training a model costs enormous sums up front, while running one costs money for every request. A distilled student attacks the second cost, which is the one that grows with usage. You pay the teacher once to create the data, and the cheaper student then handles every request.

Distillation alone doesn’t explain Chinese AI pricing, though. Several efficiency techniques stack together, and the leading Chinese models use most of them at once.

Mixture-of-experts is the easiest factor to put numbers on. DeepSeek’s V4 Pro carries 1.6 trillion total parameters but activates only 49 billion for each token. The model holds frontier-scale knowledge while paying small-model compute costs on every request.

The efficiency shows up on the rate card. As of August 2026, DeepSeek lists V4 Pro output below $2 per million tokens, while OpenAI lists its flagship GPT-5.6 Sol at $30 for the same volume. The same price gap shows up against every major cloud's own rate card.

DeepSeek is the clearest documented case of model distillation inside the Chinese AI industry. The lab released R1 on January 20, 2025, as an open-weight reasoning model under an MIT license. R1 itself was not distilled from anything. Its training process, later published in Nature, centered on reinforcement learning, where the model earned rewards for correct answers and learned to reason.

Alongside R1, DeepSeek released six smaller distilled models. The lab generated roughly 800,000 training samples with R1. These samples were used to fine-tune open models from the Qwen and Llama families, as its documentation describes. The distilled family ranges from 1.5 billion to 70 billion parameters.

The results held up. DeepSeek reported that the 14-billion-parameter version beat QwQ-32B-Preview, a reasoning model more than twice its size. In DeepSeek’s experiments, distilling from R1 also outperformed applying reinforcement learning to the same small base model. Letting the big model learn first was the cheaper path.

The costs are striking too. DeepSeek reported that training its V3 base model took about $5.6 million in GPU time. The figure covers only the final training run, not earlier research or infrastructure. Figures published alongside the Nature paper put the R1 reinforcement-learning stage at about $294,000 on top. American labs don’t publish comparable numbers, though outside estimates for frontier training runs sit in the hundreds of millions of dollars.

Chinese AI labs use distillation openly, but claims of unauthorized distillation of American models sit in a different category. News coverage often merges the two, and the merge is misleading.

Start with what’s established. DeepSeek publicly documented distilling its own R1 model into smaller Qwen-based and Llama-based models. OpenAI sells distillation tooling of its own. Using distillation proves no wrongdoing by itself.

The allegations are separate. Reuters reported that OpenAI accused DeepSeek of ongoing efforts to “free-ride on the capabilities” of American frontier labs. The accusation came in a February 12, 2026 memo to the US House Select Committee on China. The memo claimed that accounts tied to DeepSeek employees masked their access through third-party routers. It also claimed employees wrote code to pull outputs at scale.

DeepSeek and its parent company High-Flyer did not respond to Reuters’ requests for comment. None of these claims has been tested in court or independently verified so far, and they deserve the label they carry in the reporting: allegations.

The distinction that matters is permission rather than technique. The real question is where the teacher’s outputs came from and whether the model’s owner authorized their use for training. Distilling your own model is standard engineering. Distilling a rival’s proprietary model against its terms of service is a contractual and legal problem.

OpenAI sells model distillation as a product. The company launched Model Distillation in its API in October 2024, months before R1 existed. The tooling stores outputs from larger models such as GPT-4o, and developers fine-tune smaller, cost-efficient models on them. Any capable model can serve as a teacher this way, which is exactly why unauthorized access to a rival’s outputs would be valuable. 

The efficiency race now has at least four serious Chinese entrants. Alibaba’s Qwen family is the most downloaded of them. Its small and mid-size models carry permissive Apache 2.0 licenses, which is why so many fine-tuned open models worldwide are Qwen underneath. Alibaba shipped new flagship releases through 2026, though its very largest models often stay API-only.

Zhipu AI’s GLM line competes head-on with Western coding models. GLM-5.2 ships under an MIT license and, as of mid-2026, sits at the top of independent open-weight intelligence rankings. Moonshot AI’s Kimi pushes scale instead. The lab released Kimi K3 in July 2026 and billed it as the largest open-source model to date, at roughly 2.8 trillion parameters. The weights arrived under a custom license rather than a standard open one.

All four labs build on the same base design: a large mixture-of-experts model with openly released weights. Open weights also change the privacy question, because a model you can download behaves nothing like a hosted app.

The privacy question starts with a simple rule: your data goes wherever the model runs. Type a prompt into a hosted chatbot or API, and the prompt travels to the provider’s servers, along with any files you attach. What happens next depends on the provider’s policy and on the laws of the country where the servers sit.

For DeepSeek’s own app and API, both answers point to China. DeepSeek’s privacy policy states that user inputs, from prompts and chat histories to uploaded files, are stored on servers in China. The policy also covers device information and keystroke patterns. The Chinese law adds the layer that worries regulators the most. 

The country’s National Intelligence Law requires organizations to cooperate with state intelligence work when asked. DeepSeek is the documented example here, but the same jurisdiction question applies to any China-hosted AI service. This includes the hosted APIs for Qwen, GLM, and Kimi.

The concerns aren’t only theoretical. In January 2025, cloud security firm Wiz reported an exposed DeepSeek database that leaked over a million log entries, including chat histories. A month later, South Korea’s privacy regulator found the app had transferred user data without consent to third parties, including ByteDance. DeepSeek pulled its app from Korean stores and restored it in April 2025 after making changes.

The concerns above all sit in DeepSeek’s hosted services. The open-weight models are static files, and a downloaded model sends nothing back to its developer. So, whether your data can reach China depends on your hosting choice, not on the model’s country of origin.

Businesses reach these models through three main routes, and each route has a different data path.

The middle route surprises many teams. Microsoft added DeepSeek R1 to Azure AI Foundry in January 2025, and Amazon offers it through Bedrock. As a result, a US company can run a Chinese model without a single byte reaching China. The weights are static files, and the hosting provider controls the data path.

Self-hosting goes further. Download GLM, Qwen, or a DeepSeek model and run it on your own GPUs, so prompts never leave your network. The same breakeven math applies here as in self-hosting versus per-token pricing. The trade-off is operational, since you carry the hardware cost and the security of the deployment yourself. Self-hosting removes the data-residency risk. However, it does not remove the risks that live inside the model itself.

Moving the hosting fixes the data path and leaves a second category untouched, the model’s own behavior. The Center for AI Standards and Innovation (CAISI) at NIST evaluated DeepSeek’s models in 2025 and found them far easier to misuse than US models.

With public jailbreak prompts, the DeepSeek models produced harmful outputs in 95% to 100% of tests, while US reference models did so in 5% to 12%. In agent-hijacking tests, DeepSeek R1 attempted to leak two-factor codes in 37% of runs against 4% for US models. A follow-up CAISI evaluation in April 2026 put DeepSeek V4 Pro about eight months behind the US frontier on capability.

Jailbreak resistance travels with the weights, so these findings apply wherever the model runs. A self-hosted Chinese model with weak built-in refusals still needs guardrails added around it. CAISI has published this depth of testing only for DeepSeek so far, and the thinner public evidence on Qwen, GLM, and Kimi argues for the same caution.

Governments responded first, and mostly by restricting access. Italy’s data protection authority ordered DeepSeek blocked in early 2025 after finding its privacy disclosures inadequate. South Korea suspended new downloads in February 2025 and allowed them again in April, once DeepSeek changed its practices to fit local law. Australia banned the app from all federal government devices. Taiwan barred it across the public sector, and US agencies including NASA and the Navy blocked it outright.

Restriction is not the only government tool. CAISI now evaluates Chinese frontier models under America’s AI Action Plan and publishes the results, which gives buyers an independent check to run before committing to a model.

Companies moved on two tracks at once. AI providers adjusted where and how they host. Microsoft and Amazon put DeepSeek’s open weights inside US cloud regions, and DeepSeek itself made compliance changes to re-enter South Korea. OpenAI, for its part, says it now detects and blocks accounts that try to extract outputs for distillation.

Enterprise buyers built their own controls. Some ban consumer AI apps on work devices, as Microsoft did with the DeepSeek app. Others route all model traffic through gateways that log prompts and strip sensitive data.

The contrast inside Microsoft captures the whole logic. The company bans the China-hosted app while serving the same model from its own cloud. The difference is the data path, not the weights.

Using these models safely comes down to six moves, and a business can make most of them this week.

• Keep sensitive data out of consumer AI apps: Don’t paste customer records, contracts, or source code into the DeepSeek app or any China-hosted chatbot. Anything typed there lands on servers your business doesn’t control.
• Switch the endpoint, keep the model: Point code built on DeepSeek at Azure AI Foundry or Amazon Bedrock instead of DeepSeek’s own API. The model behaves the same, and the data stays in your cloud region.
• Block what you’ve ruled out: Use mobile device management to block banned AI apps on work phones and laptops, the way Microsoft did internally. A technical block beats a memo.
• Self-host for regulated data: Run an open-weight model such as Qwen or a distilled DeepSeek on your own GPUs with a serving tool like vLLM. Prompts then never leave your network.
• Add human review around weak models: DeepSeek’s models failed most of CAISI’s jailbreak tests, so keep a person approving any AI action that moves money or data.
• Get answers in writing: Ask every AI vendor where your data is stored and whether it trains on your prompts. Put the answers into a data processing agreement with retention limits.

Chinese AI models can be good and cheap because they are optimized for cost per unit of capability. Model distillation is one of the strongest tools for doing exactly that. DeepSeek documented the approach in the open, and Qwen, GLM, and Kimi now run the same playbook at frontier scale.

The privacy risk is real but specific. It lives in China-hosted apps and APIs rather than in the weights themselves. Businesses that host open models in their own environment get the economics without the data exposure. The one condition is to add the guardrails the models lack.

No single trick creates the cost advantage, and no single ban resolves the risk. The next test comes from American price cuts, already reshaping the market. We covered the pricing battle Grok 4.5 started across the industry.

Everything above assumes you’ll make the hosting and model choices yourself, and the harder part is matching them to real workloads and compliance rules. We’re happy to walk through your setup on a free consultation call with our data and AI services team and show where a cheaper model would fit.