Qwen

Qwen (also rendered as Tongyi Qianwen, 通义千问) is a family of large language models developed by Alibaba Cloud. The series spans small open-weight dense models suitable for edge deployment through to proprietary trillion-parameter mixture-of-experts systems served via the Alibaba Cloud Model Studio platform. Qwen is one of the most actively maintained open-weight model families and a primary alternative to Llama, Mistral, and DeepSeek in the open ecosystem.

The first generation, Qwen-7B, was released in August 2023 under a permissive licence and was followed by progressively larger and more capable releases through 2024. The Qwen2 series introduced grouped-query attention, longer context windows, and dedicated math and code variants. The Qwen3 series, released in April 2025, includes dense models at 0.6B, 1.7B, 4B, 8B, 14B, and 32B parameters and mixture-of-experts variants at 30B (3B active) and 235B (22B active). All Qwen3 models support a unified "thinking" and "non-thinking" inference mode that allows the same checkpoint to either reason step by step or respond directly.

Architecture and capabilities

Qwen models follow a decoder-only transformer design with refinements borrowed from contemporary research: rotary position embeddings, SwiGLU activation, RMSNorm, and grouped-query attention. Larger variants adopt sparse mixture-of-experts layers. Tokenisation uses a byte-pair encoding vocabulary tuned for Chinese, English, and major Asian languages.

The Qwen3 generation was trained on approximately 36 trillion tokens drawn from web text, code, mathematics, and synthetic reasoning traces. Alibaba reports improvements across mathematics, coding, logical reasoning, instruction following, and tool use, with the model series leading several open benchmarks at the time of release. Native support for over 100 languages includes Bahasa Malaysia, Bahasa Indonesia, Thai, Vietnamese, and Tagalog, which has driven adoption in Southeast Asia.

Qwen3-Max and long context

Qwen3-Max is the proprietary flagship of the family, with more than one trillion parameters and a context window extended to 256K tokens in standard configuration. A long-context variant, Qwen3-Instruct-2507, supports inputs of up to one million tokens, placing it among the longest production-deployed context windows alongside Gemini 1.5 and Claude Sonnet 4. Qwen3-Max is served only via the Alibaba Cloud API and has ranked in the top three of the LMArena public evaluation leaderboard.

| Model | Total params | Active params | Context | Licence | |---|---|---|---|---| | Qwen3-0.6B | 0.6B | 0.6B | 32K | Apache 2.0 | | Qwen3-8B | 8B | 8B | 128K | Apache 2.0 | | Qwen3-32B | 32B | 32B | 128K | Apache 2.0 | | Qwen3-30B-A3B | 30B | 3B | 128K | Apache 2.0 | | Qwen3-235B-A22B | 235B | 22B | 128K | Apache 2.0 | | Qwen3-Max | >1T | undisclosed | 256K-1M | Proprietary |

Specialised variants

The Qwen family includes specialised checkpoints for coding (Qwen-Coder), mathematics (Qwen-Math), and vision-language understanding (Qwen-VL). The Qwen2.5-VL release covers image and video reasoning, document parsing, and grounded perception, and Qwen2.5-Omni adds speech understanding and generation. These variants are commonly used as base models for fine-tuning by third parties.

References

1. Qwen Team, Alibaba Cloud. (2024). Qwen2 Technical Report. arXiv:2407.10671.
2. Qwen Team. (2025). Qwen3 Technical Report. arXiv:2505.09388.
3. Alibaba Cloud. (2025). Alibaba Introduces Qwen3, Setting New Benchmark in Open-Source AI with Hybrid Reasoning. Alibaba Cloud Community.
4. Malaysia Digital Economy Corporation. (2024). National AI Roadmap and Foundation Model Landscape. MDEC publications.