SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning)
📖 Table of Contents
☎ Groups
You can contact us and communicate with us by adding our group:
📝 Introduction
🍲 ms-swift is a large model and multimodal large model fine-tuning and deployment framework provided by the ModelScope community. It now supports training (pre-training, fine-tuning, human alignment), inference, evaluation, quantization, and deployment for 600+ text-only large models and 400+ multimodal large models. Large models include: Qwen3, Qwen3.5, InternLM3, GLM4.5, Mistral, DeepSeek-R1, Llama4, etc. Multimodal large models include: Qwen3-VL, Qwen3-Omni, Kimi-K3, Llava, InternVL3.5, MiniCPM-V-4, Ovis2.5, GLM4.5-V, DeepSeek-VL2, etc.
🍔 In addition, ms-swift integrates the latest training technologies, including Megatron parallelism techniques such as TP, PP, CP, EP to accelerate training, as well as numerous GRPO algorithm family reinforcement learning algorithms including: GRPO, DAPO, GSPO, SAPO, CISPO, RLOO, Reinforce++, etc. to enhance model intelligence. ms-swift supports a wide range of training tasks, including preference learning algorithms such as DPO, KTO, RM, CPO, SimPO, ORPO, as well as Embedding, Reranker, and sequence classification tasks. ms-swift provides full-pipeline support for large model training, including acceleration for inference, evaluation, and deployment modules using vLLM, SGLang, and LMDeploy, as well as model quantization using GPTQ, AWQ, BNB, and FP8 technologies.
Why Choose ms-swift?
- 🍎 Model Types: Supports 600+ text-only large models, 400+ multimodal large models, and All-to-All full modality models from training to deployment full pipeline, with Day-0 support for popular models.
- Dataset Types: Built-in 150+ datasets for pre-training, fine-tuning, human alignment, multimodal and various other tasks, with support for custom datasets. Users only need to prepare datasets for one-click training.
- Hardware Support: Supports NVIDIA GPUs (A100/H100/RTX series, etc., via CUDA), AMD GPUs (MI300 series, etc., via ROCm), domestic hardware such as Ascend NPU, MetaX GPU and Moore Threads GPU (via MUSA), as well as Apple MPS and CPU.
- Lightweight Training: Supports lightweight fine-tuning methods such as LoRA, QLoRA, DoRA, LoRA+, LLaMAPro, LongLoRA, LoRA-GA, ReFT, RS-LoRA, Adapter, LISA, etc.
- Quantized Training: Supports training on BNB, AWQ, GPTQ, AQLM, HQQ, EETQ quantized models, requiring only 9GB training resources for 7B models.
- Memory Optimization: GaLore, Q-Galore, UnSloth, Liger-Kernel, Flash-Attention 2/3, and Ulysses and Ring-Attention sequence parallelism techniques support, reducing memory consumption for long-text training.
- Distributed Training: Supports distributed data parallelism (DDP), device_map simple model parallelism, DeepSpeed ZeRO2 ZeRO3, FSDP/FSDP2, and Megatron distributed training technologies.
- 🍓 Multimodal Training: Supports multimodal packing technology to improve training speed by 100%+, supports mixed modality data training with text, images, video and audio, and supports independent control of vit/aligner/llm.
- Agent Training: Supports Agent templates, allowing one dataset to be used for training different models.
- 🍊 Training Tasks: Supports pre-training and instruction fine-tuning, as well as training tasks such as DPO, GKD, KTO, RM, CPO, SimPO, ORPO, and supports Embedding/Reranker and sequence classification tasks.
- 🥥 Megatron Parallelism: Provides TP/PP/SP/CP/ETP/EP/VPP parallel strategies to significantly boost MoE model training speed. Supports full-parameter and LoRA training methods for 300+ pure text large models and 100+ multimodal large models. Supports CPT/SFT/GRPO/DPO/KTO/RM training tasks.
- 🍉 Reinforcement Learning: Built-in rich GRPO family algorithms, including GRPO, DAPO, GSPO, SAPO, CISPO, CHORD, RLOO, Reinforce++, etc. Supports synchronous and asynchronous vLLM engine inference acceleration, with extensible reward functions, multi-turn inference Schedulers, and environments through plugins.
🎉 News
- 🔥 2026.10.29: Day-0 Support for NVIDIA-Nemotron-3.5-Transcribe, Refer to the SFT examples.
- 🔥 2026.10.22: Support EntroPack flexible bitrate model quantization, enabling fine-grained control of the model's bitrate based on the VRAM budget, with support for Transformers inference and LoRA training. See the best practices and the QLoRA example.
- 🔥 2026.10.19: Support for EmbeddingGemma-2 (740M, multimodal embedding model, unifying text/image/video/audio in a single 768d vector space). Inference and training verified with infonce loss fine-tuning.
- 🔥 2026.10.15: Day-0 Support for NVIDIA-Nemotron-3.5-Super-VL-120B-A12B, Refer to the SFT examples.
- 🔥 2026.10.13: Support for training and inference of decision models (typed-decision System-1 scoring models) via
--task_type decision, which performs a single forward pass and outputs calibrated probabilities over each question's option set. Supports JEV-27B-VL, Clef, OmniJev and Kev (0.8b/4b/9b), with proper scoring rule losses (distill_rps, smooth_brier, ce_rps). Refer to the and .
🛠️ Installation
To install using pip:
pip install ms-swift -U
# Using uv
pip install uv
uv pip install ms-swift -U --torch-backend=auto
To install from source:
# pip install git+https://github.com/modelscope/ms-swift.git
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
# The main branch is for swift 4.x. To install swift 3.x, please run the following command:
# git checkout release/3.12
pip install -e .
# Using uv
uv pip install -e . --torch-backend=auto
Running Environment:
| Range | Recommended | Notes |
|---|
| python | >=3.10 | 3.12 | |
| cuda | | cuda12.8/13.0 | No need to install if using CPU, NPU, MPS |
| torch | >=2.0 | 2.8.0/2.11.0 | |
| transformers | >=4.33 | 4.57.6/5.12.1 | |
| modelscope | >=1.23 | | |
| datasets | >=3.0,<4.8.5 | 3.6.0/4.8.4 | |
| peft | >=0.11,<0.21 | | |
| flash_attn | | 2.8.3/4.0.0b15 | |
| trl | >=0.15,<1.0 | 0.29.1 | RLHF |
| deepspeed | >=0.19.3,<0.20 | 0.19.7 | Training |
| vllm | >=0.5.1 | 0.11.0/0.23.0 | Inference/Deployment |
| sglang | >=0.4.6 | | Inference/Deployment |
| evalscope | >=1.0 | | Evaluation |
| gradio | | 5.32.1 | Web-UI/App |
For more optional dependencies, you can refer to here.
🚀 Quick Start
10 minutes of self-cognition fine-tuning of Qwen3-4B-Instruct-2507 on a single 3090 GPU:
Command Line Interface (Recommended)
# 13GB
CUDA_VISIBLE_DEVICES=0 \
swift sft \
--model Qwen/Qwen3-4B-Instruct-2507 \
--tuner_type lora \
--dataset 'AI-ModelScope/alpaca-gpt4-data-zh#500' \
'AI-ModelScope/alpaca-gpt4-data-en#500' \
'swift/self-cognition#500' \
--torch_dtype bfloat16 \
--num_train_epochs 1 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--learning_rate 1e-4 \
--lora_rank 8 \
--lora_alpha 32 \
--target_modules all-linear \
--gradient_accumulation_steps 16 \
--eval_steps 50 \
--save_steps 50 \
--save_total_limit 2 \
--logging_steps 5 \
--max_length 2048 \
--output_dir output \
--warmup_ratio 0.05 \
--dataloader_num_workers 4 \
--model_author swift \
--model_name swift-robot
Tips:
- If you want to train with a custom dataset, you can refer to this guide to organize your dataset format and specify
--dataset <dataset_path>.
- The
--model_author and --model_name parameters are only effective when the dataset includes swift/self-cognition.
- To train with a different model, simply modify
--model <model_id/model_path>.
- By default, ModelScope is used for downloading models and datasets. If you want to use HuggingFace, simply specify
--use_hf true.
After training is complete, use the following command to infer with the trained weights:
- Here,
--adapters should be replaced with the last checkpoint folder generated during training. Since the adapters folder contains the training parameter file args.json, there is no need to specify --model, --system separately; Swift will automatically read these parameters. To disable this behavior, you can set --load_args false.
# Using an interactive command line for inference.
CUDA_VISIBLE_DEVICES=0 \
swift infer \
--adapters output/vx-xxx/checkpoint-xxx \
--stream true \
--temperature 0 \
--max_new_tokens 2048
# merge-lora and use vLLM for inference acceleration
CUDA_VISIBLE_DEVICES=0 \
swift infer \
--adapters output/vx-xxx/checkpoint-xxx \
--stream true \
--merge_lora true \
--infer_backend vllm \
--vllm_max_model_len 8192 \
--temperature 0 \
--max_new_tokens 2048
Finally, use the following command to push the model to ModelScope:
CUDA_VISIBLE_DEVICES=0 \
swift export \
--adapters output/vx-xxx/checkpoint-xxx \
--push_to_hub true \
--hub_model_id '<your-model-id>' \
--hub_token '<your-sdk-token>' \
--use_hf false
Web-UI
The Web-UI is a zero-threshold training and deployment interface solution based on Gradio interface technology. For more details, you can check here.
SWIFT_UI_LANG=en swift web-ui
Using Python
ms-swift also supports training and inference using Python. Below is pseudocode for training and inference. For more details, you can refer to here.
Training:
from peft import LoraConfig, get_peft_model
from swift import get_model_processor, get_template, load_dataset, EncodePreprocessor
from swift.trainers import Seq2SeqTrainer, Seq2SeqTrainingArguments
# Retrieve the model and template, and add a trainable LoRA module
model, tokenizer = get_model_processor(model_id_or_path, ...)
template = get_template(tokenizer, ...)
lora_config = LoraConfig(...)
model = get_peft_model(model, lora_config)
# Download and load the dataset, and encode the text into tokens
train_dataset, val_dataset = load_dataset(dataset_id_or_path, ...)
train_dataset = EncodePreprocessor(template=template)(train_dataset, num_proc=num_proc)
val_dataset = EncodePreprocessor(template=template)(val_dataset, num_proc=num_proc)
# Train the model
training_args = Seq2SeqTrainingArguments(...)
trainer = Seq2SeqTrainer(
model=model,
args=training_args,
template=template,
train_dataset=train_dataset,
eval_dataset=val_dataset,
)
trainer.train()
Inference:
from swift import TransformersEngine, InferRequest, RequestConfig
# Perform inference using the native Transformers engine
engine = TransformersEngine(model_id_or_path, adapters=[lora_checkpoint])
infer_request = InferRequest(messages=[{'role': 'user', 'content': 'who are you?'}])
request_config = RequestConfig(max_tokens=max_new_tokens, temperature=temperature)
resp_list = engine.infer([infer_request], request_config)
print(f'response: {resp_list[0].choices[0].message.content}')
✨ Usage
Here is a minimal example of training to deployment using ms-swift. For more details, you can check the examples.
- If you want to use other models or datasets (including multimodal models and datasets), you only need to modify
--model to specify the corresponding model's ID or path, and modify --dataset to specify the corresponding dataset's ID or path.
- By default, ModelScope is used for downloading models and datasets. If you want to use HuggingFace, simply specify
--use_hf true.
Training
Supported Training Methods:
Pre-training:
# 8*A100
NPROC_PER_NODE=8 \
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
swift pt \
--model Qwen/Qwen3-4B-Base \
--dataset swift/chinese-c4 \
--streaming true \
--tuner_type full \
--deepspeed zero2 \
--output_dir output \
--max_steps 10000 \
...
Fine-tuning:
CUDA_VISIBLE_DEVICES=0 swift sft \
--model Qwen/Qwen3-4B-Instruct-2507 \
--dataset AI-ModelScope/alpaca-gpt4-data-en \
--tuner_type lora \
--output_dir output \
...
RLHF:
CUDA_VISIBLE_DEVICES=0 swift rlhf \
--rlhf_type dpo \
--model Qwen/Qwen3-4B-Instruct-2507 \
--dataset hjh0119/shareAI-Llama3-DPO-zh-en-emoji \
--tuner_type lora \
--output_dir output \
...
Megatron-SWIFT
ms-swift supports using Megatron parallelism techniques to accelerate training, including large-scale cluster training and MoE model training. The following training methods are supported:
NPROC_PER_NODE=2 CUDA_VISIBLE_DEVICES=0,1 megatron sft \
--model Qwen/Qwen3-4B-Instruct-2507 \
--save_safetensors true \
--dataset AI-ModelScope/alpaca-gpt4-data-zh \
--tuner_type lora \
--output_dir output \
...
Reinforcement Learning
ms-swift supports a rich set of GRPO family algorithms:
CUDA_VISIBLE_DEVICES=0,1,2,3 NPROC_PER_NODE=4 \
swift rlhf \
--rlhf_type grpo \
--model Qwen/Qwen3-4B-Instruct-2507 \
--tuner_type lora \
--use_vllm true \
--vllm_mode colocate \
--dataset AI-MO/NuminaMath-TIR#10000 \
--output_dir output \
...
Inference
CUDA_VISIBLE_DEVICES=0 swift infer \
--model Qwen/Qwen3-4B-Instruct-2507 \
--stream true \
--infer_backend transformers \
--max_new_tokens 2048
Interface Inference
CUDA_VISIBLE_DEVICES=0 swift app \
--model Qwen/Qwen3-4B-Instruct-2507 \
--stream true \
--infer_backend transformers \
--max_new_tokens 2048
Deployment
CUDA_VISIBLE_DEVICES=0 swift deploy \
--model Qwen/Qwen3-4B-Instruct-2507 \
--infer_backend vllm
Sampling
CUDA_VISIBLE_DEVICES=0 swift sample \
--model Qwen/Qwen3-4B-Instruct-2507 \
--sampler_engine transformers \
--num_return_sequences 5 \
--dataset AI-ModelScope/alpaca-gpt4-data-zh#5
Evaluation
CUDA_VISIBLE_DEVICES=0 swift eval \
--model Qwen/Qwen3-4B-Instruct-2507 \
--infer_backend sglang \
--eval_backend OpenCompass \
--eval_dataset ARC_c
Quantization
CUDA_VISIBLE_DEVICES=0 swift export \
--model Qwen/Qwen3-4B-Instruct-2507 \
--quant_method fp8 \
--dataset AI-ModelScope/alpaca-gpt4-data-zh \
--output_dir Qwen3-4B-Instruct-2507-FP8
Push Model
swift export \
--model <model-path> \
--push_to_hub true \
--hub_model_id '<model-id>' \
--hub_token '<sdk-token>'
🏛 License
This framework is licensed under the Apache License (Version 2.0). For models and datasets, please refer to the original resource page and follow the corresponding License.
📎 Citation
@misc{zhao2024swiftascalablelightweightinfrastructure,
title={SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning},
author={Yuze Zhao and Jintao Huang and Jinghan Hu and Xingjun Wang and Yunlin Mao and Daoze Zhang and Zeyinzi Jiang and Zhikai Wu and Baole Ai and Ang Wang and Wenmeng Zhou and Yingda Chen},
year={2024},
eprint={2408.05517},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2408.05517},
}
Star History
