Jump to
With H2O LLM Studio, you can
- easily and effectively fine-tune LLMs without the need for any coding experience.
- use a graphical user interface (GUI) specially designed for large language models.
- fine-tune any LLM using a large variety of hyperparameters.
- use recent fine-tuning techniques such as Low-Rank Adaptation (LoRA) and 8-bit model training with a low memory footprint.
- use Reinforcement Learning (RL) to fine-tune your model (experimental).
- use advanced evaluation metrics to judge generated answers by the model.
- track and compare your model performance visually.
- chat with your model and get instant feedback on your model performance.
- easily export your model to the Hugging Face Hub and share it with the community.
Quickstart
For questions, discussing, or just hanging out, come and join our Discord!
Use cloud-based runpod.io instance to run the latest version of H2O LLM Studio with GUI.
Using CLI for fine-tuning LLMs:
What's New
- PR 788 New problem type for Causal Regression Modeling allows to train single target regression data using LLMs.
- PR 747 Fully removed RLHF in favor of DPO/IPO/KTO optimization.
- PR 741 Removing separate max length settings for prompt and answer in favor of a single
max_length settings better resembling chat_template functionality from transformers.
- PR 599 Added
KTOPairLoss for DPO modeling allowing to train models with simple preference data. Data currently needs to be manually prepared by randomly matching positive and negative examples as pairs.
- PR 592 Starting to deprecate RLHF in favor of DPO/IPO optimization. Training is disabled, but old experiments are still viewable. RLHF will be fully removed in a future release.
- PR 530 Introduced a new problem type for DPO/IPO optimization. This optimization technique can be used as an alternative to RLHF.
- PR 288 Introduced DeepSpeed for sharded training allowing to train larger models on machines with multiple GPUs. Requires NVLink. This feature replaces FSDP and offers more flexibility. DeepSpeed requires a system installation of CUDA Toolkit and we recommend using version 12.1. See Recommended Install.
- PR 449 New problem type for Causal Classification Modeling allows to train binary and multiclass models using LLMs.
- PR 364 User secrets are now handled more securely and flexible. Support for handling secrets using the 'keyring' library was added. User settings are tried to be migrated automatically.
Please note that due to current rapid development we cannot guarantee full backwards compatibility of new functionality. We thus recommend to pin the version of the framework to the one you used for your experiments. For resetting, please delete/backup your data and output folders.
Setup
H2O LLM Studio requires a machine with Ubuntu 16.04+ and at least one recent NVIDIA GPU with NVIDIA drivers version >= 580 (PyTorch runs on CUDA 13). For larger models, we recommend at least 24GB of GPU memory.
For more information about installation prerequisites, see the Set up H2O LLM Studio guide in the documentation.
For a performance comparison of different GPUs, see the H2O LLM Studio performance guide in the documentation.
Recommended Install
The recommended way to install H2O LLM Studio is using uv with Python 3.12. To install Python 3.12 on Ubuntu 20.04+, execute the following commands:
Installing NVIDIA Drivers (if required)
If deploying on a 'bare metal' machine running Ubuntu, one may need to install the required NVIDIA drivers and CUDA. The following commands show how to retrieve the latest drivers and CUDA 13.0 for a machine running Ubuntu 22.04 as an example. One can update the following based on their OS.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
