LLM Finetune Configuration Generator - LoRA Adapter Config
Automatically calculate rank, alpha, target modules, learning rate and batch size for fine-tuning LoRA/QLoRA. Output ready-to-use JSON HuggingFace PEFT. Based on the LoRA paper (Hu et al., 2022) and best practices from Karpathy.
Heuristic configuration based on the LoRA paper, Karpathy's best practices and Hugging Face's PEFT documentation. The values are recommended starting points – always experiment with your own validation loss to optimize for your specific dataset.
Fast Presets
Generated LORA configuration
{
"peft_type": "LORA",
"task_type": "CAUSAL_LM",
"r": 16,
"lora_alpha": 32,
"lora_dropout": 0.05,
"target_modules": [
"q_proj",
"k_proj",
"v_proj",
"o_proj"
],
"bias": "none",
"inference_mode": false
}Come utilizzare LoRA Adapter Config
Choose Base Model and Task
Choose one of the 5 base models (Llama 3.1 8B/70B, Mistral 7B, Qwen 2.5 14B, Gemma 2 9B) and task type (instruction tuning, domain adaptation, code, reasoning, multilingual). Alternatively, load one of the 5 common presets.
Set VRAM and dataset
Indicate available VRAM in GB, dataset size (number of rows), and quantization level (FP16, QLoRA 4-bit or 8-bit).
Read generated configuration
Get calculated rank (r), alpha, target modules, learning rate, batch size and gradient accumulation automatically, with parameter rationale and ready-to-copy HuggingFace PEFT JSON.
Suggerimenti
- Start with a similar preset to your use case and then fine-tune VRAM and dataset size instead of configuring everything from scratch.
- If you have limited VRAM, prefer QLoRA 4-bit: in most tasks, the loss of quality is marginal compared to memory savings.
- Open the "Rationale" section to understand why each value is before launching an expensive training: helps identify obviously wrong configurations for your hardware.
Domande frequenti
How are rank (r) and alpha values calculated?
Rank and alpha are derived from the size of the dataset and the task type: smaller datasets and simpler tasks use lower ranks to avoid overfitting, while more complex tasks (reasoning, code) or larger datasets justify higher ranks. The specific reason is visible in the "Rationale" table of the result.
What's the difference between QLoRA 4-bit and 8-bit?
QLoRA 4-bit reduces VRAM usage by about 70% with minimal quality degradation, while QLoRA 8-bit saves around 45% of VRAM with very similar quality to FP16. FP16 offers the highest quality but requires full VRAM without quantization.
How is batch size and gradient accumulation chosen?
Batch size and gradient accumulation steps balanced based on available VRAM, calculating an effective batch size (batch x accumulation steps) to approximate stable training without overloading the GPU's memory.
Is the generated JSON compatible with any end-to-end tuning library?
JSON is formatted for HuggingFace PEFT (LoraConfig), the most widely used standard for fine-tuning LoRA/QLoRA: if you use another framework, you may need to adjust parameter names, but numeric values (r, alpha, target modules, learning rate) remain valid as a starting point.
Why do target modules change based on the base model?
Each architecture (Llama, Mistral, Qwen, Gemma) displays different layer names/projections (e.g. q_proj, v_proj, gate_proj): the tool selects the correct target modules for the chosen base model's architecture, avoiding common configuration errors when copying-pasting config from one model to another.