LLM Fine-tuning & PEFT Techniques

Efficient LLM Fine-tuning

Why Fine-tune?

• Adapt models to domain-specific tasks
• Improve performance on your data
• Reduce prompt engineering needs
• Cost-effective vs training from scratch

Full Fine-tuning vs PEFT

Full Fine-tuning

• Updates all model parameters
• Requires massive GPU memory
• Risk of catastrophic forgetting
• Cost: ₹10,000+ for small models

PEFT (Parameter-Efficient)

• Updates only 0.1-1% of parameters
• Runs on consumer GPUs
• Preserves pre-trained knowledge
• Cost: ₹100-1,000

Popular PEFT Methods

1. LoRA (Low-Rank Adaptation)

Most popular method for LLM fine-tuning

• Adds trainable low-rank matrices
• Freezes original weights
• Typical rank: 8-64
• Memory: 3GB for 7B model

2. QLoRA (Quantized LoRA)

• 4-bit quantization + LoRA
• Even lower memory usage
• Fine-tune 65B models on 1x RTX 4090
• Minimal performance loss

3. Prefix Tuning

• Prepends learned vectors to input
• Fast and efficient
• Good for few-shot learning

4. Adapters

• Inserts small modules between layers
• Each task gets its own adapter
• Easy to switch between tasks

Practical Implementation

Using Hugging Face PEFT

from peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=16,  # rank
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
)

model = get_peft_model(base_model, config)

Best Practices

• Start with LoRA (r=8 or 16)
• Use QLoRA for large models
• Target attention layers first
• Monitor for overfitting
• Use validation set

Indian Cloud Options

• E2E Networks: A100 GPUs, ₹80/hour
• Lambda Labs: A100 access, $1.10/hour
• Google Colab Pro: Good for learning, ₹850/month
• Vast.ai: Cheapest option, from $0.20/hour

Free Resources

• Hugging Face PEFT documentation
• QLoRA paper and GitHub
• Sebastian Raschka's LLM workshops
• Maxime Labonne's LLM course