Notes ยท Munna Suprathik
Fine-Tuning Was the Wrong Answer
By Munna Suprathik, Generative AI Engineer.
I spent a whole weekend fine-tuning a Mistral 7B with LoRA because I had decided, before checking, that the task needed it. Curated the dataset, formatted it, ran the training, watched the loss go down, felt like a real ML engineer.
Then someone solved the exact same problem with a better prompt and a small retrieval step, in an afternoon, for the price of a few API calls. My fine-tune was not even better. It was just more work and a model I now had to host, version, and eval separately.
Lots of knobs to turn. Most days you should not.
Here is the thing about LoRA and PEFT: they are great when the base model genuinely lacks a capability or a style you cannot coax out of it, and they are a very seductive way to feel productive when the real fix is upstream. My order of operations now is boring and it saves me every time. Sharpen the prompt. Give it the right context through retrieval. Only reach for fine-tuning when the model truly cannot do the thing, not when I have not asked it properly.
Most "the model can't do this" bugs are "I explained it badly" bugs wearing a costume.