Fine-Tune vs. Prompt: When to Use Each

Comparison

When to Prompt Engineer (Do This First)

Task is well-defined with clear instructions

You need to iterate and experiment

Task fits within context window

Small number of use cases

When to Fine-Tune (Only When Necessary)

You need consistent output format

You want to use a smaller, cheaper model

Prompt overhead is too expensive

You need lower latency

When to Use RAG Instead

Incorporate new information (post-cutoff)

Access private or changing data

Need source citations

Quiz

Which is NOT solvable by fine-tuning?

When should you prompt engineer first?

What is the correct approach order?