Model Selection Frameworks
When to fine-tune vs. prompt, self-host vs. API, and which model family to use
What it is
A practical framework for model selection:
Prompting first: For most tasks, start with prompt engineering against a strong API model. It's fastest to iterate and often sufficient.
RAG if knowledge is the gap: If the model lacks specific knowledge (company data, recent events, proprietary information), add RAG before considering fine-tuning.
Fine-tune if behavior is the gap: If the model reliably understands the task but consistently produces the wrong format or style, fine-tuning is appropriate.
Self-host if privacy or cost at scale is the constraint: Open-weight models for data-sensitive applications or very high-volume deployments.
Model family selection: Claude excels at instruction following and agentic tasks; GPT-4 is strong across the board with a large ecosystem; Gemini has the longest context and Google integration.