AWS’s AI customisation ladder says: start with the prompt, not the GPU bill

The most expensive AI customisation mistake may be reaching for training before proving that a better prompt or a small retrieval layer would do the job. AWS has published an eight-step framework that starts with using a model as-is and ends with building a custom model from scratch.

AWS AI Watch analysis

What happened

In its generative AI customisation guide, AWS groups the options into three broad moves: use the model, enhance it with surrounding tools and data, or train its weights. The practical rule is to begin at the simplest step, then move up only when the current approach fails on accuracy, latency or domain requirements.

Key findings

• Prompt before fine-tuning
  Clear instructions, examples and structured context may solve problems that teams otherwise send straight to training.
• RAG adds changing knowledge
  Retrieval-augmented generation can ground answers in private or frequently updated documents without changing model weights.
• Caching tackles repetition
  Prompt caching and context optimisation can reduce cost and latency for workloads that reuse large instruction sets.
• Distillation makes models smaller
  A smaller student model can imitate a larger teacher for a validated task, trading generality for speed and lower cost.
• Fine-tuning changes behaviour
  Updating weights suits specialised tone, formats and task behaviour when prompting and retrieval are not enough.
• Continued pre-training adds knowledge
  Large unlabeled corpora can extend a model’s foundational understanding of a domain.
• Custom training is the deep end
  Building from scratch offers maximum control but demands substantial data, compute, expertise and patience.

Why it matters

The useful takeaway is a decision rule, not another staircase-shaped diagram for the conference wall. Teams can test the cheap, reversible options first, measure where they fail, then spend more only when the evidence says they must.

That also makes the framework a useful antidote to two familiar habits: fine-tuning a model when the real problem is poor instructions, or endlessly polishing prompts when the model simply lacks the required private knowledge. The guide’s limits are worth keeping in view: its examples and service mapping come from AWS, and the advice is a framework rather than an independent benchmark of every approach.

Our read

Start with a baseline prompt and a test set. Add retrieval if the missing ingredient is current or private information. Consider caching when repeated context is inflating the bill, distillation when a validated task needs to run faster or cheaper, and weight updates only when the simpler layers have genuinely failed.

In other words, earn the GPU bill. Your future self, and possibly your finance team, will be grateful.

What to watch

• Whether teams publish measured break-even points between RAG, distillation and fine-tuning.
• How AWS’s service recommendations compare with open-source or vendor-neutral alternatives.
• Whether smaller distilled models retain reliability outside the narrow tasks used for validation.
• Which workloads actually justify continued pre-training or a custom model from scratch.

Discussion spark: When a generative AI system falls short, which signal should trigger the move from prompting and retrieval to changing the model’s weights?

not affiliated with or endorsed by Amazon Web Services (AWS)