StepFun details Step 5 Preview self-hosting, starting at four B300 GPUs

StepFun has documented self-hosting configurations for Step 5 Preview, with its FP8 examples starting at four NVIDIA B300 GPUs or eight H200s. The useful development is an operational recipe for running the model, including a crucial distinction: loading its weights does not guarantee enough memory for its advertised million-token context.

StepFun Watch analysis

What happened

StepFun’s official model card, at its 15 October revision, sets out deployment paths for FP8 and BF16 checkpoints, including an OpenAI-compatible API and offline generation. This moves the practical discussion beyond API access to what an operator needs to run the model themselves.

The hardware requirements differ substantially by precision. The FP8 serving examples use one node with four B300 GPUs or eight H200s. BF16 examples call for eight B300 GPUs or two nodes containing 16 H200s in total.

The dated official model-card revision also explains that the one-million-token context limit covers input and output combined. Operators short of memory should reduce the configured context length or concurrency, rather than assume every listed configuration can accommodate a full-length request.

Why it matters

Step 5 Preview has 600 billion total parameters, with 27 billion active. That sparse architecture reduces the parameters involved in computation, but operators still need a deployment sized for the checkpoint, runtime buffers and attention cache. The smaller active-parameter number is not a hardware shopping list.

Precision is another consequential choice. StepFun explicitly warns that setting a BF16 runtime option on an FP8 checkpoint does not convert its quantised weights into BF16. Teams wanting BF16 must use the appropriate checkpoint, with the larger hardware configurations that entails.

For organisations evaluating self-hosted coding agents or document workflows, these details make capacity planning more concrete. They can now distinguish a configuration intended to load the model from one provisioned for their required document length and number of simultaneous requests.

Our read

The headline parameter count attracts attention; the deployment instructions help someone make a decision. Four B300s is still serious infrastructure, but a documented starting configuration is considerably more useful than being told a model is merely “deployable”.

Start with the precision and context length your workload actually needs, then measure throughput and concurrency. A million-token ceiling is worth having, but there is little glory in reserving it for every request while everyone else waits in the queue.

The supplied revision identifies a custom licence rather than a standard open-source licence. Treat deployment feasibility and permission to use the model as separate checks; the licence label alone does not establish the available rights.

What to watch

• Measured throughput and latency for the documented FP8 and BF16 configurations.
• How usable context length changes as simultaneous requests increase.
• The custom licence’s terms for intended commercial deployments.

Discussion spark: For self-hosted agents, would you prioritise million-token requests or higher concurrency at shorter context lengths, and which workload makes that trade-off worthwhile?

not affiliated with or endorsed by StepFun