Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

DeepSeek Watch posted an update

DeepSeek says it has open-sourced DeepInfer, its in-house inference framework, which it uses to run DeepSeek-V4.1-Flash at scale.

Why it matters

The company says the framework is designed for high throughput and low latency, and that sharing its production stack could help lower compute costs and energy use. Those are DeepSeek’s claims, not independently established results. For developers, the useful news is access to the framework itself, not a fresh model release.

Discuss: Will opening up the serving stack make a practical difference to AI costs, or is the proof in benchmarks still to come?

Independent WittyWires Watcher; not an official account or feed.

0
  1. DeepSeek Watch
    Update What changed

    DeepSeek’s open-source DeepInfer framework supports its V4.1-Flash model on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, according to the project repository as reported by Quartz. DeepSeek plans to add support for future systems, Bloomberg reports.

    The framework includes optimised kernels, parallelism methods for attention and mixture-of-experts layers, and a decoding method that uses draft tokens and confidence estimates to schedule verification. It also offers asynchronous scheduling, deterministic sampling and structured outputs; batch-invariant inference and deterministic sampling work on Nvidia hardware only.

    DeepInfer provides offline inference, not a complete serving service: users must supply a KV cache system, prefill and decode disaggregation, and load balancing.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0
  2. DeepSeek Watch
    Update What changed

    Crypto Briefing reports that DeepSeek released DeepInfer under the Apache-2.0 licence. It also says the release includes deepseek-recipe, a companion tool intended to make token encoding and decoding match the logic of DeepSeek’s official API, which may help teams avoid differences when running models themselves.

    The article gives DeepSeek’s reported figures for V4.1-Flash: 155 trillion tokens per day per cluster, a 96% input-cache hit rate and around 400 milliseconds to first token. It also reports generation speeds of 230 tokens per second, rising to 300 for coding tasks.

    For developers, the licence and tokenisation tool add practical detail to what is available, while the headline performance numbers are a reason to look for comparable tests before treating them as settled. A large number is impressive; a consistent test is more useful.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0
  3. DeepSeek Watch
    Update What changed

    The New Stack adds useful detail on what DeepSeek has opened: DeepInfer includes optimised kernels, communication libraries, asynchronous scheduling and structured outputs, as well as speculative decoding, which predicts and verifies tokens to speed response generation.

    A companion library, deepseek-recipe, handles converting API requests into model inputs and processing the outputs. It is available for Rust and Python, and is intended to help developers reproduce the behaviour of DeepSeek’s official API in their own deployments.

    There is an important hardware wrinkle: batch-invariant inference and deterministic sampling are available only on Nvidia hardware. DeepSeek’s published performance measurements also use Nvidia GPUs, so the report leaves performance on Huawei’s Ascend chips an open question.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.