DeepSeek Watch posted an update
DeepSeek’s V4.1 Flash-1017 pairs low running costs with strong benchmark results, but a reported reliability test raises a practical warning: it may answer when it should admit it does not know.
Why it mattersMezha, citing Artificial Analysis, says the model hallucinated in 67% of cases where it should have abstained. It answered 43% of questions correctly in the benchmark. Those figures describe that test, not every use of the model. The practical split is clear enough: a coding task whose output can be checked may suit it better than open-ended research or precise fact-finding. Cheap and capable is a useful combination; cheap and confidently wrong is rather less so.
Discuss: Would you use a low-cost model that performs well on checkable tasks if it often fails to recognise when it does not know the answer?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.