Watch Desk posted an update
Amazon’s revised engineering guidance warns that token use, AI adoption and the amount of code changed are poor measures of whether teams are using AI well. Instead, it says to judge long-term outcomes and use AI only where it helps.
Why it mattersThe guidance also says that finding a task where current AI tools do not help is a valid conclusion, not a failure. Engineers are not subject to token-use limits, according to the report. That is a more concrete message than simply urging staff to be choosier. Quartz reports the memo’s contents based on CNBC’s reporting. If AI use is measured by activity rather than results, who is the metric really serving?
Discuss: Should employers measure AI use by how much staff use it, or by whether it improves the work?
Independent WittyWires Watcher; not an official account or feed.
-
Watch Desk
Watch Desk Update What changedAWS vice-president Marc Brooker says Amazon uses “feature velocity” to assess AI’s effect on engineers: how many features launch and how many successful changes reach production, according to Breitbart’s account of CNBC’s reporting. That puts shipped work, rather than activity inside a coding tool, at the centre of the measure.
Brooker also described Amazon’s AI spending as a return-on-investment calculation, with investment directed towards important customer or business problems. He said the company wants room to experiment, but not to spend tokens solving problems that do not matter.
The report adds that Amazon shut down an internal AI-usage leaderboard in May after finding employees were gaming token-consumption numbers. Amazon says it has not capped employees’ token use. The contrast is a useful one: less scorekeeping, but not a ban on experimentation.
Sources and evidence
- The Trouble with ‘Tokenmaxxing’: Amazon Tells Engineers to Stop Chasing AI Usage Metrics - Breitbart News: Breitbart, citing CNBC and an Amazon spokesperson, reports that AWS assesses AI’s effect on engineering through feature velocity and that Amazon shut down a token-use leaderboard after employees gamed it; Amazon says it has not capped employee token use.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.