A vibe check for AI coding

TL;DR: A recent string of outages at Amazon’s website and cloud services has revived questions about the risks of AI-assisted coding. Amazon publicly disputes claims that AI-generated code was responsible, but internal discussions and reports suggest generative AI tools may have played a role–highlighting an existential challenge in the tech industry about the risks of using AI to code.

What happened: Yesterday, Amazon engineers devoted a team meeting to “a deep dive” on a recent slew of outages, according to the Financial Times. A memo urging attendance initially referenced “GenAI-assisted changes” and “GenAI tools,” but CNBC reported that the bullet point was later scrubbed from the document ahead of the meeting.

Despite all the cloak-and-dagger, the outages themselves have been quite public. The FT reported last month that Amazon’s coding agent Kiro was responsible for multiple Amazon Web Services outages—including one in December when Kiro took down AWS for 13 hours after “deleting and then recreating” part of its environment. Amazon published a whole blog post disputing the issues, blaming the problems on humans rather than AI. Separately, another store outage was tied to Amazon’s coding assistant Q, according to an internal document obtained by Business Insider.

The company is now reportedly tightening oversight, requiring senior engineers to sign off on AI-assisted changes from junior developers.

An Amazon spokesperson disputed the reported new code review process. “As part of normal business, the meeting will include a review of the availability of our website and app as we focus on continual improvement,” they said of yesterday’s meeting.

AI code is everywhere now: The debate over Amazon’s outages comes as software development becomes one of the most widely adopted uses of generative AI. Microsoft CEO Satya Nadella and Google CEO Sundar Pichai have both said that AI is already writing around 30% of their new code. Some top engineers at Anthropic and OpenAI claim they no longer even write code. They just review AI output.

The tongue-in-cheek term “vibe coding” has also caught on for a reason: AI-assisted programming has a reputation for playing faster and looser than the traditionally exacting process of writing code. The result? Quicker development but messier code.

That’s created a new bottleneck: Companies can generate far more software than they can easily review. (This week, Anthropic released a new AI code-review tool to catch bugs in AI-generated code before it reaches production.)

That speed can come with risks: “This sort of issue will become more prevalent because right now when a human operator acts, they do things with an understanding of the overall environment and knowledge of what they should and should not do,” according to research company Forrester Principal Analyst Brent Ellis. “An AI however will use whatever resources it has access to in order to try to achieve the goal it is given.”

Still, some analysts say it’s too early to conclude that AI-generated code will lead to more outages overall.

“The bar for AI code is the human error rate,” Constellation Research Principal Analyst Holger Mueller said. But, he noted, AI code could create more widespread security vulnerabilities and errors if all of the big cloud companies are drawing from the same AI coding platform.

The bottom line: AI coding has quickly become the norm at many of the biggest tech companies. But the Amazon outages offer a glimpse of the trade-off: When companies generate far more code, they also have to figure out how to review and control it. —PK