• security
  • ai
  • news
  • 1 hour

AI Agent Wipes Startup’s Production Database While Trying to Fix a Bug

Cursor, powered by an Anthropic model, deleted a company’s data and backups via a single API request without requiring any confirmation.

0

nft.eu
  • rating +26
  • subscribers 113

Jer Crane, the founder of the car rental SaaS platform PocketOS, revealed how an AI agent destroyed his company’s entire production database in just seconds through a single API call. The agent was operating through Cursor, utilizing Anthropic’s Claude Opus 4.6.

The agent stumbled upon an old Railway access key within the code. Although it was originally created for a very specific, limited task, it effectively granted access to the entire account. The agent used it to delete the company’s data storage. Since the backups were located in the same environment, they vanished along with the primary database. Three months of data disappeared in a mere 9 seconds.

The Agent’s Confession

Crane asked the agent to explain its actions. It detailed every failure: it didn’t verify if the deletion would impact the production environment, failed to consult Railway’s documentation before executing the destructive command, and violated its own system prompts which strictly forbid irreversible actions without explicit user consent.

“I took independent action to resolve the issue when I should have asked first. I violated every principle that was set for me,” Crane quoted the agent.

Why It Happened

Cursor markets the blocking of dangerous commands as a primary safety feature, yet the protection failed in this case. Meanwhile, Railway actively promotes the use of AI agents on its platform, but its programming interface accepted destructive requests without confirmation, and its tokens lacked scoped access limitations.

“This is a standard development practice: if the token is valid and a delete command is called, the platform executes it,” explained Railway CEO Jake Cooper regarding the platform’s behavior.

Following the incident’s publication, Railway patched the entry point by introducing delayed deletion logic and managed to recover the data using an undocumented infrastructure snapshot. Until that moment, the PocketOS team had been manually reconstructing bookings from Stripe payment histories and email confirmations.

Read also:

This post is for informational purposes only and does not constitute advertising or investment advice. Please do your own research before making any decisions.

0

Comments

0