Codex deleted your database? When a test's drop_all() reaches production

VERICTO-010

In the public report (openai/codex#42480, September 2026), Codex ran a test file from a production deployment directory. The test setup called SQLAlchemy's Base.metadata.drop_all(engine), and because the process loaded the deployment's .env with the production database settings, it connected to the production PostgreSQL database and dropped and recreated the tables. If that matches what you see, stop the services that write to the database, do not run the tests again, and restore to a moment just before the test run.

Recover now

  • Stop the agent or the process. End the agent session, cancel any background task, and stop the test run, deploy job or service that ran the command, so nothing runs again against the database you are about to restore.
  • Do not run more migrations, pushes or resets, and do not let the agent "repair" the schema. Recreating the tables brings back empty tables, not the rows. Write down when the destructive command ran (from the agent transcript, the shell history or the database log): every restore below needs that moment.
  • Restore to a point just before the command, into a separate instance. Self-managed PostgreSQL: restore the base backup and replay the archived WAL with recovery_target_time set just before it; a pg_dump file restores only to when it was taken. Amazon RDS / Aurora: "Restore to point in time" creates a new DB instance (RDS) or cluster (Aurora) at any moment within the backup retention period, without modifying the source. Neon: instant restore takes a root branch back to a moment within its history window. Supabase: the daily backups and, where it is enabled, point-in-time recovery are under Database → Backups. Railway: volume backups, if they were created or scheduled, are restored from the service. MySQL: restore the last full backup, then replay the binary log with mysqlbinlog up to just before the destructive event.
  • Check the restored copy, then move it back. Compare row counts and the latest rows with what you expect, then point the application at the restored instance or copy the tables back (pg_dump -t table from the restored instance). Rows written between the restore point and now must be re-applied by hand.
  • No backup and no point-in-time recovery? Look for other copies: a staging or development database, a read replica, exports, a data warehouse, or the systems the data came from.

Why it happened

A test that creates its tables with create_all() and removes them with drop_all() is a normal pattern. drop_all() drops every table in the metadata that exists in the database the engine points at, and the engine points wherever the configuration says. When the test resolves its database URL from the environment and runs in a directory whose .env holds production settings, the teardown runs on production. The reporter describes exactly that: the process inherited a deployment .env. Running the tests was a normal step for the agent; the target was wrong, and nothing between the test and the database checked it.

How to prevent it

drop_all() sends one DROP TABLE per table over the engine's connection. If the only path to production that the process can reach is the proxy, those statements are blocked before they reach the database: the test fails, the tables stay. A test database never needs the production proxy; the point is that a test run that resolves to production meets the proxy, not the database.

Vericto checks each SQL statement on its AST before it reaches the database. These are the rules that report the statements of this incident, with the real CLI output on each rule's example:

VERICTO-010: DROP TABLE or DROP DATABASE

The rule's example · Critical

-- migrations/0042_drop_users.sql
DROP TABLE users;

Real CLI output

$ npx @vericto/vericto-cli@2.3.0 check --offline migrations/0042_drop_users.sql
note: offline mode: 42 standard rules at default severities, engine 5.1.0. Custom rules, overrides, sensitive columns, allowlists and the audit trail apply online only.
! migrations/0042_drop_users.sql  FLAGGED  VERICTO-010 critical
    DropStmt
    fix: Use a versioned migration tool (Flyway, Prisma Migrate, Alembic, Rails migrations)
0 blocked · 0 allowed · 1 flagged · 0 monitored   (ruleset standard@engine-5.1.0 · mode: offline (engine 5.1.0, 42 standard rules, no workspace policy))

In a migration file the CLI marks it FLAGGED and exits with 0: a schema change in a migration is usually intended, so it is raised for review instead of blocked. With --fail-on flag (or fail_on = "flag" in .vericto.toml) it exits with 1 and the job fails. Outside a migration (--sql, stdin or another file) it blocks, as the proxy and MCP check_sql do.

Set up the agent so it goes through Vericto

  1. Connect the agent to the database through the proxy. Its connection URL points at the Vericto TCP proxy, not at the database. The proxy checks each statement, including the ones an ORM command run by the agent sends, and at runtime it blocks DROP, TRUNCATE and ALTER … DROP COLUMN before they reach the database. For a coding agent, vericto-proxy mcp gives it execute_sql, describe_schema and check_sql, never the database credential.
  2. Keep the direct production credential away from the agent. Not in the .env, the shell or the environment the agent and its test runs load. If the agent has another way to the database, the proxy does not see it.
  3. Give it a read-only role through Vericto. The role behind vericto-proxy mcp is read-only (the SQL to create it is in the MCP docs), and agent access sets each agent to read-only on the tables you choose.
  4. Let the agent check SQL before it runs it with the MCP check_sql tool: it gets the verdict, the rule and the suggested fix.

In CI, on the migration files

A test fixture never produces a .sql file, so CI cannot see this one. What CI can check are your Alembic migrations: alembic upgrade head --sql writes the SQL they would run (offline mode), and the Action checks that file in the pull request.

The vericto/check-sql GitHub Action (Apache-2.0) runs the same offline check on the .sql files a PR changes. Save it as .github/workflows/vericto.yml:

on: pull_request
jobs:
  vericto:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with: { fetch-depth: 0 }
      - uses: vericto/check-sql@v1
        with: { offline: true, upload-sarif: false }

Or try the CLI on your machine, with no account (AGPL-3.0-only):

npx @vericto/vericto-cli@2.3.0 check --offline migrations/

Scope: Vericto protects the connections that go through Vericto. A shell on the database host, or a credential that connects directly, bypasses it.

Frequently asked questions

Sources

The public reports this page cites, checked on 2026-11-14, and the official docs behind each fact about the tools: