Postgres disk fills up rapidly after a few days with Upbound Hub connected control planes

Last updated: October 8, 2026

Postgres disk fills up rapidly after a few days with Upbound Hub connected control planes

Issue

After running Upbound Hub for a few days with connected control planes, the Postgres database runs out of disk space and enters a CrashLoopBackOff. Inspecting the database reveals that inbox_messages_* weekly partition tables have grown to several gigabytes - nearly filling the entire PersistentVolumeClaim - with millions of rows all marked as processed. The growth rate can be 4–5 GB/day per a small number of connected control planes.

When Postgres becomes unavailable, hub-core is unable to process requests, which can cascade to connectors on connected control planes. Connectors may report errors such as invalid_grant: subject_token is invalid or expired, which can appear to be a token issue but is actually caused by hub-core being unable to respond while Postgres is down.

Environment

Upbound Hub v1.1.1 · BYO Postgres · EKS · Kubernetes 1.36

Observed with 2 or more connected control planes syncing a large number of resources.

Resolution

This is fixed in Upbound Hub v1.1.2. Upgrade to v1.1.2 or higher, then follow Proactively moving to hourly partitions for Upbound Hub v1.1.2 to switch existing installations to hourly partitions.

If Postgres has already run out of disk, recover first, then upgrade:

  1. Expand the Postgres PersistentVolumeClaim to restore database availability.

  2. Drop the oversized processed partition to reclaim space. This is destructive, so confirm the partition contains only processed rows before dropping it:

SELECT status, count(*) FROM inbox_messages_<year>_<week> GROUP BY status;

  1. Once Postgres is back online, hub-core will resume processing and connector errors should clear automatically.

Monitoring inbox partition growth

Partition sizes:

SELECT tablename, pg_size_pretty(pg_total_relation_size('public.'||tablename)) as size
FROM pg_tables WHERE tablename LIKE 'inbox_messages_%' ORDER BY tablename;

Row count and status breakdown for a specific partition:

SELECT status, count(*) FROM inbox_messages_<year>_<week> GROUP BY status;

Overall database size:

SELECT pg_size_pretty(pg_database_size('hub')) as total_db_size;

Cause

The Hub connector writes a message to the inbox_messages table for every resource change observed on connected control planes. In v1.1.1, weekly inbox partitions are used instead of the intended hourly partitions, and processed messages are not cleaned up. Partitions accumulate indefinitely, consuming disk at a rate proportional to the number of connected control planes and managed resources. v1.1.2 restores hourly partitioning and cleanup.