When a managed PostgreSQL primary becomes unreachable, it may still be accepting writes. ClickHouse’s account of its failover design explains how it tries to fence off that old server before promoting a standby, and why that safeguard has limits.
ClickHouse Watch analysis
What happened
ClickHouse describes several ways its managed PostgreSQL control plane can stop clients reaching an old primary. On bare metal, it can cut the virtual machine’s network link from the host. On AWS, it detaches a separate network interface used for database traffic. On Google Cloud, it removes a dynamic network interface. The approaches are designed to cut existing connections as well as new ones, which simply changing DNS or blocking new connections may not do.
The post also explains why failover can be awkward: a network partition can leave clients writing to a primary the control plane cannot reach. PostgreSQL’s replication acknowledgements do not necessarily prevent that. ClickHouse says it tries several fencing methods in parallel, then proceeds with promotion once one succeeds.
There is a consequential qualification. The post says the control plane is deliberately designed to proceed with promotion even if every fencing method fails. That is a fail-open choice, not evidence that every failover is unsafe, but it makes the trade-off plain: avoiding a prolonged outage can come at the cost of promoting without a confirmed fence.
Why it matters
Failover is not just a matter of pointing traffic at a standby. If the old primary is still reachable by some clients, two servers can accept writes, creating the split-brain problem the fencing process is meant to prevent. The post’s provider-specific examples show why database operators need to consider existing connections, network layout and what happens when the control plane itself cannot reach the machine.
The policy for when to promote matters too. ClickHouse’s explanation gives readers a concrete design choice to examine: wait for a person when every fence fails, or keep the service moving and accept the remaining uncertainty.
Our read
This is a useful infrastructure post because it gets past the comforting diagram where a standby simply takes over. ClickHouse lays out practical fencing mechanisms and openly describes its fail-open behaviour. Operators should pay attention to that final choice, and ask whether their own recovery procedures make the same trade-off.
What to watch
- How ClickHouse’s design handles cases where every fencing mechanism fails.
- Whether the company publishes further detail on safeguards against simultaneous writes.
- How the provider-specific fencing methods behave as its managed PostgreSQL service evolves.
Discussion spark: If every fencing method fails during a database outage, should an operator allow automatic promotion to restore service, or require human approval to reduce split-brain risk?
Sources and evidence
- Source update (4 November 2026, 13:50 UTC)
not affiliated with or endorsed by ClickHouse