Skip to content

Commit fed8a7b

Browse files
MisterRaindroptuhaihe
authored andcommitted
interconnect: restart in fast mode so the test does not race crash recovery
The test sets shared_preload_libraries and restarts with "gpstop -raiq". An immediate shutdown skips the shutdown checkpoint, so the control file is left in a state other than DB_SHUTDOWNED and the next startup performs crash recovery: xlogrecovery.c sets InRecovery, xlog.c calls PerformWalRecovery(), which signals PMSIGNAL_RECOVERY_STARTED, and the postmaster moves to PM_RECOVERY. In that state canAcceptConnections() answers CAC_NOTCONSISTENT, reported as "the database system is not accepting connections" with detail "Hot standby mode is disabled". gpstart makes exactly such a connection right after pg_ctl returns, to read the segment configuration, so gpstop -r exits CRITICAL and the restart is reported as failed. The damage does not stop there. psql gives up at the \c that follows, so every statement in the file is skipped and the test fails as a whole; the cleanup at the end of the file never runs; and gpstart never got past starting the coordinator in admin mode, so the cluster is left with no segments up. Suites that run after this one in the same job then lose their Gather Motion nodes and fail as well. Shut down fast instead. A fast shutdown writes the shutdown checkpoint, the control file says DB_SHUTDOWNED, no recovery runs, PM_RECOVERY is never entered, and CAC_NOTCONSISTENT cannot be returned -- the failure becomes unreachable rather than merely less likely. Fast is also what the rest of the tree already uses: gpstop -raf/-arf appear in dozens of places, and this file was the only user of -raiq. Measured on a three-segment demo cluster, dirtying 1.5M coordinator rows before each restart so that recovery is slow enough to lose the race reliably: -raiq failed 2/2 with the message above, -rafq passed 3/3 with all three segments still up afterwards. pg_controldata confirms the mechanism at the other end -- "in production" after an immediate shutdown, "shut down" after a fast one. The test still passes under pg_regress with the change.
1 parent e045165 commit fed8a7b

1 file changed

Lines changed: 8 additions & 2 deletions

File tree

‎contrib/interconnect/sql/interconnect.sql‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
-- start_ignore
22
\! gpconfig -c shared_preload_libraries -v "interconnect"
3-
\! gpstop -raiq
3+
-- Restart in fast mode, not immediate: an immediate shutdown skips the
4+
-- shutdown checkpoint, so the next startup runs crash recovery, and while the
5+
-- postmaster is in PM_RECOVERY it rejects the connection gpstart makes to read
6+
-- the segment configuration ("the database system is not accepting
7+
-- connections"). gpstop -r then fails, psql gives up at the \c below, and the
8+
-- whole file is skipped with only the coordinator left running.
9+
\! gpstop -rafq
410
\c
511
DROP TABLE IF EXISTS test_ic_data;
612
CREATE EXTENSION IF NOT EXISTS interconnect;
@@ -83,5 +89,5 @@ DROP EXTENSION interconnect;
8389

8490
-- start_ignore
8591
\! gpconfig -r shared_preload_libraries
86-
\! gpstop -raiq
92+
\! gpstop -rafq
8793
-- end_ignore

0 commit comments

Comments
 (0)