[Bug] Walreciever crashes with SIGABRT for user with scram-sha-256 password

[HustonMmmavr]

Apache Cloudberry version

The main branch 15d75c6. Older versions should be affected too because the problem exists at greenplum

What happened

Walreciever crashes with SIGABRT when the connection with primary closes (for ex reconnect) if replication user uses a scram-sha-256 password.

There is a stack trace:

(gdb) bt 
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x000076dc2424527e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x000076dc242288ff in __GI_abort () at ./stdlib/abort.c:79
#5  0x000076dc242297b6 in __libc_message_impl (fmt=fmt@entry=0x76dc243ce8d7 "%s\n") at ../sysdeps/posix/libc_fatal.c:134
#6  0x000076dc242a90d5 in malloc_printerr (str=str@entry=0x76dc243d1520 "munmap_chunk(): invalid pointer") at ./malloc/malloc.c:5775
#7  0x000076dc242a955c in munmap_chunk (p=<optimized out>) at ./malloc/malloc.c:3040
#8  0x000076dc242adefa in __GI___libc_free (mem=0x5d2ba9297988) at ./malloc/malloc.c:3388
#9  0x00005d2b6cc4608f in scram_free (opaq=0x5d2ba92d8230) at fe-auth-scram.c:190
#10 0x00005d2b6cc2ffe2 in pqDropConnection (conn=0x5d2ba92d7820, flushInput=false) at fe-connect.c:612
#11 0x00005d2b6cc44b62 in pqReadData (conn=0x5d2ba92d7820) at fe-misc.c:808
#12 0x00005d2b6cc3df28 in PQconsumeInput (conn=0x5d2ba92d7820) at fe-exec.c:2042
#13 0x00005d2b6ce6ae94 in libpqrcv_PQgetResult (streamConn=0x5d2ba92d7820) at libpqwalreceiver/libpqwalreceiver.c:802
#14 0x00005d2b6ce6b070 in libpqrcv_receive (conn=0x5d2ba9297848, buffer=0x7ffc2d8b1b58, wait_fd=0x7ffc2d8b1b28) at libpqwalreceiver/libpqwalreceiver.c:879
#15 0x00005d2b6ce5d572 in WalReceiverMain () at walreceiver.c:480
#16 0x00005d2b6cdeb4f8 in AuxiliaryProcessMain (auxtype=WalReceiverProcess) at auxprocess.c:161
#17 0x00005d2b6cdf8a19 in StartChildProcess (type=WalReceiverProcess) at postmaster.c:6065
#18 0x00005d2b6cdf903a in MaybeStartWalReceiver () at postmaster.c:6315
#19 0x00005d2b6cdf877d in process_pm_pmsignal () at postmaster.c:5857
#20 0x00005d2b6cdf3320 in ServerLoop () at postmaster.c:2098
#21 0x00005d2b6cdf2a3f in PostmasterMain (argc=7, argv=0x5d2ba92961b0) at postmaster.c:1750
#22 0x00005d2b6cc4984d in main (argc=7, argv=0x5d2ba92961b0) at main.c:260

This problem occurs due to linkage frontend code into backend, the scram_free expects that all memory is allocated via malloc and uses free to release it. But the fe-auth-scram.c is compiled into backend binary where saslprep.c compiled without FRONTEND macro - all allocation macros uses postgres memory contexts, so all allocation in pg_saslprep include output password is performed with its usage 1 2. Thus free fails to release palloc'd memory.

What you think should happen instead

No SIGABRT

How to reproduce

How to reproduce:

• create a cluster with mirrors

  DATADIRS=$HOME/cloudberry-data  PORT_BASE=7000 NUM_PRIMARY_MIRROR_PAIRS=1 WITH_MIRRORS=true make create-demo-cluster

• set up current user scram-sha-256 password

  set password_encryption to 'scram-sha-256';
  alter user gpadmin  with password 'password';

• for dbfast1 change pg_hba.conf- add new record for replication database and gpadmin user with auth method to scram-sha-256 (all other disable)

  host    replication     gpadmin             127.0.0.1/32         scram-sha-256
  

  sed -i 's/.*replication.*/#&/g' ~/cloudberry-data/dbfast1/demoDataDir0/pg_hba.conf
  echo 'host    replication     gpadmin             127.0.0.1/32         scram-sha-256' >> ~/cloudberry-data/dbfast1/demoDataDir0/pg_hba.conf

• create password record:

  echo '*:*:*:gpadmin:password' > $HOME/.pgpass && chmod 0600 $HOME/.pgpass

• reload the config

  gpstop -u

• reload the mirror - to enable scram connection

   pg_ctl stop -D ~/cloudberry-data/dbfast_mirror1/demoDataDir0/
   pg_ctl start -D ~/cloudberry-data/dbfast_mirror1/demoDataDir0/ -o '-c gp_role=execute -p 7003'

• stop the primary

  pg_ctl stop -D ~/cloudberry-data/dbfast1/demoDataDir0

• the log of dbfast_mirror1 would contain the info about the problem

Operating System

Ubuntu 24.04.4

Anything else

No response

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct.

[github-actions]

Hi, @HustonMmmavr welcome!🎊 Thanks for taking the time to point this out.🙌