I: pbuilder: network access will be disabled during build I: Current time: Tue Feb 11 01:33:23 +14 2025 I: pbuilder-time-stamp: 1739187203 I: Building the build Environment I: extracting base tarball [/var/cache/pbuilder/bullseye-reproducible-base.tgz] I: copying local configuration W: --override-config is not set; not updating apt.conf Read the manpage for details. I: mounting /proc filesystem I: mounting /sys filesystem I: creating /{dev,run}/shm I: mounting /dev/pts filesystem I: redirecting /dev/ptmx to /dev/pts/ptmx I: policy-rc.d already exists I: using eatmydata during job I: Copying source file I: copying [golang-github-hashicorp-raft_1.1.2-1.dsc] I: copying [./golang-github-hashicorp-raft_1.1.2.orig.tar.xz] I: copying [./golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz] I: Extracting source gpgv: unknown type of key resource 'trustedkeys.kbx' gpgv: keyblock resource '/tmp/dpkg-verify-sig.CPYwhOi7/trustedkeys.kbx': General error gpgv: Signature made Sun Jan 19 07:24:28 2020 gpgv: using RSA key 50BC7CF939D20C272A6B065652B6BBD953968D1B gpgv: Can't check signature: No public key dpkg-source: warning: failed to verify signature on ./golang-github-hashicorp-raft_1.1.2-1.dsc dpkg-source: info: extracting golang-github-hashicorp-raft in golang-github-hashicorp-raft-1.1.2 dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2.orig.tar.xz dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz dpkg-source: info: using patch list from debian/patches/series dpkg-source: info: applying t-skip--TestNetworkTransport_AppendEntriesPipeline_CloseStreams.patch dpkg-source: info: applying t-skip--TestRaft_AutoSnapshot.patch dpkg-source: info: applying t-skip--TestRaft_LeadershipTransferLeaderRejectsClientRequests.patch dpkg-source: info: applying t-skip--TestRaft_RecoverCluster.patch dpkg-source: info: applying t-skip--TestRaft_SnapshotRestore_PeerChange.patch I: using fakeroot in build. I: Installing the build-deps I: user script /srv/workspace/pbuilder/63060/tmp/hooks/D01_modify_environment starting debug: Running on ionos6-i386. I: Changing host+domainname to test build reproducibility I: Adding a custom variable just for the fun of it... I: Changing /bin/sh to bash '/bin/sh' -> '/bin/bash' lrwxrwxrwx 1 root root 9 Feb 10 11:33 /bin/sh -> /bin/bash I: Setting pbuilder2's login shell to /bin/bash I: Setting pbuilder2's GECOS to second user,second room,second work-phone,second home-phone,second other I: user script /srv/workspace/pbuilder/63060/tmp/hooks/D01_modify_environment finished I: user script /srv/workspace/pbuilder/63060/tmp/hooks/D02_print_environment starting I: set BASH=/bin/sh BASHOPTS=checkwinsize:cmdhist:complete_fullquote:extquote:force_fignore:globasciiranges:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath BASH_ALIASES=() BASH_ARGC=() BASH_ARGV=() BASH_CMDS=() BASH_LINENO=([0]="12" [1]="0") BASH_SOURCE=([0]="/tmp/hooks/D02_print_environment" [1]="/tmp/hooks/D02_print_environment") BASH_VERSINFO=([0]="5" [1]="1" [2]="4" [3]="1" [4]="release" [5]="i686-pc-linux-gnu") BASH_VERSION='5.1.4(1)-release' BUILDDIR=/build/reproducible-path BUILDUSERGECOS='second user,second room,second work-phone,second home-phone,second other' BUILDUSERNAME=pbuilder2 BUILD_ARCH=i386 DEBIAN_FRONTEND=noninteractive DEB_BUILD_OPTIONS='buildinfo=+all reproducible=+all,-fixfilepath parallel=15 ' DIRSTACK=() DISTRIBUTION=bullseye EUID=0 FUNCNAME=([0]="Echo" [1]="main") GROUPS=() HOME=/root HOSTNAME=i-capture-the-hostname HOSTTYPE=i686 HOST_ARCH=i386 IFS=' ' INVOCATION_ID=5529d8d1cd1b4a2a9b4eedde0fded182 LANG=C LANGUAGE=de_CH:de LC_ALL=C LD_LIBRARY_PATH=/usr/lib/libeatmydata LD_PRELOAD=libeatmydata.so MACHTYPE=i686-pc-linux-gnu MAIL=/var/mail/root OPTERR=1 OPTIND=1 OSTYPE=linux-gnu PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path PBCURRENTCOMMANDLINEOPERATION=build PBUILDER_OPERATION=build PBUILDER_PKGDATADIR=/usr/share/pbuilder PBUILDER_PKGLIBDIR=/usr/lib/pbuilder PBUILDER_SYSCONFDIR=/etc PIPESTATUS=([0]="0") POSIXLY_CORRECT=y PPID=63060 PS4='+ ' PWD=/ SHELL=/bin/bash SHELLOPTS=braceexpand:errexit:hashall:interactive-comments:posix SHLVL=3 SUDO_COMMAND='/usr/bin/timeout -k 24.1h 24h /usr/bin/ionice -c 3 /usr/bin/nice -n 11 /usr/bin/unshare --uts -- /usr/sbin/pbuilder --build --configfile /srv/reproducible-results/rbuild-debian/r-b-build.IdhfiL2y/pbuilderrc_1JhH --distribution bullseye --hookdir /etc/pbuilder/rebuild-hooks --debbuildopts -b --basetgz /var/cache/pbuilder/bullseye-reproducible-base.tgz --buildresult /srv/reproducible-results/rbuild-debian/r-b-build.IdhfiL2y/b2 --logfile b2/build.log golang-github-hashicorp-raft_1.1.2-1.dsc' SUDO_GID=112 SUDO_UID=107 SUDO_USER=jenkins TERM=unknown TZ=/usr/share/zoneinfo/Etc/GMT-14 UID=0 USER=root _='I: set' http_proxy=http://85.184.249.68:3128 I: uname -a Linux i-capture-the-hostname 6.1.0-17-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) x86_64 GNU/Linux I: ls -l /bin total 5776 -rwxr-xr-x 1 root root 1367848 Mar 27 2022 bash -rwxr-xr-x 3 root root 38280 Jul 20 2020 bunzip2 -rwxr-xr-x 3 root root 38280 Jul 20 2020 bzcat lrwxrwxrwx 1 root root 6 Jul 20 2020 bzcmp -> bzdiff -rwxr-xr-x 1 root root 2225 Jul 20 2020 bzdiff lrwxrwxrwx 1 root root 6 Jul 20 2020 bzegrep -> bzgrep -rwxr-xr-x 1 root root 4877 Sep 4 2019 bzexe lrwxrwxrwx 1 root root 6 Jul 20 2020 bzfgrep -> bzgrep -rwxr-xr-x 1 root root 3775 Jul 20 2020 bzgrep -rwxr-xr-x 3 root root 38280 Jul 20 2020 bzip2 -rwxr-xr-x 1 root root 17768 Jul 20 2020 bzip2recover lrwxrwxrwx 1 root root 6 Jul 20 2020 bzless -> bzmore -rwxr-xr-x 1 root root 1297 Jul 20 2020 bzmore -rwxr-xr-x 1 root root 38824 Sep 22 2020 cat -rwxr-xr-x 1 root root 71624 Sep 22 2020 chgrp -rwxr-xr-x 1 root root 67528 Sep 22 2020 chmod -rwxr-xr-x 1 root root 75752 Sep 22 2020 chown -rwxr-xr-x 1 root root 157960 Sep 22 2020 cp -rwxr-xr-x 1 root root 128724 Dec 10 2020 dash -rwxr-xr-x 1 root root 124904 Sep 22 2020 date -rwxr-xr-x 1 root root 92172 Sep 22 2020 dd -rwxr-xr-x 1 root root 100752 Sep 22 2020 df -rwxr-xr-x 1 root root 153964 Sep 22 2020 dir -rwxr-xr-x 1 root root 83644 Jan 20 2022 dmesg lrwxrwxrwx 1 root root 8 Nov 7 2019 dnsdomainname -> hostname lrwxrwxrwx 1 root root 8 Nov 7 2019 domainname -> hostname -rwxr-xr-x 1 root root 34664 Sep 22 2020 echo -rwxr-xr-x 1 root root 28 Jan 25 2023 egrep -rwxr-xr-x 1 root root 34664 Sep 22 2020 false -rwxr-xr-x 1 root root 28 Jan 25 2023 fgrep -rwxr-xr-x 1 root root 71928 Jan 20 2022 findmnt -rwsr-xr-x 1 root root 30112 Feb 26 2021 fusermount -rwxr-xr-x 1 root root 210488 Jan 25 2023 grep -rwxr-xr-x 2 root root 2346 Apr 10 2022 gunzip -rwxr-xr-x 1 root root 6447 Apr 10 2022 gzexe -rwxr-xr-x 1 root root 100952 Apr 10 2022 gzip -rwxr-xr-x 1 root root 21916 Nov 7 2019 hostname -rwxr-xr-x 1 root root 83980 Sep 22 2020 ln -rwxr-xr-x 1 root root 55572 Feb 7 2020 login -rwxr-xr-x 1 root root 153964 Sep 22 2020 ls -rwxr-xr-x 1 root root 153124 Jan 20 2022 lsblk -rwxr-xr-x 1 root root 96328 Sep 22 2020 mkdir -rwxr-xr-x 1 root root 79912 Sep 22 2020 mknod -rwxr-xr-x 1 root root 47048 Sep 22 2020 mktemp -rwxr-xr-x 1 root root 58920 Jan 20 2022 more -rwsr-xr-x 1 root root 50720 Jan 20 2022 mount -rwxr-xr-x 1 root root 13856 Jan 20 2022 mountpoint -rwxr-xr-x 1 root root 157996 Sep 22 2020 mv lrwxrwxrwx 1 root root 8 Nov 7 2019 nisdomainname -> hostname lrwxrwxrwx 1 root root 14 Dec 16 2021 pidof -> /sbin/killall5 -rwxr-xr-x 1 root root 38824 Sep 22 2020 pwd lrwxrwxrwx 1 root root 4 Mar 27 2022 rbash -> bash -rwxr-xr-x 1 root root 46984 Sep 22 2020 readlink -rwxr-xr-x 1 root root 75720 Sep 22 2020 rm -rwxr-xr-x 1 root root 46984 Sep 22 2020 rmdir -rwxr-xr-x 1 root root 22292 Sep 27 2020 run-parts -rwxr-xr-x 1 root root 125036 Dec 22 2018 sed lrwxrwxrwx 1 root root 9 Feb 10 11:33 sh -> /bin/bash -rwxr-xr-x 1 root root 34696 Sep 22 2020 sleep -rwxr-xr-x 1 root root 83880 Sep 22 2020 stty -rwsr-xr-x 1 root root 79396 Jan 20 2022 su -rwxr-xr-x 1 root root 34696 Sep 22 2020 sync -rwxr-xr-x 1 root root 602584 Feb 17 2021 tar -rwxr-xr-x 1 root root 13860 Sep 27 2020 tempfile -rwxr-xr-x 1 root root 108520 Sep 22 2020 touch -rwxr-xr-x 1 root root 34664 Sep 22 2020 true -rwxr-xr-x 1 root root 17768 Feb 26 2021 ulockmgr_server -rwsr-xr-x 1 root root 30236 Jan 20 2022 umount -rwxr-xr-x 1 root root 34664 Sep 22 2020 uname -rwxr-xr-x 2 root root 2346 Apr 10 2022 uncompress -rwxr-xr-x 1 root root 153964 Sep 22 2020 vdir -rwxr-xr-x 1 root root 63024 Jan 20 2022 wdctl lrwxrwxrwx 1 root root 8 Nov 7 2019 ypdomainname -> hostname -rwxr-xr-x 1 root root 1984 Apr 10 2022 zcat -rwxr-xr-x 1 root root 1678 Apr 10 2022 zcmp -rwxr-xr-x 1 root root 5898 Apr 10 2022 zdiff -rwxr-xr-x 1 root root 29 Apr 10 2022 zegrep -rwxr-xr-x 1 root root 29 Apr 10 2022 zfgrep -rwxr-xr-x 1 root root 2081 Apr 10 2022 zforce -rwxr-xr-x 1 root root 8049 Apr 10 2022 zgrep -rwxr-xr-x 1 root root 2206 Apr 10 2022 zless -rwxr-xr-x 1 root root 1842 Apr 10 2022 zmore -rwxr-xr-x 1 root root 4577 Apr 10 2022 znew I: user script /srv/workspace/pbuilder/63060/tmp/hooks/D02_print_environment finished -> Attempting to satisfy build-dependencies -> Creating pbuilder-satisfydepends-dummy package Package: pbuilder-satisfydepends-dummy Version: 0.invalid.0 Architecture: i386 Maintainer: Debian Pbuilder Team Description: Dummy package to satisfy dependencies with aptitude - created by pbuilder This package was created automatically by pbuilder to satisfy the build-dependencies of the package being currently built. Depends: debhelper (>= 12~), dh-golang, golang-any, golang-github-armon-go-metrics-dev, golang-github-hashicorp-go-hclog-dev (>= 0.9.2~), golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~) dpkg-deb: building package 'pbuilder-satisfydepends-dummy' in '/tmp/satisfydepends-aptitude/pbuilder-satisfydepends-dummy.deb'. Selecting previously unselected package pbuilder-satisfydepends-dummy. (Reading database ... 17763 files and directories currently installed.) Preparing to unpack .../pbuilder-satisfydepends-dummy.deb ... Unpacking pbuilder-satisfydepends-dummy (0.invalid.0) ... dpkg: pbuilder-satisfydepends-dummy: dependency problems, but configuring anyway as you requested: pbuilder-satisfydepends-dummy depends on debhelper (>= 12~); however: Package debhelper is not installed. pbuilder-satisfydepends-dummy depends on dh-golang; however: Package dh-golang is not installed. pbuilder-satisfydepends-dummy depends on golang-any; however: Package golang-any is not installed. pbuilder-satisfydepends-dummy depends on golang-github-armon-go-metrics-dev; however: Package golang-github-armon-go-metrics-dev is not installed. pbuilder-satisfydepends-dummy depends on golang-github-hashicorp-go-hclog-dev (>= 0.9.2~); however: Package golang-github-hashicorp-go-hclog-dev is not installed. pbuilder-satisfydepends-dummy depends on golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~); however: Package golang-github-hashicorp-go-msgpack-dev is not installed. Setting up pbuilder-satisfydepends-dummy (0.invalid.0) ... Reading package lists... Building dependency tree... Reading state information... Initializing package states... Writing extended state information... Building tag database... pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) The following NEW packages will be installed: autoconf{a} automake{a} autopoint{a} autotools-dev{a} bsdextrautils{a} debhelper{a} dh-autoreconf{a} dh-golang{a} dh-strip-nondeterminism{a} dwz{a} file{a} gettext{a} gettext-base{a} golang-1.15-go{a} golang-1.15-src{a} golang-any{a} golang-github-alecthomas-units-dev{a} golang-github-armon-go-metrics-dev{a} golang-github-beorn7-perks-dev{a} golang-github-cespare-xxhash-dev{a} golang-github-circonus-labs-circonus-gometrics-dev{a} golang-github-circonus-labs-circonusllhist-dev{a} golang-github-creack-pty-dev{a} golang-github-datadog-datadog-go-dev{a} golang-github-davecgh-go-spew-dev{a} golang-github-dgrijalva-jwt-go-dev{a} golang-github-fatih-color-dev{a} golang-github-go-kit-kit-dev{a} golang-github-go-logfmt-logfmt-dev{a} golang-github-go-stack-stack-dev{a} golang-github-golang-mock-dev{a} golang-github-google-go-cmp-dev{a} golang-github-hashicorp-go-cleanhttp-dev{a} golang-github-hashicorp-go-hclog-dev{a} golang-github-hashicorp-go-immutable-radix-dev{a} golang-github-hashicorp-go-msgpack-dev{a} golang-github-hashicorp-go-retryablehttp-dev{a} golang-github-hashicorp-go-uuid-dev{a} golang-github-hashicorp-golang-lru-dev{a} golang-github-jpillora-backoff-dev{a} golang-github-json-iterator-go-dev{a} golang-github-julienschmidt-httprouter-dev{a} golang-github-kr-pretty-dev{a} golang-github-kr-text-dev{a} golang-github-mattn-go-colorable-dev{a} golang-github-mattn-go-isatty-dev{a} golang-github-modern-go-concurrent-dev{a} golang-github-modern-go-reflect2-dev{a} golang-github-mwitkow-go-conntrack-dev{a} golang-github-opentracing-opentracing-go-dev{a} golang-github-pascaldekloe-goe-dev{a} golang-github-pkg-errors-dev{a} golang-github-pmezard-go-difflib-dev{a} golang-github-prometheus-client-golang-dev{a} golang-github-prometheus-client-model-dev{a} golang-github-prometheus-common-dev{a} golang-github-prometheus-procfs-dev{a} golang-github-sirupsen-logrus-dev{a} golang-github-stretchr-objx-dev{a} golang-github-stretchr-testify-dev{a} golang-github-tv42-httpunix-dev{a} golang-github-vmihailenco-tagparser-dev{a} golang-glog-dev{a} golang-go{a} golang-golang-x-net-dev{a} golang-golang-x-oauth2-dev{a} golang-golang-x-oauth2-google-dev{a} golang-golang-x-sync-dev{a} golang-golang-x-sys-dev{a} golang-golang-x-text-dev{a} golang-golang-x-time-dev{a} golang-golang-x-xerrors-dev{a} golang-google-cloud-compute-metadata-dev{a} golang-google-genproto-dev{a} golang-google-grpc-dev{a} golang-gopkg-alecthomas-kingpin.v2-dev{a} golang-gopkg-check.v1-dev{a} golang-gopkg-mgo.v2-dev{a} golang-gopkg-tomb.v2-dev{a} golang-gopkg-vmihailenco-msgpack.v2-dev{a} golang-gopkg-yaml.v2-dev{a} golang-gopkg-yaml.v3-dev{a} golang-goprotobuf-dev{a} golang-protobuf-extensions-dev{a} golang-src{a} groff-base{a} intltool-debian{a} libarchive-zip-perl{a} libdebhelper-perl{a} libelf1{a} libfile-stripnondeterminism-perl{a} libicu67{a} libmagic-mgc{a} libmagic1{a} libpipeline1{a} libprotobuf-dev{a} libprotobuf-lite23{a} libprotobuf23{a} libprotoc23{a} libsasl2-2{a} libsasl2-dev{a} libsasl2-modules-db{a} libsigsegv2{a} libsub-override-perl{a} libtool{a} libuchardet0{a} libxml2{a} m4{a} man-db{a} po-debconf{a} protobuf-compiler{a} sensible-utils{a} zlib1g-dev{a} The following packages are RECOMMENDED but will NOT be installed: curl libarchive-cpio-perl libltdl-dev libmail-sendmail-perl libsasl2-modules lynx pkg-config pkgconf wget 0 packages upgraded, 113 newly installed, 0 to remove and 0 not upgraded. Need to get 95.8 MB of archives. After unpacking 538 MB will be used. Writing extended state information... Get: 1 http://deb.debian.org/debian bullseye/main i386 bsdextrautils i386 2.36.1-8+deb11u1 [149 kB] Get: 2 http://deb.debian.org/debian bullseye/main i386 libuchardet0 i386 0.0.7-1 [67.9 kB] Get: 3 http://deb.debian.org/debian bullseye/main i386 groff-base i386 1.22.4-6 [952 kB] Get: 4 http://deb.debian.org/debian bullseye/main i386 libpipeline1 i386 1.5.3-1 [36.8 kB] Get: 5 http://deb.debian.org/debian bullseye/main i386 man-db i386 2.9.4-2 [1367 kB] Get: 6 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-sys-dev all 0.0~git20210124.22da62e-1 [308 kB] Get: 7 http://deb.debian.org/debian bullseye/main i386 golang-github-sirupsen-logrus-dev all 1.7.0-2 [45.3 kB] Get: 8 http://deb.debian.org/debian bullseye/main i386 sensible-utils all 0.0.14 [14.8 kB] Get: 9 http://deb.debian.org/debian bullseye/main i386 libmagic-mgc i386 1:5.39-3+deb11u1 [273 kB] Get: 10 http://deb.debian.org/debian bullseye/main i386 libmagic1 i386 1:5.39-3+deb11u1 [135 kB] Get: 11 http://deb.debian.org/debian bullseye/main i386 file i386 1:5.39-3+deb11u1 [69.2 kB] Get: 12 http://deb.debian.org/debian bullseye/main i386 gettext-base i386 0.21-4 [176 kB] Get: 13 http://deb.debian.org/debian bullseye/main i386 libsigsegv2 i386 2.13-1 [35.1 kB] Get: 14 http://deb.debian.org/debian bullseye/main i386 m4 i386 1.4.18-5 [206 kB] Get: 15 http://deb.debian.org/debian bullseye/main i386 autoconf all 2.69-14 [313 kB] Get: 16 http://deb.debian.org/debian bullseye/main i386 autotools-dev all 20180224.1+nmu1 [77.1 kB] Get: 17 http://deb.debian.org/debian bullseye/main i386 automake all 1:1.16.3-2 [814 kB] Get: 18 http://deb.debian.org/debian bullseye/main i386 autopoint all 0.21-4 [510 kB] Get: 19 http://deb.debian.org/debian bullseye/main i386 libdebhelper-perl all 13.3.4 [189 kB] Get: 20 http://deb.debian.org/debian bullseye/main i386 libtool all 2.4.6-15 [513 kB] Get: 21 http://deb.debian.org/debian bullseye/main i386 dh-autoreconf all 20 [17.1 kB] Get: 22 http://deb.debian.org/debian bullseye/main i386 libarchive-zip-perl all 1.68-1 [104 kB] Get: 23 http://deb.debian.org/debian bullseye/main i386 libsub-override-perl all 0.09-2 [10.2 kB] Get: 24 http://deb.debian.org/debian bullseye/main i386 libfile-stripnondeterminism-perl all 1.12.0-1 [26.3 kB] Get: 25 http://deb.debian.org/debian bullseye/main i386 dh-strip-nondeterminism all 1.12.0-1 [15.4 kB] Get: 26 http://deb.debian.org/debian bullseye/main i386 libelf1 i386 0.183-1 [171 kB] Get: 27 http://deb.debian.org/debian bullseye/main i386 dwz i386 0.13+20210201-1 [179 kB] Get: 28 http://deb.debian.org/debian bullseye/main i386 libicu67 i386 67.1-7 [8775 kB] Get: 29 http://deb.debian.org/debian bullseye/main i386 libxml2 i386 2.9.10+dfsg-6.7+deb11u4 [728 kB] Get: 30 http://deb.debian.org/debian bullseye/main i386 gettext i386 0.21-4 [1322 kB] Get: 31 http://deb.debian.org/debian bullseye/main i386 intltool-debian all 0.35.0+20060710.5 [26.8 kB] Get: 32 http://deb.debian.org/debian bullseye/main i386 po-debconf all 1.0.21+nmu1 [248 kB] Get: 33 http://deb.debian.org/debian bullseye/main i386 debhelper all 13.3.4 [1049 kB] Get: 34 http://deb.debian.org/debian bullseye/main i386 dh-golang all 1.51 [24.6 kB] Get: 35 http://deb.debian.org/debian bullseye/main i386 golang-1.15-src i386 1.15.15-1~deb11u4 [13.9 MB] Get: 36 http://deb.debian.org/debian bullseye/main i386 golang-1.15-go i386 1.15.15-1~deb11u4 [46.3 MB] Get: 37 http://deb.debian.org/debian bullseye/main i386 golang-src i386 2:1.15~1 [4856 B] Get: 38 http://deb.debian.org/debian bullseye/main i386 golang-go i386 2:1.15~1 [24.0 kB] Get: 39 http://deb.debian.org/debian bullseye/main i386 golang-any i386 2:1.15~1 [4980 B] Get: 40 http://deb.debian.org/debian bullseye/main i386 golang-github-alecthomas-units-dev all 0.0~git20201120.1786d5e-1 [7504 B] Get: 41 http://deb.debian.org/debian bullseye/main i386 golang-github-circonus-labs-circonusllhist-dev all 0.0~git20191022.ec08cde-1 [11.1 kB] Get: 42 http://deb.debian.org/debian bullseye/main i386 golang-github-hashicorp-go-cleanhttp-dev all 0.5.2-1 [10.8 kB] Get: 43 http://deb.debian.org/debian bullseye/main i386 golang-github-mattn-go-isatty-dev all 0.0.12-1 [6472 B] Get: 44 http://deb.debian.org/debian bullseye/main i386 golang-github-mattn-go-colorable-dev all 0.1.7-1 [9936 B] Get: 45 http://deb.debian.org/debian bullseye/main i386 golang-github-fatih-color-dev all 1.7.0-1 [11.4 kB] Get: 46 http://deb.debian.org/debian bullseye/main i386 golang-github-davecgh-go-spew-dev all 1.1.1-2 [29.7 kB] Get: 47 http://deb.debian.org/debian bullseye/main i386 golang-github-pmezard-go-difflib-dev all 1.0.0-3 [12.3 kB] Get: 48 http://deb.debian.org/debian bullseye/main i386 golang-github-stretchr-objx-dev all 0.3.0-1 [25.4 kB] Get: 49 http://deb.debian.org/debian bullseye/main i386 golang-gopkg-yaml.v3-dev all 3.0.0~git20200121.a6ecf24-3 [70.5 kB] Get: 50 http://deb.debian.org/debian bullseye/main i386 golang-github-stretchr-testify-dev all 1.6.1-2 [60.4 kB] Get: 51 http://deb.debian.org/debian bullseye/main i386 golang-github-hashicorp-go-hclog-dev all 0.11.0-1 [17.9 kB] Get: 52 http://deb.debian.org/debian bullseye/main i386 golang-github-hashicorp-go-retryablehttp-dev all 0.6.4-2 [17.5 kB] Get: 53 http://deb.debian.org/debian bullseye/main i386 golang-github-pkg-errors-dev all 0.9.1-1 [13.0 kB] Get: 54 http://deb.debian.org/debian bullseye/main i386 golang-github-tv42-httpunix-dev all 0.0~git20150427.b75d861-2 [3744 B] Get: 55 http://deb.debian.org/debian bullseye/main i386 golang-github-circonus-labs-circonus-gometrics-dev all 2.3.1-3 [65.9 kB] Get: 56 http://deb.debian.org/debian bullseye/main i386 golang-github-datadog-datadog-go-dev all 2.1.0-3 [14.9 kB] Get: 57 http://deb.debian.org/debian bullseye/main i386 golang-github-hashicorp-go-uuid-dev all 1.0.2-1 [8764 B] Get: 58 http://deb.debian.org/debian bullseye/main i386 golang-github-hashicorp-golang-lru-dev all 0.5.4-2 [14.7 kB] Get: 59 http://deb.debian.org/debian bullseye/main i386 golang-github-hashicorp-go-immutable-radix-dev all 1.3.0-1 [25.0 kB] Get: 60 http://deb.debian.org/debian bullseye/main i386 golang-github-pascaldekloe-goe-dev all 0.1.0-2 [21.7 kB] Get: 61 http://deb.debian.org/debian bullseye/main i386 golang-github-beorn7-perks-dev all 1.0.1-1 [12.1 kB] Get: 62 http://deb.debian.org/debian bullseye/main i386 golang-github-cespare-xxhash-dev all 2.1.1-1 [8748 B] Get: 63 http://deb.debian.org/debian bullseye/main i386 golang-github-modern-go-concurrent-dev all 1.0.3-1.1 [4668 B] Get: 64 http://deb.debian.org/debian bullseye/main i386 golang-github-modern-go-reflect2-dev all 1.0.1-1 [10.6 kB] Get: 65 http://deb.debian.org/debian bullseye/main i386 golang-github-json-iterator-go-dev all 1.1.10-2 [69.4 kB] Get: 66 http://deb.debian.org/debian bullseye/main i386 zlib1g-dev i386 1:1.2.11.dfsg-2+deb11u2 [194 kB] Get: 67 http://deb.debian.org/debian bullseye/main i386 libprotobuf23 i386 3.12.4-1+deb11u1 [976 kB] Get: 68 http://deb.debian.org/debian bullseye/main i386 libprotobuf-lite23 i386 3.12.4-1+deb11u1 [263 kB] Get: 69 http://deb.debian.org/debian bullseye/main i386 libprotobuf-dev i386 3.12.4-1+deb11u1 [1337 kB] Get: 70 http://deb.debian.org/debian bullseye/main i386 libprotoc23 i386 3.12.4-1+deb11u1 [872 kB] Get: 71 http://deb.debian.org/debian bullseye/main i386 protobuf-compiler i386 3.12.4-1+deb11u1 [75.4 kB] Get: 72 http://deb.debian.org/debian bullseye/main i386 golang-goprotobuf-dev i386 1.3.4-2+b6 [1442 kB] Get: 73 http://deb.debian.org/debian bullseye/main i386 golang-github-prometheus-client-model-dev all 0.2.0-2 [8476 B] Get: 74 http://deb.debian.org/debian bullseye/main i386 golang-github-dgrijalva-jwt-go-dev all 3.2.0-3 [33.6 kB] Get: 75 http://deb.debian.org/debian bullseye/main i386 golang-github-go-logfmt-logfmt-dev all 0.5.0-2 [13.4 kB] Get: 76 http://deb.debian.org/debian bullseye/main i386 golang-github-go-stack-stack-dev all 1.8.0-1 [8696 B] Get: 77 http://deb.debian.org/debian bullseye/main i386 golang-github-opentracing-opentracing-go-dev all 1.2.0-1 [29.4 kB] Get: 78 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-text-dev all 0.3.6-1 [3857 kB] Get: 79 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-net-dev all 1:0.0+git20210119.5f4716e+dfsg-4 [659 kB] Get: 80 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-time-dev all 0.0+git20200630.3af7569-1 [10.1 kB] Get: 81 http://deb.debian.org/debian bullseye/main i386 golang-github-golang-mock-dev all 1.3.1-2 [35.1 kB] Get: 82 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-xerrors-dev all 0.0~git20191204.9bdfabe-1 [14.2 kB] Get: 83 http://deb.debian.org/debian bullseye/main i386 golang-github-google-go-cmp-dev all 0.5.4-1 [83.7 kB] Get: 84 http://deb.debian.org/debian bullseye/main i386 golang-glog-dev all 0.0~git20160126.23def4e-3 [17.3 kB] Get: 85 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-oauth2-dev all 0.0~git20190604.0f29369-2 [31.9 kB] Get: 86 http://deb.debian.org/debian bullseye/main i386 golang-google-cloud-compute-metadata-dev all 0.56.0-1 [34.0 kB] Get: 87 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-oauth2-google-dev all 0.0~git20190604.0f29369-2 [13.2 kB] Get: 88 http://deb.debian.org/debian bullseye/main i386 golang-google-genproto-dev all 0.0~git20200413.b5235f6-1 [4037 kB] Get: 89 http://deb.debian.org/debian bullseye/main i386 golang-google-grpc-dev all 1.27.1-1 [478 kB] Get: 90 http://deb.debian.org/debian bullseye/main i386 golang-github-go-kit-kit-dev all 0.6.0-3 [103 kB] Get: 91 http://deb.debian.org/debian bullseye/main i386 golang-github-julienschmidt-httprouter-dev all 1.3.0-1 [19.3 kB] Get: 92 http://deb.debian.org/debian bullseye/main i386 golang-github-jpillora-backoff-dev all 1.0.0-1.1 [3712 B] Get: 93 http://deb.debian.org/debian bullseye/main i386 golang-github-mwitkow-go-conntrack-dev all 0.0~git20190716.2f06839-2 [14.6 kB] Get: 94 http://deb.debian.org/debian bullseye/main i386 golang-gopkg-alecthomas-kingpin.v2-dev all 2.2.6-3 [42.8 kB] Get: 95 http://deb.debian.org/debian bullseye/main i386 golang-gopkg-yaml.v2-dev all 2.4.0-1 [61.3 kB] Get: 96 http://deb.debian.org/debian bullseye/main i386 golang-protobuf-extensions-dev all 1.0.1-1 [29.6 kB] Get: 97 http://deb.debian.org/debian bullseye/main i386 golang-github-prometheus-common-dev all 0.15.0-2 [102 kB] Get: 98 http://deb.debian.org/debian bullseye/main i386 golang-golang-x-sync-dev all 0.0~git20210220.036812b-1 [19.1 kB] Get: 99 http://deb.debian.org/debian bullseye/main i386 golang-github-prometheus-procfs-dev all 0.3.0-2 [110 kB] Get: 100 http://deb.debian.org/debian bullseye/main i386 golang-github-prometheus-client-golang-dev all 1.9.0-2 [134 kB] Get: 101 http://deb.debian.org/debian bullseye/main i386 golang-github-armon-go-metrics-dev all 0.3.4-1 [30.5 kB] Get: 102 http://deb.debian.org/debian bullseye/main i386 golang-github-creack-pty-dev all 1.1.11-1 [8380 B] Get: 103 http://deb.debian.org/debian bullseye/main i386 golang-github-vmihailenco-tagparser-dev all 0.1.1-2 [4440 B] Get: 104 http://deb.debian.org/debian bullseye/main i386 golang-gopkg-vmihailenco-msgpack.v2-dev all 4.3.1-2 [27.5 kB] Get: 105 http://deb.debian.org/debian bullseye/main i386 golang-gopkg-tomb.v2-dev all 0.0~git20161208.d5d1b58-3 [6840 B] Get: 106 http://deb.debian.org/debian bullseye/main i386 libsasl2-modules-db i386 2.1.27+dfsg-2.1+deb11u1 [70.1 kB] Get: 107 http://deb.debian.org/debian bullseye/main i386 libsasl2-2 i386 2.1.27+dfsg-2.1+deb11u1 [110 kB] Get: 108 http://deb.debian.org/debian bullseye/main i386 libsasl2-dev i386 2.1.27+dfsg-2.1+deb11u1 [295 kB] Get: 109 http://deb.debian.org/debian bullseye/main i386 golang-github-kr-text-dev all 0.2.0-1 [11.1 kB] Get: 110 http://deb.debian.org/debian bullseye/main i386 golang-github-kr-pretty-dev all 0.2.1+git20200831.59b4212-1 [14.6 kB] Get: 111 http://deb.debian.org/debian bullseye/main i386 golang-gopkg-check.v1-dev all 0.0+git20200902.038fdea-1 [32.1 kB] Get: 112 http://deb.debian.org/debian bullseye/main i386 golang-gopkg-mgo.v2-dev all 2016.08.01-7 [316 kB] Get: 113 http://deb.debian.org/debian bullseye/main i386 golang-github-hashicorp-go-msgpack-dev all 0.5.5-1 [43.3 kB] Fetched 95.8 MB in 3s (31.4 MB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package bsdextrautils. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 17763 files and directories currently installed.) Preparing to unpack .../000-bsdextrautils_2.36.1-8+deb11u1_i386.deb ... Unpacking bsdextrautils (2.36.1-8+deb11u1) ... Selecting previously unselected package libuchardet0:i386. Preparing to unpack .../001-libuchardet0_0.0.7-1_i386.deb ... Unpacking libuchardet0:i386 (0.0.7-1) ... Selecting previously unselected package groff-base. Preparing to unpack .../002-groff-base_1.22.4-6_i386.deb ... Unpacking groff-base (1.22.4-6) ... Selecting previously unselected package libpipeline1:i386. Preparing to unpack .../003-libpipeline1_1.5.3-1_i386.deb ... Unpacking libpipeline1:i386 (1.5.3-1) ... Selecting previously unselected package man-db. Preparing to unpack .../004-man-db_2.9.4-2_i386.deb ... Unpacking man-db (2.9.4-2) ... Selecting previously unselected package golang-golang-x-sys-dev. Preparing to unpack .../005-golang-golang-x-sys-dev_0.0~git20210124.22da62e-1_all.deb ... Unpacking golang-golang-x-sys-dev (0.0~git20210124.22da62e-1) ... Selecting previously unselected package golang-github-sirupsen-logrus-dev. Preparing to unpack .../006-golang-github-sirupsen-logrus-dev_1.7.0-2_all.deb ... Unpacking golang-github-sirupsen-logrus-dev (1.7.0-2) ... Selecting previously unselected package sensible-utils. Preparing to unpack .../007-sensible-utils_0.0.14_all.deb ... Unpacking sensible-utils (0.0.14) ... Selecting previously unselected package libmagic-mgc. Preparing to unpack .../008-libmagic-mgc_1%3a5.39-3+deb11u1_i386.deb ... Unpacking libmagic-mgc (1:5.39-3+deb11u1) ... Selecting previously unselected package libmagic1:i386. Preparing to unpack .../009-libmagic1_1%3a5.39-3+deb11u1_i386.deb ... Unpacking libmagic1:i386 (1:5.39-3+deb11u1) ... Selecting previously unselected package file. Preparing to unpack .../010-file_1%3a5.39-3+deb11u1_i386.deb ... Unpacking file (1:5.39-3+deb11u1) ... Selecting previously unselected package gettext-base. Preparing to unpack .../011-gettext-base_0.21-4_i386.deb ... Unpacking gettext-base (0.21-4) ... Selecting previously unselected package libsigsegv2:i386. Preparing to unpack .../012-libsigsegv2_2.13-1_i386.deb ... Unpacking libsigsegv2:i386 (2.13-1) ... Selecting previously unselected package m4. Preparing to unpack .../013-m4_1.4.18-5_i386.deb ... Unpacking m4 (1.4.18-5) ... Selecting previously unselected package autoconf. Preparing to unpack .../014-autoconf_2.69-14_all.deb ... Unpacking autoconf (2.69-14) ... Selecting previously unselected package autotools-dev. Preparing to unpack .../015-autotools-dev_20180224.1+nmu1_all.deb ... Unpacking autotools-dev (20180224.1+nmu1) ... Selecting previously unselected package automake. Preparing to unpack .../016-automake_1%3a1.16.3-2_all.deb ... Unpacking automake (1:1.16.3-2) ... Selecting previously unselected package autopoint. Preparing to unpack .../017-autopoint_0.21-4_all.deb ... Unpacking autopoint (0.21-4) ... Selecting previously unselected package libdebhelper-perl. Preparing to unpack .../018-libdebhelper-perl_13.3.4_all.deb ... Unpacking libdebhelper-perl (13.3.4) ... Selecting previously unselected package libtool. Preparing to unpack .../019-libtool_2.4.6-15_all.deb ... Unpacking libtool (2.4.6-15) ... Selecting previously unselected package dh-autoreconf. Preparing to unpack .../020-dh-autoreconf_20_all.deb ... Unpacking dh-autoreconf (20) ... Selecting previously unselected package libarchive-zip-perl. Preparing to unpack .../021-libarchive-zip-perl_1.68-1_all.deb ... Unpacking libarchive-zip-perl (1.68-1) ... Selecting previously unselected package libsub-override-perl. Preparing to unpack .../022-libsub-override-perl_0.09-2_all.deb ... Unpacking libsub-override-perl (0.09-2) ... Selecting previously unselected package libfile-stripnondeterminism-perl. Preparing to unpack .../023-libfile-stripnondeterminism-perl_1.12.0-1_all.deb ... Unpacking libfile-stripnondeterminism-perl (1.12.0-1) ... Selecting previously unselected package dh-strip-nondeterminism. Preparing to unpack .../024-dh-strip-nondeterminism_1.12.0-1_all.deb ... Unpacking dh-strip-nondeterminism (1.12.0-1) ... Selecting previously unselected package libelf1:i386. Preparing to unpack .../025-libelf1_0.183-1_i386.deb ... Unpacking libelf1:i386 (0.183-1) ... Selecting previously unselected package dwz. Preparing to unpack .../026-dwz_0.13+20210201-1_i386.deb ... Unpacking dwz (0.13+20210201-1) ... Selecting previously unselected package libicu67:i386. Preparing to unpack .../027-libicu67_67.1-7_i386.deb ... Unpacking libicu67:i386 (67.1-7) ... Selecting previously unselected package libxml2:i386. Preparing to unpack .../028-libxml2_2.9.10+dfsg-6.7+deb11u4_i386.deb ... Unpacking libxml2:i386 (2.9.10+dfsg-6.7+deb11u4) ... Selecting previously unselected package gettext. Preparing to unpack .../029-gettext_0.21-4_i386.deb ... Unpacking gettext (0.21-4) ... Selecting previously unselected package intltool-debian. Preparing to unpack .../030-intltool-debian_0.35.0+20060710.5_all.deb ... Unpacking intltool-debian (0.35.0+20060710.5) ... Selecting previously unselected package po-debconf. Preparing to unpack .../031-po-debconf_1.0.21+nmu1_all.deb ... Unpacking po-debconf (1.0.21+nmu1) ... Selecting previously unselected package debhelper. Preparing to unpack .../032-debhelper_13.3.4_all.deb ... Unpacking debhelper (13.3.4) ... Selecting previously unselected package dh-golang. Preparing to unpack .../033-dh-golang_1.51_all.deb ... Unpacking dh-golang (1.51) ... Selecting previously unselected package golang-1.15-src. Preparing to unpack .../034-golang-1.15-src_1.15.15-1~deb11u4_i386.deb ... Unpacking golang-1.15-src (1.15.15-1~deb11u4) ... Selecting previously unselected package golang-1.15-go. Preparing to unpack .../035-golang-1.15-go_1.15.15-1~deb11u4_i386.deb ... Unpacking golang-1.15-go (1.15.15-1~deb11u4) ... Selecting previously unselected package golang-src:i386. Preparing to unpack .../036-golang-src_2%3a1.15~1_i386.deb ... Unpacking golang-src:i386 (2:1.15~1) ... Selecting previously unselected package golang-go. Preparing to unpack .../037-golang-go_2%3a1.15~1_i386.deb ... Unpacking golang-go (2:1.15~1) ... Selecting previously unselected package golang-any:i386. Preparing to unpack .../038-golang-any_2%3a1.15~1_i386.deb ... Unpacking golang-any:i386 (2:1.15~1) ... Selecting previously unselected package golang-github-alecthomas-units-dev. Preparing to unpack .../039-golang-github-alecthomas-units-dev_0.0~git20201120.1786d5e-1_all.deb ... Unpacking golang-github-alecthomas-units-dev (0.0~git20201120.1786d5e-1) ... Selecting previously unselected package golang-github-circonus-labs-circonusllhist-dev. Preparing to unpack .../040-golang-github-circonus-labs-circonusllhist-dev_0.0~git20191022.ec08cde-1_all.deb ... Unpacking golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ... Selecting previously unselected package golang-github-hashicorp-go-cleanhttp-dev. Preparing to unpack .../041-golang-github-hashicorp-go-cleanhttp-dev_0.5.2-1_all.deb ... Unpacking golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ... Selecting previously unselected package golang-github-mattn-go-isatty-dev. Preparing to unpack .../042-golang-github-mattn-go-isatty-dev_0.0.12-1_all.deb ... Unpacking golang-github-mattn-go-isatty-dev (0.0.12-1) ... Selecting previously unselected package golang-github-mattn-go-colorable-dev. Preparing to unpack .../043-golang-github-mattn-go-colorable-dev_0.1.7-1_all.deb ... Unpacking golang-github-mattn-go-colorable-dev (0.1.7-1) ... Selecting previously unselected package golang-github-fatih-color-dev. Preparing to unpack .../044-golang-github-fatih-color-dev_1.7.0-1_all.deb ... Unpacking golang-github-fatih-color-dev (1.7.0-1) ... Selecting previously unselected package golang-github-davecgh-go-spew-dev. Preparing to unpack .../045-golang-github-davecgh-go-spew-dev_1.1.1-2_all.deb ... Unpacking golang-github-davecgh-go-spew-dev (1.1.1-2) ... Selecting previously unselected package golang-github-pmezard-go-difflib-dev. Preparing to unpack .../046-golang-github-pmezard-go-difflib-dev_1.0.0-3_all.deb ... Unpacking golang-github-pmezard-go-difflib-dev (1.0.0-3) ... Selecting previously unselected package golang-github-stretchr-objx-dev. Preparing to unpack .../047-golang-github-stretchr-objx-dev_0.3.0-1_all.deb ... Unpacking golang-github-stretchr-objx-dev (0.3.0-1) ... Selecting previously unselected package golang-gopkg-yaml.v3-dev. Preparing to unpack .../048-golang-gopkg-yaml.v3-dev_3.0.0~git20200121.a6ecf24-3_all.deb ... Unpacking golang-gopkg-yaml.v3-dev (3.0.0~git20200121.a6ecf24-3) ... Selecting previously unselected package golang-github-stretchr-testify-dev. Preparing to unpack .../049-golang-github-stretchr-testify-dev_1.6.1-2_all.deb ... Unpacking golang-github-stretchr-testify-dev (1.6.1-2) ... Selecting previously unselected package golang-github-hashicorp-go-hclog-dev. Preparing to unpack .../050-golang-github-hashicorp-go-hclog-dev_0.11.0-1_all.deb ... Unpacking golang-github-hashicorp-go-hclog-dev (0.11.0-1) ... Selecting previously unselected package golang-github-hashicorp-go-retryablehttp-dev. Preparing to unpack .../051-golang-github-hashicorp-go-retryablehttp-dev_0.6.4-2_all.deb ... Unpacking golang-github-hashicorp-go-retryablehttp-dev (0.6.4-2) ... Selecting previously unselected package golang-github-pkg-errors-dev. Preparing to unpack .../052-golang-github-pkg-errors-dev_0.9.1-1_all.deb ... Unpacking golang-github-pkg-errors-dev (0.9.1-1) ... Selecting previously unselected package golang-github-tv42-httpunix-dev. Preparing to unpack .../053-golang-github-tv42-httpunix-dev_0.0~git20150427.b75d861-2_all.deb ... Unpacking golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-2) ... Selecting previously unselected package golang-github-circonus-labs-circonus-gometrics-dev. Preparing to unpack .../054-golang-github-circonus-labs-circonus-gometrics-dev_2.3.1-3_all.deb ... Unpacking golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-3) ... Selecting previously unselected package golang-github-datadog-datadog-go-dev. Preparing to unpack .../055-golang-github-datadog-datadog-go-dev_2.1.0-3_all.deb ... Unpacking golang-github-datadog-datadog-go-dev (2.1.0-3) ... Selecting previously unselected package golang-github-hashicorp-go-uuid-dev. Preparing to unpack .../056-golang-github-hashicorp-go-uuid-dev_1.0.2-1_all.deb ... Unpacking golang-github-hashicorp-go-uuid-dev (1.0.2-1) ... Selecting previously unselected package golang-github-hashicorp-golang-lru-dev. Preparing to unpack .../057-golang-github-hashicorp-golang-lru-dev_0.5.4-2_all.deb ... Unpacking golang-github-hashicorp-golang-lru-dev (0.5.4-2) ... Selecting previously unselected package golang-github-hashicorp-go-immutable-radix-dev. Preparing to unpack .../058-golang-github-hashicorp-go-immutable-radix-dev_1.3.0-1_all.deb ... Unpacking golang-github-hashicorp-go-immutable-radix-dev (1.3.0-1) ... Selecting previously unselected package golang-github-pascaldekloe-goe-dev. Preparing to unpack .../059-golang-github-pascaldekloe-goe-dev_0.1.0-2_all.deb ... Unpacking golang-github-pascaldekloe-goe-dev (0.1.0-2) ... Selecting previously unselected package golang-github-beorn7-perks-dev. Preparing to unpack .../060-golang-github-beorn7-perks-dev_1.0.1-1_all.deb ... Unpacking golang-github-beorn7-perks-dev (1.0.1-1) ... Selecting previously unselected package golang-github-cespare-xxhash-dev. Preparing to unpack .../061-golang-github-cespare-xxhash-dev_2.1.1-1_all.deb ... Unpacking golang-github-cespare-xxhash-dev (2.1.1-1) ... Selecting previously unselected package golang-github-modern-go-concurrent-dev. Preparing to unpack .../062-golang-github-modern-go-concurrent-dev_1.0.3-1.1_all.deb ... Unpacking golang-github-modern-go-concurrent-dev (1.0.3-1.1) ... Selecting previously unselected package golang-github-modern-go-reflect2-dev. Preparing to unpack .../063-golang-github-modern-go-reflect2-dev_1.0.1-1_all.deb ... Unpacking golang-github-modern-go-reflect2-dev (1.0.1-1) ... Selecting previously unselected package golang-github-json-iterator-go-dev. Preparing to unpack .../064-golang-github-json-iterator-go-dev_1.1.10-2_all.deb ... Unpacking golang-github-json-iterator-go-dev (1.1.10-2) ... Selecting previously unselected package zlib1g-dev:i386. Preparing to unpack .../065-zlib1g-dev_1%3a1.2.11.dfsg-2+deb11u2_i386.deb ... Unpacking zlib1g-dev:i386 (1:1.2.11.dfsg-2+deb11u2) ... Selecting previously unselected package libprotobuf23:i386. Preparing to unpack .../066-libprotobuf23_3.12.4-1+deb11u1_i386.deb ... Unpacking libprotobuf23:i386 (3.12.4-1+deb11u1) ... Selecting previously unselected package libprotobuf-lite23:i386. Preparing to unpack .../067-libprotobuf-lite23_3.12.4-1+deb11u1_i386.deb ... Unpacking libprotobuf-lite23:i386 (3.12.4-1+deb11u1) ... Selecting previously unselected package libprotobuf-dev:i386. Preparing to unpack .../068-libprotobuf-dev_3.12.4-1+deb11u1_i386.deb ... Unpacking libprotobuf-dev:i386 (3.12.4-1+deb11u1) ... Selecting previously unselected package libprotoc23:i386. Preparing to unpack .../069-libprotoc23_3.12.4-1+deb11u1_i386.deb ... Unpacking libprotoc23:i386 (3.12.4-1+deb11u1) ... Selecting previously unselected package protobuf-compiler. Preparing to unpack .../070-protobuf-compiler_3.12.4-1+deb11u1_i386.deb ... Unpacking protobuf-compiler (3.12.4-1+deb11u1) ... Selecting previously unselected package golang-goprotobuf-dev. Preparing to unpack .../071-golang-goprotobuf-dev_1.3.4-2+b6_i386.deb ... Unpacking golang-goprotobuf-dev (1.3.4-2+b6) ... Selecting previously unselected package golang-github-prometheus-client-model-dev. Preparing to unpack .../072-golang-github-prometheus-client-model-dev_0.2.0-2_all.deb ... Unpacking golang-github-prometheus-client-model-dev (0.2.0-2) ... Selecting previously unselected package golang-github-dgrijalva-jwt-go-dev. Preparing to unpack .../073-golang-github-dgrijalva-jwt-go-dev_3.2.0-3_all.deb ... Unpacking golang-github-dgrijalva-jwt-go-dev (3.2.0-3) ... Selecting previously unselected package golang-github-go-logfmt-logfmt-dev. Preparing to unpack .../074-golang-github-go-logfmt-logfmt-dev_0.5.0-2_all.deb ... Unpacking golang-github-go-logfmt-logfmt-dev (0.5.0-2) ... Selecting previously unselected package golang-github-go-stack-stack-dev. Preparing to unpack .../075-golang-github-go-stack-stack-dev_1.8.0-1_all.deb ... Unpacking golang-github-go-stack-stack-dev (1.8.0-1) ... Selecting previously unselected package golang-github-opentracing-opentracing-go-dev. Preparing to unpack .../076-golang-github-opentracing-opentracing-go-dev_1.2.0-1_all.deb ... Unpacking golang-github-opentracing-opentracing-go-dev (1.2.0-1) ... Selecting previously unselected package golang-golang-x-text-dev. Preparing to unpack .../077-golang-golang-x-text-dev_0.3.6-1_all.deb ... Unpacking golang-golang-x-text-dev (0.3.6-1) ... Selecting previously unselected package golang-golang-x-net-dev. Preparing to unpack .../078-golang-golang-x-net-dev_1%3a0.0+git20210119.5f4716e+dfsg-4_all.deb ... Unpacking golang-golang-x-net-dev (1:0.0+git20210119.5f4716e+dfsg-4) ... Selecting previously unselected package golang-golang-x-time-dev. Preparing to unpack .../079-golang-golang-x-time-dev_0.0+git20200630.3af7569-1_all.deb ... Unpacking golang-golang-x-time-dev (0.0+git20200630.3af7569-1) ... Selecting previously unselected package golang-github-golang-mock-dev. Preparing to unpack .../080-golang-github-golang-mock-dev_1.3.1-2_all.deb ... Unpacking golang-github-golang-mock-dev (1.3.1-2) ... Selecting previously unselected package golang-golang-x-xerrors-dev. Preparing to unpack .../081-golang-golang-x-xerrors-dev_0.0~git20191204.9bdfabe-1_all.deb ... Unpacking golang-golang-x-xerrors-dev (0.0~git20191204.9bdfabe-1) ... Selecting previously unselected package golang-github-google-go-cmp-dev. Preparing to unpack .../082-golang-github-google-go-cmp-dev_0.5.4-1_all.deb ... Unpacking golang-github-google-go-cmp-dev (0.5.4-1) ... Selecting previously unselected package golang-glog-dev. Preparing to unpack .../083-golang-glog-dev_0.0~git20160126.23def4e-3_all.deb ... Unpacking golang-glog-dev (0.0~git20160126.23def4e-3) ... Selecting previously unselected package golang-golang-x-oauth2-dev. Preparing to unpack .../084-golang-golang-x-oauth2-dev_0.0~git20190604.0f29369-2_all.deb ... Unpacking golang-golang-x-oauth2-dev (0.0~git20190604.0f29369-2) ... Selecting previously unselected package golang-google-cloud-compute-metadata-dev. Preparing to unpack .../085-golang-google-cloud-compute-metadata-dev_0.56.0-1_all.deb ... Unpacking golang-google-cloud-compute-metadata-dev (0.56.0-1) ... Selecting previously unselected package golang-golang-x-oauth2-google-dev. Preparing to unpack .../086-golang-golang-x-oauth2-google-dev_0.0~git20190604.0f29369-2_all.deb ... Unpacking golang-golang-x-oauth2-google-dev (0.0~git20190604.0f29369-2) ... Selecting previously unselected package golang-google-genproto-dev. Preparing to unpack .../087-golang-google-genproto-dev_0.0~git20200413.b5235f6-1_all.deb ... Unpacking golang-google-genproto-dev (0.0~git20200413.b5235f6-1) ... Selecting previously unselected package golang-google-grpc-dev. Preparing to unpack .../088-golang-google-grpc-dev_1.27.1-1_all.deb ... Unpacking golang-google-grpc-dev (1.27.1-1) ... Selecting previously unselected package golang-github-go-kit-kit-dev. Preparing to unpack .../089-golang-github-go-kit-kit-dev_0.6.0-3_all.deb ... Unpacking golang-github-go-kit-kit-dev (0.6.0-3) ... Selecting previously unselected package golang-github-julienschmidt-httprouter-dev. Preparing to unpack .../090-golang-github-julienschmidt-httprouter-dev_1.3.0-1_all.deb ... Unpacking golang-github-julienschmidt-httprouter-dev (1.3.0-1) ... Selecting previously unselected package golang-github-jpillora-backoff-dev. Preparing to unpack .../091-golang-github-jpillora-backoff-dev_1.0.0-1.1_all.deb ... Unpacking golang-github-jpillora-backoff-dev (1.0.0-1.1) ... Selecting previously unselected package golang-github-mwitkow-go-conntrack-dev. Preparing to unpack .../092-golang-github-mwitkow-go-conntrack-dev_0.0~git20190716.2f06839-2_all.deb ... Unpacking golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-2) ... Selecting previously unselected package golang-gopkg-alecthomas-kingpin.v2-dev. Preparing to unpack .../093-golang-gopkg-alecthomas-kingpin.v2-dev_2.2.6-3_all.deb ... Unpacking golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-3) ... Selecting previously unselected package golang-gopkg-yaml.v2-dev. Preparing to unpack .../094-golang-gopkg-yaml.v2-dev_2.4.0-1_all.deb ... Unpacking golang-gopkg-yaml.v2-dev (2.4.0-1) ... Selecting previously unselected package golang-protobuf-extensions-dev. Preparing to unpack .../095-golang-protobuf-extensions-dev_1.0.1-1_all.deb ... Unpacking golang-protobuf-extensions-dev (1.0.1-1) ... Selecting previously unselected package golang-github-prometheus-common-dev. Preparing to unpack .../096-golang-github-prometheus-common-dev_0.15.0-2_all.deb ... Unpacking golang-github-prometheus-common-dev (0.15.0-2) ... Selecting previously unselected package golang-golang-x-sync-dev. Preparing to unpack .../097-golang-golang-x-sync-dev_0.0~git20210220.036812b-1_all.deb ... Unpacking golang-golang-x-sync-dev (0.0~git20210220.036812b-1) ... Selecting previously unselected package golang-github-prometheus-procfs-dev. Preparing to unpack .../098-golang-github-prometheus-procfs-dev_0.3.0-2_all.deb ... Unpacking golang-github-prometheus-procfs-dev (0.3.0-2) ... Selecting previously unselected package golang-github-prometheus-client-golang-dev. Preparing to unpack .../099-golang-github-prometheus-client-golang-dev_1.9.0-2_all.deb ... Unpacking golang-github-prometheus-client-golang-dev (1.9.0-2) ... Selecting previously unselected package golang-github-armon-go-metrics-dev. Preparing to unpack .../100-golang-github-armon-go-metrics-dev_0.3.4-1_all.deb ... Unpacking golang-github-armon-go-metrics-dev (0.3.4-1) ... Selecting previously unselected package golang-github-creack-pty-dev. Preparing to unpack .../101-golang-github-creack-pty-dev_1.1.11-1_all.deb ... Unpacking golang-github-creack-pty-dev (1.1.11-1) ... Selecting previously unselected package golang-github-vmihailenco-tagparser-dev. Preparing to unpack .../102-golang-github-vmihailenco-tagparser-dev_0.1.1-2_all.deb ... Unpacking golang-github-vmihailenco-tagparser-dev (0.1.1-2) ... Selecting previously unselected package golang-gopkg-vmihailenco-msgpack.v2-dev. Preparing to unpack .../103-golang-gopkg-vmihailenco-msgpack.v2-dev_4.3.1-2_all.deb ... Unpacking golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ... Selecting previously unselected package golang-gopkg-tomb.v2-dev. Preparing to unpack .../104-golang-gopkg-tomb.v2-dev_0.0~git20161208.d5d1b58-3_all.deb ... Unpacking golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ... Selecting previously unselected package libsasl2-modules-db:i386. Preparing to unpack .../105-libsasl2-modules-db_2.1.27+dfsg-2.1+deb11u1_i386.deb ... Unpacking libsasl2-modules-db:i386 (2.1.27+dfsg-2.1+deb11u1) ... Selecting previously unselected package libsasl2-2:i386. Preparing to unpack .../106-libsasl2-2_2.1.27+dfsg-2.1+deb11u1_i386.deb ... Unpacking libsasl2-2:i386 (2.1.27+dfsg-2.1+deb11u1) ... Selecting previously unselected package libsasl2-dev. Preparing to unpack .../107-libsasl2-dev_2.1.27+dfsg-2.1+deb11u1_i386.deb ... Unpacking libsasl2-dev (2.1.27+dfsg-2.1+deb11u1) ... Selecting previously unselected package golang-github-kr-text-dev. Preparing to unpack .../108-golang-github-kr-text-dev_0.2.0-1_all.deb ... Unpacking golang-github-kr-text-dev (0.2.0-1) ... Selecting previously unselected package golang-github-kr-pretty-dev. Preparing to unpack .../109-golang-github-kr-pretty-dev_0.2.1+git20200831.59b4212-1_all.deb ... Unpacking golang-github-kr-pretty-dev (0.2.1+git20200831.59b4212-1) ... Selecting previously unselected package golang-gopkg-check.v1-dev. Preparing to unpack .../110-golang-gopkg-check.v1-dev_0.0+git20200902.038fdea-1_all.deb ... Unpacking golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ... Selecting previously unselected package golang-gopkg-mgo.v2-dev. Preparing to unpack .../111-golang-gopkg-mgo.v2-dev_2016.08.01-7_all.deb ... Unpacking golang-gopkg-mgo.v2-dev (2016.08.01-7) ... Selecting previously unselected package golang-github-hashicorp-go-msgpack-dev. Preparing to unpack .../112-golang-github-hashicorp-go-msgpack-dev_0.5.5-1_all.deb ... Unpacking golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ... Setting up libpipeline1:i386 (1.5.3-1) ... Setting up golang-gopkg-yaml.v2-dev (2.4.0-1) ... Setting up golang-github-hashicorp-go-uuid-dev (1.0.2-1) ... Setting up bsdextrautils (2.36.1-8+deb11u1) ... update-alternatives: using /usr/bin/write.ul to provide /usr/bin/write (write) in auto mode Setting up golang-github-creack-pty-dev (1.1.11-1) ... Setting up libicu67:i386 (67.1-7) ... Setting up golang-github-golang-mock-dev (1.3.1-2) ... Setting up golang-github-stretchr-objx-dev (0.3.0-1) ... Setting up libmagic-mgc (1:5.39-3+deb11u1) ... Setting up golang-github-pkg-errors-dev (0.9.1-1) ... Setting up golang-github-hashicorp-golang-lru-dev (0.5.4-2) ... Setting up golang-1.15-src (1.15.15-1~deb11u4) ... Setting up libarchive-zip-perl (1.68-1) ... Setting up golang-gopkg-yaml.v3-dev (3.0.0~git20200121.a6ecf24-3) ... Setting up libdebhelper-perl (13.3.4) ... Setting up golang-golang-x-sys-dev (0.0~git20210124.22da62e-1) ... Setting up libmagic1:i386 (1:5.39-3+deb11u1) ... Setting up golang-github-pmezard-go-difflib-dev (1.0.0-3) ... Setting up golang-github-modern-go-concurrent-dev (1.0.3-1.1) ... Setting up gettext-base (0.21-4) ... Setting up golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ... Setting up libprotobuf23:i386 (3.12.4-1+deb11u1) ... Setting up file (1:5.39-3+deb11u1) ... Setting up golang-github-go-stack-stack-dev (1.8.0-1) ... Setting up golang-github-beorn7-perks-dev (1.0.1-1) ... Setting up golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ... Setting up libsasl2-modules-db:i386 (2.1.27+dfsg-2.1+deb11u1) ... Setting up golang-github-cespare-xxhash-dev (2.1.1-1) ... Setting up golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ... Setting up golang-github-jpillora-backoff-dev (1.0.0-1.1) ... Setting up golang-github-davecgh-go-spew-dev (1.1.1-2) ... Setting up libprotobuf-lite23:i386 (3.12.4-1+deb11u1) ... Setting up autotools-dev (20180224.1+nmu1) ... Setting up golang-github-pascaldekloe-goe-dev (0.1.0-2) ... Setting up golang-github-go-logfmt-logfmt-dev (0.5.0-2) ... Setting up golang-github-kr-text-dev (0.2.0-1) ... Setting up golang-github-stretchr-testify-dev (1.6.1-2) ... Setting up golang-github-hashicorp-go-immutable-radix-dev (1.3.0-1) ... Setting up libprotoc23:i386 (3.12.4-1+deb11u1) ... Setting up libsigsegv2:i386 (2.13-1) ... Setting up golang-golang-x-sync-dev (0.0~git20210220.036812b-1) ... Setting up autopoint (0.21-4) ... Setting up golang-glog-dev (0.0~git20160126.23def4e-3) ... Setting up libsasl2-2:i386 (2.1.27+dfsg-2.1+deb11u1) ... Setting up golang-github-julienschmidt-httprouter-dev (1.3.0-1) ... Setting up zlib1g-dev:i386 (1:1.2.11.dfsg-2+deb11u2) ... Setting up golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-2) ... Setting up sensible-utils (0.0.14) ... Setting up golang-golang-x-text-dev (0.3.6-1) ... Setting up libuchardet0:i386 (0.0.7-1) ... Setting up golang-github-prometheus-procfs-dev (0.3.0-2) ... Setting up golang-github-alecthomas-units-dev (0.0~git20201120.1786d5e-1) ... Setting up golang-github-modern-go-reflect2-dev (1.0.1-1) ... Setting up libsub-override-perl (0.09-2) ... Setting up golang-github-dgrijalva-jwt-go-dev (3.2.0-3) ... Setting up golang-github-datadog-datadog-go-dev (2.1.0-3) ... Setting up golang-golang-x-xerrors-dev (0.0~git20191204.9bdfabe-1) ... Setting up golang-src:i386 (2:1.15~1) ... Setting up golang-1.15-go (1.15.15-1~deb11u4) ... Setting up libelf1:i386 (0.183-1) ... Setting up golang-github-vmihailenco-tagparser-dev (0.1.1-2) ... Setting up libxml2:i386 (2.9.10+dfsg-6.7+deb11u4) ... Setting up golang-github-kr-pretty-dev (0.2.1+git20200831.59b4212-1) ... Setting up libfile-stripnondeterminism-perl (1.12.0-1) ... Setting up golang-github-google-go-cmp-dev (0.5.4-1) ... Setting up golang-github-mattn-go-isatty-dev (0.0.12-1) ... Setting up golang-github-sirupsen-logrus-dev (1.7.0-2) ... Setting up golang-github-opentracing-opentracing-go-dev (1.2.0-1) ... Setting up gettext (0.21-4) ... Setting up protobuf-compiler (3.12.4-1+deb11u1) ... Setting up golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ... Setting up golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-3) ... Setting up libtool (2.4.6-15) ... Setting up golang-go (2:1.15~1) ... Setting up golang-github-mattn-go-colorable-dev (0.1.7-1) ... Setting up m4 (1.4.18-5) ... Setting up golang-github-json-iterator-go-dev (1.1.10-2) ... Setting up golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ... Setting up golang-any:i386 (2:1.15~1) ... Setting up intltool-debian (0.35.0+20060710.5) ... Setting up libprotobuf-dev:i386 (3.12.4-1+deb11u1) ... Setting up libsasl2-dev (2.1.27+dfsg-2.1+deb11u1) ... Setting up golang-gopkg-mgo.v2-dev (2016.08.01-7) ... Setting up golang-goprotobuf-dev (1.3.4-2+b6) ... Setting up golang-golang-x-net-dev (1:0.0+git20210119.5f4716e+dfsg-4) ... Setting up autoconf (2.69-14) ... Setting up dh-strip-nondeterminism (1.12.0-1) ... Setting up dwz (0.13+20210201-1) ... Setting up groff-base (1.22.4-6) ... Setting up golang-github-prometheus-client-model-dev (0.2.0-2) ... Setting up golang-golang-x-oauth2-dev (0.0~git20190604.0f29369-2) ... Setting up automake (1:1.16.3-2) ... update-alternatives: using /usr/bin/automake-1.16 to provide /usr/bin/automake (automake) in auto mode Setting up golang-golang-x-time-dev (0.0+git20200630.3af7569-1) ... Setting up golang-github-fatih-color-dev (1.7.0-1) ... Setting up golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ... Setting up po-debconf (1.0.21+nmu1) ... Setting up golang-protobuf-extensions-dev (1.0.1-1) ... Setting up golang-google-cloud-compute-metadata-dev (0.56.0-1) ... Setting up man-db (2.9.4-2) ... Not building database; man-db/auto-update is not 'true'. Setting up dh-autoreconf (20) ... Setting up golang-golang-x-oauth2-google-dev (0.0~git20190604.0f29369-2) ... Setting up golang-github-hashicorp-go-hclog-dev (0.11.0-1) ... Setting up debhelper (13.3.4) ... Setting up golang-github-hashicorp-go-retryablehttp-dev (0.6.4-2) ... Setting up dh-golang (1.51) ... Setting up golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-3) ... Setting up golang-google-genproto-dev (0.0~git20200413.b5235f6-1) ... Setting up golang-github-prometheus-client-golang-dev (1.9.0-2) ... Setting up golang-github-armon-go-metrics-dev (0.3.4-1) ... Setting up golang-google-grpc-dev (1.27.1-1) ... Setting up golang-github-go-kit-kit-dev (0.6.0-3) ... Setting up golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-2) ... Setting up golang-github-prometheus-common-dev (0.15.0-2) ... Processing triggers for libc-bin (2.31-13+deb11u6) ... Reading package lists... Building dependency tree... Reading state information... Reading extended state information... Initializing package states... Writing extended state information... Building tag database... -> Finished parsing the build-deps Reading package lists... Building dependency tree... Reading state information... fakeroot is already the newest version (1.25.3-1.1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. I: Building the package I: user script /srv/workspace/pbuilder/63060/tmp/hooks/A99_set_merged_usr starting Not re-configuring usrmerge for bullseye I: user script /srv/workspace/pbuilder/63060/tmp/hooks/A99_set_merged_usr finished hostname: Name or service not known I: Running cd /build/reproducible-path/golang-github-hashicorp-raft-1.1.2/ && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-buildpackage -us -uc -b && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-genchanges -S > ../golang-github-hashicorp-raft_1.1.2-1_source.changes dpkg-buildpackage: info: source package golang-github-hashicorp-raft dpkg-buildpackage: info: source version 1.1.2-1 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Dmitry Smirnov dpkg-source --before-build . dpkg-buildpackage: info: host architecture i386 fakeroot debian/rules clean dh clean --buildsystem=golang --with=golang dh_auto_clean -O--buildsystem=golang dh_autoreconf_clean -O--buildsystem=golang dh_clean -O--buildsystem=golang debian/rules build dh build --buildsystem=golang --with=golang dh_update_autotools_config -O--buildsystem=golang dh_autoreconf -O--buildsystem=golang dh_auto_configure -O--buildsystem=golang dh_auto_build -O--buildsystem=golang cd obj-i686-linux-gnu && go install -trimpath -v -p 15 github.com/hashicorp/raft github.com/hashicorp/raft/bench internal/unsafeheader crypto/subtle crypto/internal/subtle internal/race encoding internal/nettrace container/list runtime/internal/atomic unicode/utf16 math/bits internal/cpu unicode/utf8 sync/atomic unicode runtime/internal/sys vendor/golang.org/x/crypto/cryptobyte/asn1 vendor/golang.org/x/crypto/internal/subtle golang.org/x/sys/internal/unsafeheader runtime/cgo internal/testlog runtime/internal/math math internal/bytealg runtime internal/reflectlite sync internal/singleflight math/rand errors sort io internal/oserror github.com/hashicorp/golang-lru/simplelru vendor/golang.org/x/net/dns/dnsmessage strconv syscall hash crypto/internal/randutil bytes strings crypto/hmac hash/crc64 hash/crc32 vendor/golang.org/x/crypto/hkdf crypto crypto/rc4 reflect vendor/golang.org/x/text/transform path bufio github.com/hashicorp/go-immutable-radix html regexp/syntax internal/syscall/execenv internal/syscall/unix time regexp context internal/poll internal/fmtsort encoding/binary os encoding/base64 crypto/cipher crypto/sha1 crypto/sha512 crypto/sha256 crypto/ed25519/internal/edwards25519 vendor/golang.org/x/crypto/poly1305 crypto/md5 golang.org/x/sys/unix encoding/pem crypto/aes crypto/des vendor/golang.org/x/crypto/chacha20 path/filepath os/signal runtime/debug fmt net vendor/golang.org/x/crypto/chacha20poly1305 io/ioutil log net/url math/big encoding/hex vendor/golang.org/x/crypto/curve25519 mime/quotedprintable vendor/golang.org/x/net/http2/hpack net/http/internal mime compress/flate encoding/json encoding/gob vendor/golang.org/x/text/unicode/norm vendor/golang.org/x/text/unicode/bidi go/token text/template/parse flag runtime/trace testing vendor/golang.org/x/text/secure/bidirule github.com/mattn/go-isatty compress/gzip github.com/mattn/go-colorable github.com/fatih/color text/template github.com/hashicorp/go-hclog vendor/golang.org/x/net/idna crypto/rand crypto/elliptic crypto/dsa encoding/asn1 crypto/ed25519 crypto/rsa html/template vendor/golang.org/x/crypto/cryptobyte crypto/x509/pkix crypto/ecdsa vendor/golang.org/x/net/http/httpproxy net/textproto crypto/x509 vendor/golang.org/x/net/http/httpguts mime/multipart crypto/tls net/http/httptrace net/http net/rpc github.com/armon/go-metrics github.com/hashicorp/go-msgpack/codec github.com/hashicorp/raft github.com/hashicorp/raft/bench dh_auto_test -O--buildsystem=golang cd obj-i686-linux-gnu && go test -vet=off -v -p 15 github.com/hashicorp/raft github.com/hashicorp/raft/bench === RUN TestCommitment_setVoters --- PASS: TestCommitment_setVoters (0.00s) === RUN TestCommitment_match_max --- PASS: TestCommitment_match_max (0.00s) === RUN TestCommitment_match_nonVoting --- PASS: TestCommitment_match_nonVoting (0.00s) === RUN TestCommitment_recalculate --- PASS: TestCommitment_recalculate (0.00s) === RUN TestCommitment_recalculate_startIndex --- PASS: TestCommitment_recalculate_startIndex (0.00s) === RUN TestCommitment_noVoterSanity --- PASS: TestCommitment_noVoterSanity (0.00s) === RUN TestCommitment_singleVoter --- PASS: TestCommitment_singleVoter (0.00s) === RUN TestConfiguration_Configuration_Clone --- PASS: TestConfiguration_Configuration_Clone (0.00s) === RUN TestConfiguration_configurations_Clone --- PASS: TestConfiguration_configurations_Clone (0.00s) === RUN TestConfiguration_hasVote --- PASS: TestConfiguration_hasVote (0.00s) === RUN TestConfiguration_checkConfiguration --- PASS: TestConfiguration_checkConfiguration (0.00s) === RUN TestConfiguration_nextConfiguration_table --- PASS: TestConfiguration_nextConfiguration_table (0.00s) === RUN TestConfiguration_nextConfiguration_prevIndex --- PASS: TestConfiguration_nextConfiguration_prevIndex (0.00s) === RUN TestConfiguration_nextConfiguration_checkConfiguration --- PASS: TestConfiguration_nextConfiguration_checkConfiguration (0.00s) === RUN TestConfiguration_encodeDecodePeers --- PASS: TestConfiguration_encodeDecodePeers (0.00s) === RUN TestConfiguration_encodeDecodeConfiguration --- PASS: TestConfiguration_encodeDecodeConfiguration (0.00s) === RUN TestDiscardSnapshotStoreImpl --- PASS: TestDiscardSnapshotStoreImpl (0.00s) === RUN TestDiscardSnapshotSinkImpl --- PASS: TestDiscardSnapshotSinkImpl (0.00s) === RUN TestFileSnapshotStoreImpl --- PASS: TestFileSnapshotStoreImpl (0.00s) === RUN TestFileSnapshotSinkImpl --- PASS: TestFileSnapshotSinkImpl (0.00s) === RUN TestFileSS_CreateSnapshotMissingParentDir testing.go:153: 2025-02-10T11:34:03.633Z [INFO] creating new snapshot: path=/tmp/raft280901713/raft366091644/snapshots/3-10-1739187243633.tmp --- PASS: TestFileSS_CreateSnapshotMissingParentDir (0.00s) === RUN TestFileSS_CreateSnapshot testing.go:153: 2025-02-10T11:34:03.637Z [INFO] creating new snapshot: path=/tmp/raft042644139/snapshots/3-10-1739187243637.tmp testing.go:153: 2025-02-10T11:34:03.640Z [WARN] found temporary snapshot: name=3-10-1739187243637.tmp --- PASS: TestFileSS_CreateSnapshot (0.00s) === RUN TestFileSS_CancelSnapshot testing.go:153: 2025-02-10T11:34:03.642Z [INFO] creating new snapshot: path=/tmp/raft367907342/snapshots/3-10-1739187243642.tmp --- PASS: TestFileSS_CancelSnapshot (0.00s) === RUN TestFileSS_Retention testing.go:153: 2025-02-10T11:34:03.644Z [INFO] creating new snapshot: path=/tmp/raft868915733/snapshots/3-10-1739187243644.tmp testing.go:153: 2025-02-10T11:34:03.647Z [INFO] creating new snapshot: path=/tmp/raft868915733/snapshots/3-11-1739187243647.tmp testing.go:153: 2025-02-10T11:34:03.650Z [INFO] creating new snapshot: path=/tmp/raft868915733/snapshots/3-12-1739187243650.tmp testing.go:153: 2025-02-10T11:34:03.653Z [INFO] reaping snapshot: path=/tmp/raft868915733/snapshots/3-10-1739187243644 testing.go:153: 2025-02-10T11:34:03.653Z [INFO] creating new snapshot: path=/tmp/raft868915733/snapshots/3-13-1739187243653.tmp testing.go:153: 2025-02-10T11:34:03.656Z [INFO] reaping snapshot: path=/tmp/raft868915733/snapshots/3-11-1739187243647 testing.go:153: 2025-02-10T11:34:03.656Z [INFO] creating new snapshot: path=/tmp/raft868915733/snapshots/3-14-1739187243656.tmp testing.go:153: 2025-02-10T11:34:03.659Z [INFO] reaping snapshot: path=/tmp/raft868915733/snapshots/3-12-1739187243650 --- PASS: TestFileSS_Retention (0.02s) === RUN TestFileSS_BadPerm --- PASS: TestFileSS_BadPerm (0.00s) === RUN TestFileSS_MissingParentDir --- PASS: TestFileSS_MissingParentDir (0.00s) === RUN TestFileSS_Ordering testing.go:153: 2025-02-10T11:34:03.660Z [INFO] creating new snapshot: path=/tmp/raft471177380/snapshots/5-130350-1739187243660.tmp testing.go:153: 2025-02-10T11:34:03.663Z [INFO] creating new snapshot: path=/tmp/raft471177380/snapshots/36-204917-1739187243663.tmp --- PASS: TestFileSS_Ordering (0.01s) === RUN TestDeferFutureSuccess --- PASS: TestDeferFutureSuccess (0.00s) === RUN TestDeferFutureError --- PASS: TestDeferFutureError (0.00s) === RUN TestDeferFutureConcurrent --- PASS: TestDeferFutureConcurrent (0.00s) === RUN TestInmemSnapshotStoreImpl --- PASS: TestInmemSnapshotStoreImpl (0.00s) === RUN TestInmemSnapshotSinkImpl --- PASS: TestInmemSnapshotSinkImpl (0.00s) === RUN TestInmemSS_CreateSnapshot --- PASS: TestInmemSS_CreateSnapshot (0.00s) === RUN TestInmemSS_OpenSnapshotTwice --- PASS: TestInmemSS_OpenSnapshotTwice (0.00s) === RUN TestInmemTransportImpl --- PASS: TestInmemTransportImpl (0.00s) === RUN TestInmemTransportWriteTimeout --- PASS: TestInmemTransportWriteTimeout (0.01s) === RUN TestRaft_Integ --- SKIP: TestRaft_Integ (0.00s) === RUN TestLogCache --- PASS: TestLogCache (0.00s) === RUN TestNetworkTransport_CloseStreams --- PASS: TestNetworkTransport_CloseStreams (0.00s) === RUN TestNetworkTransport_StartStop --- PASS: TestNetworkTransport_StartStop (0.00s) === RUN TestNetworkTransport_Heartbeat_FastPath --- PASS: TestNetworkTransport_Heartbeat_FastPath (0.00s) === RUN TestNetworkTransport_AppendEntries --- PASS: TestNetworkTransport_AppendEntries (0.00s) === RUN TestNetworkTransport_AppendEntriesPipeline --- PASS: TestNetworkTransport_AppendEntriesPipeline (0.00s) === RUN TestNetworkTransport_AppendEntriesPipeline_CloseStreams net_transport_test.go:339: Disabled unreliable test --- SKIP: TestNetworkTransport_AppendEntriesPipeline_CloseStreams (0.00s) === RUN TestNetworkTransport_RequestVote --- PASS: TestNetworkTransport_RequestVote (0.00s) === RUN TestNetworkTransport_InstallSnapshot --- PASS: TestNetworkTransport_InstallSnapshot (0.00s) === RUN TestNetworkTransport_EncodeDecode --- PASS: TestNetworkTransport_EncodeDecode (0.00s) === RUN TestNetworkTransport_EncodeDecode_AddressProvider --- PASS: TestNetworkTransport_EncodeDecode_AddressProvider (0.00s) === RUN TestNetworkTransport_PooledConn --- PASS: TestNetworkTransport_PooledConn (0.00s) === RUN TestNetworkTransport_ListenBackoff net_transport_test.go:718: countingWriter: 2025-02-10T11:34:03.683Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:03.688Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:03.698Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:03.718Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:03.758Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:03.838Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:03.998Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:04.318Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:04.959Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:05.959Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2025-02-10T11:34:06.959Z [ERROR] test: failed to accept connection: error="intentional error in test" --- PASS: TestNetworkTransport_ListenBackoff (4.00s) === RUN TestPeersJSON_BadConfiguration --- PASS: TestPeersJSON_BadConfiguration (0.00s) === RUN TestPeersJSON_ReadPeersJSON --- PASS: TestPeersJSON_ReadPeersJSON (0.00s) === RUN TestPeersJSON_ReadConfigJSON --- PASS: TestPeersJSON_ReadConfigJSON (0.00s) === RUN TestRaft_StartStop testLoggerAdapter verbose: server-5a5ae570-a20c-77b7-62d3-7aaa51d80fc4: 2025-02-10T11:34:07.684Z [INFO] server-5a5ae570-a20c-77b7-62d3-7aaa51d80fc4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5a5ae570-a20c-77b7-62d3-7aaa51d80fc4 Address:5a5ae570-a20c-77b7-62d3-7aaa51d80fc4}]" testing.go:149: server-5a5ae570-a20c-77b7-62d3-7aaa51d80fc4: 2025-02-10T11:34:07.684Z [INFO] server-5a5ae570-a20c-77b7-62d3-7aaa51d80fc4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5a5ae570-a20c-77b7-62d3-7aaa51d80fc4 Address:5a5ae570-a20c-77b7-62d3-7aaa51d80fc4}]" --- PASS: TestRaft_StartStop (0.00s) === RUN TestRaft_AfterShutdown testLoggerAdapter verbose: server-2919e3a4-7739-8c12-39fb-54ac5c5843f1: 2025-02-10T11:34:07.684Z [INFO] server-2919e3a4-7739-8c12-39fb-54ac5c5843f1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2919e3a4-7739-8c12-39fb-54ac5c5843f1 Address:2919e3a4-7739-8c12-39fb-54ac5c5843f1}]" testing.go:149: server-2919e3a4-7739-8c12-39fb-54ac5c5843f1: 2025-02-10T11:34:07.684Z [INFO] server-2919e3a4-7739-8c12-39fb-54ac5c5843f1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2919e3a4-7739-8c12-39fb-54ac5c5843f1 Address:2919e3a4-7739-8c12-39fb-54ac5c5843f1}]" --- PASS: TestRaft_AfterShutdown (0.00s) === RUN TestRaft_LiveBootstrap testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.685Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: initial configuration: index=0 servers=[] testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.685Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-196ee12f-d0a1-7443-78b4-23de8d3db955: 2025-02-10T11:34:07.685Z [INFO] server-196ee12f-d0a1-7443-78b4-23de8d3db955: initial configuration: index=0 servers=[] testing.go:149: server-196ee12f-d0a1-7443-78b4-23de8d3db955: 2025-02-10T11:34:07.685Z [INFO] server-196ee12f-d0a1-7443-78b4-23de8d3db955: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.685Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: entering follower state: follower="Node at 20ccb388-7eac-7a49-4054-462a1a6abfba [Follower]" leader= testLoggerAdapter verbose: server-bfd680a6-3138-b572-07f4-d76107385122: 2025-02-10T11:34:07.685Z [INFO] server-bfd680a6-3138-b572-07f4-d76107385122: initial configuration: index=0 servers=[] testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.685Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: entering follower state: follower="Node at 20ccb388-7eac-7a49-4054-462a1a6abfba [Follower]" leader= testing.go:149: server-bfd680a6-3138-b572-07f4-d76107385122: 2025-02-10T11:34:07.685Z [INFO] server-bfd680a6-3138-b572-07f4-d76107385122: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-bfd680a6-3138-b572-07f4-d76107385122: 2025-02-10T11:34:07.685Z [INFO] server-bfd680a6-3138-b572-07f4-d76107385122: entering follower state: follower="Node at bfd680a6-3138-b572-07f4-d76107385122 [Follower]" leader= testing.go:149: server-bfd680a6-3138-b572-07f4-d76107385122: 2025-02-10T11:34:07.685Z [INFO] server-bfd680a6-3138-b572-07f4-d76107385122: entering follower state: follower="Node at bfd680a6-3138-b572-07f4-d76107385122 [Follower]" leader= testLoggerAdapter verbose: cluster: 2025-02-10T11:34:07.685Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:07.685Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-196ee12f-d0a1-7443-78b4-23de8d3db955: 2025-02-10T11:34:07.685Z [INFO] server-196ee12f-d0a1-7443-78b4-23de8d3db955: entering follower state: follower="Node at 196ee12f-d0a1-7443-78b4-23de8d3db955 [Follower]" leader= testing.go:149: server-196ee12f-d0a1-7443-78b4-23de8d3db955: 2025-02-10T11:34:07.685Z [INFO] server-196ee12f-d0a1-7443-78b4-23de8d3db955: entering follower state: follower="Node at 196ee12f-d0a1-7443-78b4-23de8d3db955 [Follower]" leader= testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [WARN] server-20ccb388-7eac-7a49-4054-462a1a6abfba: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [WARN] server-20ccb388-7eac-7a49-4054-462a1a6abfba: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: entering candidate state: node="Node at 20ccb388-7eac-7a49-4054-462a1a6abfba [Candidate]" term=2 testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: entering candidate state: node="Node at 20ccb388-7eac-7a49-4054-462a1a6abfba [Candidate]" term=2 testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: election won: tally=2 testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: election won: tally=2 testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: entering leader state: leader="Node at 20ccb388-7eac-7a49-4054-462a1a6abfba [Leader]" testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: entering leader state: leader="Node at 20ccb388-7eac-7a49-4054-462a1a6abfba [Leader]" testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: added peer, starting replication: peer=server-196ee12f-d0a1-7443-78b4-23de8d3db955 testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: added peer, starting replication: peer=server-196ee12f-d0a1-7443-78b4-23de8d3db955 testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: added peer, starting replication: peer=server-bfd680a6-3138-b572-07f4-d76107385122 testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: added peer, starting replication: peer=server-bfd680a6-3138-b572-07f4-d76107385122 testLoggerAdapter verbose: server-196ee12f-d0a1-7443-78b4-23de8d3db955: 2025-02-10T11:34:07.737Z [WARN] server-196ee12f-d0a1-7443-78b4-23de8d3db955: failed to get previous log: previous-index=1 last-index=0 error="log not found" testLoggerAdapter verbose: server-bfd680a6-3138-b572-07f4-d76107385122: 2025-02-10T11:34:07.737Z [WARN] server-bfd680a6-3138-b572-07f4-d76107385122: failed to get previous log: previous-index=1 last-index=0 error="log not found" testing.go:149: server-196ee12f-d0a1-7443-78b4-23de8d3db955: 2025-02-10T11:34:07.737Z [WARN] server-196ee12f-d0a1-7443-78b4-23de8d3db955: failed to get previous log: previous-index=1 last-index=0 error="log not found" testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [WARN] server-20ccb388-7eac-7a49-4054-462a1a6abfba: appendEntries rejected, sending older logs: peer="{Voter server-196ee12f-d0a1-7443-78b4-23de8d3db955 196ee12f-d0a1-7443-78b4-23de8d3db955}" next=1 testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [WARN] server-20ccb388-7eac-7a49-4054-462a1a6abfba: appendEntries rejected, sending older logs: peer="{Voter server-196ee12f-d0a1-7443-78b4-23de8d3db955 196ee12f-d0a1-7443-78b4-23de8d3db955}" next=1 testing.go:149: server-bfd680a6-3138-b572-07f4-d76107385122: 2025-02-10T11:34:07.737Z [WARN] server-bfd680a6-3138-b572-07f4-d76107385122: failed to get previous log: previous-index=1 last-index=0 error="log not found" testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: pipelining replication: peer="{Voter server-196ee12f-d0a1-7443-78b4-23de8d3db955 196ee12f-d0a1-7443-78b4-23de8d3db955}" testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: pipelining replication: peer="{Voter server-196ee12f-d0a1-7443-78b4-23de8d3db955 196ee12f-d0a1-7443-78b4-23de8d3db955}" testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [WARN] server-20ccb388-7eac-7a49-4054-462a1a6abfba: appendEntries rejected, sending older logs: peer="{Voter server-bfd680a6-3138-b572-07f4-d76107385122 bfd680a6-3138-b572-07f4-d76107385122}" next=1 testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [WARN] server-20ccb388-7eac-7a49-4054-462a1a6abfba: appendEntries rejected, sending older logs: peer="{Voter server-bfd680a6-3138-b572-07f4-d76107385122 bfd680a6-3138-b572-07f4-d76107385122}" next=1 testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: pipelining replication: peer="{Voter server-bfd680a6-3138-b572-07f4-d76107385122 bfd680a6-3138-b572-07f4-d76107385122}" testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.737Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: pipelining replication: peer="{Voter server-bfd680a6-3138-b572-07f4-d76107385122 bfd680a6-3138-b572-07f4-d76107385122}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:07.842Z [INFO] stable state for Follower reached at 2025-02-10 11:34:07.737266215 +0000 UTC m=+4.106170527 (2 nodes), 51.261022ms from start of poll, 51.687875ms from cluster start. Timeout at 2025-02-10 11:34:07.842368191 +0000 UTC m=+4.211272488, 105.101961ms after stability testing.go:149: cluster: 2025-02-10T11:34:07.842Z [INFO] stable state for Follower reached at 2025-02-10 11:34:07.737266215 +0000 UTC m=+4.106170527 (2 nodes), 51.261022ms from start of poll, 51.687875ms from cluster start. Timeout at 2025-02-10 11:34:07.842368191 +0000 UTC m=+4.211272488, 105.101961ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:07.842Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:07.842Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:07.947Z [INFO] stable state for Leader reached at 2025-02-10 11:34:07.842474736 +0000 UTC m=+4.211379035 (1 nodes), 396ns from start of poll, 156.896383ms from cluster start. Timeout at 2025-02-10 11:34:07.947549651 +0000 UTC m=+4.316453971, 105.074936ms after stability testing.go:149: cluster: 2025-02-10T11:34:07.947Z [INFO] stable state for Leader reached at 2025-02-10 11:34:07.842474736 +0000 UTC m=+4.211379035 (1 nodes), 396ns from start of poll, 156.896383ms from cluster start. Timeout at 2025-02-10 11:34:07.947549651 +0000 UTC m=+4.316453971, 105.074936ms after stability testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.957Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: aborting pipeline replication: peer="{Voter server-bfd680a6-3138-b572-07f4-d76107385122 bfd680a6-3138-b572-07f4-d76107385122}" testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.957Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: aborting pipeline replication: peer="{Voter server-bfd680a6-3138-b572-07f4-d76107385122 bfd680a6-3138-b572-07f4-d76107385122}" testLoggerAdapter verbose: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.957Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: aborting pipeline replication: peer="{Voter server-196ee12f-d0a1-7443-78b4-23de8d3db955 196ee12f-d0a1-7443-78b4-23de8d3db955}" testing.go:149: server-20ccb388-7eac-7a49-4054-462a1a6abfba: 2025-02-10T11:34:07.957Z [INFO] server-20ccb388-7eac-7a49-4054-462a1a6abfba: aborting pipeline replication: peer="{Voter server-196ee12f-d0a1-7443-78b4-23de8d3db955 196ee12f-d0a1-7443-78b4-23de8d3db955}" --- PASS: TestRaft_LiveBootstrap (0.27s) === RUN TestRaft_RecoverCluster_NoState testLoggerAdapter verbose: server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: 2025-02-10T11:34:07.958Z [INFO] server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: initial configuration: index=0 servers=[] testing.go:149: server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: 2025-02-10T11:34:07.958Z [INFO] server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: 2025-02-10T11:34:07.958Z [INFO] server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: entering follower state: follower="Node at 91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8 [Follower]" leader= testing.go:149: server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: 2025-02-10T11:34:07.958Z [INFO] server-91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8: entering follower state: follower="Node at 91d2ee6e-388a-0c83-9702-e7ff2dd0a1b8 [Follower]" leader= --- PASS: TestRaft_RecoverCluster_NoState (0.00s) === RUN TestRaft_RecoverCluster raft_test.go:119: Disabled unreliable test --- SKIP: TestRaft_RecoverCluster (0.00s) === RUN TestRaft_HasExistingState testLoggerAdapter verbose: server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: 2025-02-10T11:34:07.959Z [INFO] server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be Address:bc2b45ce-150b-fb9f-cf00-a71bad6c18be} {Suffrage:Voter ID:server-5fa4e35b-c9ad-3195-e842-b7efa886a761 Address:5fa4e35b-c9ad-3195-e842-b7efa886a761}]" testing.go:149: server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: 2025-02-10T11:34:07.959Z [INFO] server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be Address:bc2b45ce-150b-fb9f-cf00-a71bad6c18be} {Suffrage:Voter ID:server-5fa4e35b-c9ad-3195-e842-b7efa886a761 Address:5fa4e35b-c9ad-3195-e842-b7efa886a761}]" testLoggerAdapter verbose: server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: 2025-02-10T11:34:07.959Z [INFO] server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: entering follower state: follower="Node at bc2b45ce-150b-fb9f-cf00-a71bad6c18be [Follower]" leader= testing.go:149: server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: 2025-02-10T11:34:07.959Z [INFO] server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be: entering follower state: follower="Node at bc2b45ce-150b-fb9f-cf00-a71bad6c18be [Follower]" leader= testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:07.959Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be Address:bc2b45ce-150b-fb9f-cf00-a71bad6c18be} {Suffrage:Voter ID:server-5fa4e35b-c9ad-3195-e842-b7efa886a761 Address:5fa4e35b-c9ad-3195-e842-b7efa886a761}]" testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:07.959Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be Address:bc2b45ce-150b-fb9f-cf00-a71bad6c18be} {Suffrage:Voter ID:server-5fa4e35b-c9ad-3195-e842-b7efa886a761 Address:5fa4e35b-c9ad-3195-e842-b7efa886a761}]" testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:07.959Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: entering follower state: follower="Node at 5fa4e35b-c9ad-3195-e842-b7efa886a761 [Follower]" leader= testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:07.959Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: entering follower state: follower="Node at 5fa4e35b-c9ad-3195-e842-b7efa886a761 [Follower]" leader= testLoggerAdapter verbose: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:07.960Z [INFO] server-14fec058-869d-8ea4-9afe-9a1f734217c2: initial configuration: index=0 servers=[] testing.go:149: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:07.960Z [INFO] server-14fec058-869d-8ea4-9afe-9a1f734217c2: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:07.960Z [INFO] server-14fec058-869d-8ea4-9afe-9a1f734217c2: entering follower state: follower="Node at 14fec058-869d-8ea4-9afe-9a1f734217c2 [Follower]" leader= testing.go:149: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:07.960Z [INFO] server-14fec058-869d-8ea4-9afe-9a1f734217c2: entering follower state: follower="Node at 14fec058-869d-8ea4-9afe-9a1f734217c2 [Follower]" leader= testLoggerAdapter verbose: cluster: 2025-02-10T11:34:07.960Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:07.960Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [WARN] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [WARN] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: entering candidate state: node="Node at 5fa4e35b-c9ad-3195-e842-b7efa886a761 [Candidate]" term=2 testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: entering candidate state: node="Node at 5fa4e35b-c9ad-3195-e842-b7efa886a761 [Candidate]" term=2 testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: election won: tally=2 testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: election won: tally=2 testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: entering leader state: leader="Node at 5fa4e35b-c9ad-3195-e842-b7efa886a761 [Leader]" testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: entering leader state: leader="Node at 5fa4e35b-c9ad-3195-e842-b7efa886a761 [Leader]" testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: added peer, starting replication: peer=server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: added peer, starting replication: peer=server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: pipelining replication: peer="{Voter server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be bc2b45ce-150b-fb9f-cf00-a71bad6c18be}" testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.010Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: pipelining replication: peer="{Voter server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be bc2b45ce-150b-fb9f-cf00-a71bad6c18be}" testLoggerAdapter verbose: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:08.055Z [WARN] server-14fec058-869d-8ea4-9afe-9a1f734217c2: no known peers, aborting election testing.go:149: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:08.055Z [WARN] server-14fec058-869d-8ea4-9afe-9a1f734217c2: no known peers, aborting election testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.115Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.010179043 +0000 UTC m=+4.379083342 (1 nodes), 49.941648ms from start of poll, 50.647713ms from cluster start. Timeout at 2025-02-10 11:34:08.115281511 +0000 UTC m=+4.484185861, 105.102519ms after stability testing.go:149: cluster: 2025-02-10T11:34:08.115Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.010179043 +0000 UTC m=+4.379083342 (1 nodes), 49.941648ms from start of poll, 50.647713ms from cluster start. Timeout at 2025-02-10 11:34:08.115281511 +0000 UTC m=+4.484185861, 105.102519ms after stability testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: updating configuration: command=AddStaging server-id=server-14fec058-869d-8ea4-9afe-9a1f734217c2 server-addr=14fec058-869d-8ea4-9afe-9a1f734217c2 servers="[{Suffrage:Voter ID:server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be Address:bc2b45ce-150b-fb9f-cf00-a71bad6c18be} {Suffrage:Voter ID:server-5fa4e35b-c9ad-3195-e842-b7efa886a761 Address:5fa4e35b-c9ad-3195-e842-b7efa886a761} {Suffrage:Voter ID:server-14fec058-869d-8ea4-9afe-9a1f734217c2 Address:14fec058-869d-8ea4-9afe-9a1f734217c2}]" testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: updating configuration: command=AddStaging server-id=server-14fec058-869d-8ea4-9afe-9a1f734217c2 server-addr=14fec058-869d-8ea4-9afe-9a1f734217c2 servers="[{Suffrage:Voter ID:server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be Address:bc2b45ce-150b-fb9f-cf00-a71bad6c18be} {Suffrage:Voter ID:server-5fa4e35b-c9ad-3195-e842-b7efa886a761 Address:5fa4e35b-c9ad-3195-e842-b7efa886a761} {Suffrage:Voter ID:server-14fec058-869d-8ea4-9afe-9a1f734217c2 Address:14fec058-869d-8ea4-9afe-9a1f734217c2}]" testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: added peer, starting replication: peer=server-14fec058-869d-8ea4-9afe-9a1f734217c2 testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: added peer, starting replication: peer=server-14fec058-869d-8ea4-9afe-9a1f734217c2 testLoggerAdapter verbose: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:08.115Z [WARN] server-14fec058-869d-8ea4-9afe-9a1f734217c2: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-14fec058-869d-8ea4-9afe-9a1f734217c2: 2025-02-10T11:34:08.115Z [WARN] server-14fec058-869d-8ea4-9afe-9a1f734217c2: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [WARN] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: appendEntries rejected, sending older logs: peer="{Voter server-14fec058-869d-8ea4-9afe-9a1f734217c2 14fec058-869d-8ea4-9afe-9a1f734217c2}" next=1 testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [WARN] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: appendEntries rejected, sending older logs: peer="{Voter server-14fec058-869d-8ea4-9afe-9a1f734217c2 14fec058-869d-8ea4-9afe-9a1f734217c2}" next=1 testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: pipelining replication: peer="{Voter server-14fec058-869d-8ea4-9afe-9a1f734217c2 14fec058-869d-8ea4-9afe-9a1f734217c2}" testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.115Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: pipelining replication: peer="{Voter server-14fec058-869d-8ea4-9afe-9a1f734217c2 14fec058-869d-8ea4-9afe-9a1f734217c2}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.120Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:08.120Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.225Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.120778336 +0000 UTC m=+4.489682638 (1 nodes), 429ns from start of poll, 161.247009ms from cluster start. Timeout at 2025-02-10 11:34:08.225794342 +0000 UTC m=+4.594698658, 105.01602ms after stability testing.go:149: cluster: 2025-02-10T11:34:08.225Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.120778336 +0000 UTC m=+4.489682638 (1 nodes), 429ns from start of poll, 161.247009ms from cluster start. Timeout at 2025-02-10 11:34:08.225794342 +0000 UTC m=+4.594698658, 105.01602ms after stability testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.225Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: aborting pipeline replication: peer="{Voter server-14fec058-869d-8ea4-9afe-9a1f734217c2 14fec058-869d-8ea4-9afe-9a1f734217c2}" testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.225Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: aborting pipeline replication: peer="{Voter server-14fec058-869d-8ea4-9afe-9a1f734217c2 14fec058-869d-8ea4-9afe-9a1f734217c2}" testLoggerAdapter verbose: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.225Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: aborting pipeline replication: peer="{Voter server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be bc2b45ce-150b-fb9f-cf00-a71bad6c18be}" testing.go:149: server-5fa4e35b-c9ad-3195-e842-b7efa886a761: 2025-02-10T11:34:08.225Z [INFO] server-5fa4e35b-c9ad-3195-e842-b7efa886a761: aborting pipeline replication: peer="{Voter server-bc2b45ce-150b-fb9f-cf00-a71bad6c18be bc2b45ce-150b-fb9f-cf00-a71bad6c18be}" --- PASS: TestRaft_HasExistingState (0.27s) === RUN TestRaft_SingleNode testLoggerAdapter verbose: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.226Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-54a76141-f09f-18ca-f88a-fc3fb5f334ef Address:54a76141-f09f-18ca-f88a-fc3fb5f334ef}]" testing.go:149: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.226Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-54a76141-f09f-18ca-f88a-fc3fb5f334ef Address:54a76141-f09f-18ca-f88a-fc3fb5f334ef}]" testLoggerAdapter verbose: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.226Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: entering follower state: follower="Node at 54a76141-f09f-18ca-f88a-fc3fb5f334ef [Follower]" leader= testing.go:149: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.226Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: entering follower state: follower="Node at 54a76141-f09f-18ca-f88a-fc3fb5f334ef [Follower]" leader= testLoggerAdapter verbose: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [WARN] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [WARN] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: entering candidate state: node="Node at 54a76141-f09f-18ca-f88a-fc3fb5f334ef [Candidate]" term=2 testing.go:149: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: entering candidate state: node="Node at 54a76141-f09f-18ca-f88a-fc3fb5f334ef [Candidate]" term=2 testLoggerAdapter verbose: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: election won: tally=1 testing.go:149: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: election won: tally=1 testLoggerAdapter verbose: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: entering leader state: leader="Node at 54a76141-f09f-18ca-f88a-fc3fb5f334ef [Leader]" testing.go:149: server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: 2025-02-10T11:34:08.278Z [INFO] server-54a76141-f09f-18ca-f88a-fc3fb5f334ef: entering leader state: leader="Node at 54a76141-f09f-18ca-f88a-fc3fb5f334ef [Leader]" --- PASS: TestRaft_SingleNode (0.05s) === RUN TestRaft_TripleNode testLoggerAdapter verbose: server-6629d9bc-8f11-04c3-94ff-d919015e30db: 2025-02-10T11:34:08.278Z [INFO] server-6629d9bc-8f11-04c3-94ff-d919015e30db: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6629d9bc-8f11-04c3-94ff-d919015e30db Address:6629d9bc-8f11-04c3-94ff-d919015e30db} {Suffrage:Voter ID:server-9c1017c0-6e05-e7e8-2bd5-034147417eed Address:9c1017c0-6e05-e7e8-2bd5-034147417eed} {Suffrage:Voter ID:server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 Address:23f6fc87-8f95-c9e5-b119-f1456ede73b7}]" testing.go:149: server-6629d9bc-8f11-04c3-94ff-d919015e30db: 2025-02-10T11:34:08.278Z [INFO] server-6629d9bc-8f11-04c3-94ff-d919015e30db: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6629d9bc-8f11-04c3-94ff-d919015e30db Address:6629d9bc-8f11-04c3-94ff-d919015e30db} {Suffrage:Voter ID:server-9c1017c0-6e05-e7e8-2bd5-034147417eed Address:9c1017c0-6e05-e7e8-2bd5-034147417eed} {Suffrage:Voter ID:server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 Address:23f6fc87-8f95-c9e5-b119-f1456ede73b7}]" testLoggerAdapter verbose: server-6629d9bc-8f11-04c3-94ff-d919015e30db: 2025-02-10T11:34:08.279Z [INFO] server-6629d9bc-8f11-04c3-94ff-d919015e30db: entering follower state: follower="Node at 6629d9bc-8f11-04c3-94ff-d919015e30db [Follower]" leader= testing.go:149: server-6629d9bc-8f11-04c3-94ff-d919015e30db: 2025-02-10T11:34:08.279Z [INFO] server-6629d9bc-8f11-04c3-94ff-d919015e30db: entering follower state: follower="Node at 6629d9bc-8f11-04c3-94ff-d919015e30db [Follower]" leader= testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.279Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6629d9bc-8f11-04c3-94ff-d919015e30db Address:6629d9bc-8f11-04c3-94ff-d919015e30db} {Suffrage:Voter ID:server-9c1017c0-6e05-e7e8-2bd5-034147417eed Address:9c1017c0-6e05-e7e8-2bd5-034147417eed} {Suffrage:Voter ID:server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 Address:23f6fc87-8f95-c9e5-b119-f1456ede73b7}]" testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.279Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6629d9bc-8f11-04c3-94ff-d919015e30db Address:6629d9bc-8f11-04c3-94ff-d919015e30db} {Suffrage:Voter ID:server-9c1017c0-6e05-e7e8-2bd5-034147417eed Address:9c1017c0-6e05-e7e8-2bd5-034147417eed} {Suffrage:Voter ID:server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 Address:23f6fc87-8f95-c9e5-b119-f1456ede73b7}]" testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.279Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: entering follower state: follower="Node at 9c1017c0-6e05-e7e8-2bd5-034147417eed [Follower]" leader= testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.279Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: entering follower state: follower="Node at 9c1017c0-6e05-e7e8-2bd5-034147417eed [Follower]" leader= testLoggerAdapter verbose: server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: 2025-02-10T11:34:08.279Z [INFO] server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6629d9bc-8f11-04c3-94ff-d919015e30db Address:6629d9bc-8f11-04c3-94ff-d919015e30db} {Suffrage:Voter ID:server-9c1017c0-6e05-e7e8-2bd5-034147417eed Address:9c1017c0-6e05-e7e8-2bd5-034147417eed} {Suffrage:Voter ID:server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 Address:23f6fc87-8f95-c9e5-b119-f1456ede73b7}]" testing.go:149: server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: 2025-02-10T11:34:08.279Z [INFO] server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6629d9bc-8f11-04c3-94ff-d919015e30db Address:6629d9bc-8f11-04c3-94ff-d919015e30db} {Suffrage:Voter ID:server-9c1017c0-6e05-e7e8-2bd5-034147417eed Address:9c1017c0-6e05-e7e8-2bd5-034147417eed} {Suffrage:Voter ID:server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 Address:23f6fc87-8f95-c9e5-b119-f1456ede73b7}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.279Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:08.279Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: 2025-02-10T11:34:08.279Z [INFO] server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: entering follower state: follower="Node at 23f6fc87-8f95-c9e5-b119-f1456ede73b7 [Follower]" leader= testing.go:149: server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: 2025-02-10T11:34:08.279Z [INFO] server-23f6fc87-8f95-c9e5-b119-f1456ede73b7: entering follower state: follower="Node at 23f6fc87-8f95-c9e5-b119-f1456ede73b7 [Follower]" leader= testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [WARN] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [WARN] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: entering candidate state: node="Node at 9c1017c0-6e05-e7e8-2bd5-034147417eed [Candidate]" term=2 testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: entering candidate state: node="Node at 9c1017c0-6e05-e7e8-2bd5-034147417eed [Candidate]" term=2 testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: election won: tally=2 testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: election won: tally=2 testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: entering leader state: leader="Node at 9c1017c0-6e05-e7e8-2bd5-034147417eed [Leader]" testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: entering leader state: leader="Node at 9c1017c0-6e05-e7e8-2bd5-034147417eed [Leader]" testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: added peer, starting replication: peer=server-6629d9bc-8f11-04c3-94ff-d919015e30db testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: added peer, starting replication: peer=server-6629d9bc-8f11-04c3-94ff-d919015e30db testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: added peer, starting replication: peer=server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: added peer, starting replication: peer=server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: pipelining replication: peer="{Voter server-6629d9bc-8f11-04c3-94ff-d919015e30db 6629d9bc-8f11-04c3-94ff-d919015e30db}" testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: pipelining replication: peer="{Voter server-6629d9bc-8f11-04c3-94ff-d919015e30db 6629d9bc-8f11-04c3-94ff-d919015e30db}" testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: pipelining replication: peer="{Voter server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 23f6fc87-8f95-c9e5-b119-f1456ede73b7}" testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.331Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: pipelining replication: peer="{Voter server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 23f6fc87-8f95-c9e5-b119-f1456ede73b7}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.436Z [INFO] stable state for Follower reached at 2025-02-10 11:34:08.33170517 +0000 UTC m=+4.700609476 (2 nodes), 52.444483ms from start of poll, 52.815261ms from cluster start. Timeout at 2025-02-10 11:34:08.436768206 +0000 UTC m=+4.805672502, 105.063026ms after stability testing.go:149: cluster: 2025-02-10T11:34:08.436Z [INFO] stable state for Follower reached at 2025-02-10 11:34:08.33170517 +0000 UTC m=+4.700609476 (2 nodes), 52.444483ms from start of poll, 52.815261ms from cluster start. Timeout at 2025-02-10 11:34:08.436768206 +0000 UTC m=+4.805672502, 105.063026ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.436Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:08.436Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.541Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.436843117 +0000 UTC m=+4.805747416 (1 nodes), 448ns from start of poll, 157.953201ms from cluster start. Timeout at 2025-02-10 11:34:08.541919964 +0000 UTC m=+4.910824262, 105.076846ms after stability testing.go:149: cluster: 2025-02-10T11:34:08.541Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.436843117 +0000 UTC m=+4.805747416 (1 nodes), 448ns from start of poll, 157.953201ms from cluster start. Timeout at 2025-02-10 11:34:08.541919964 +0000 UTC m=+4.910824262, 105.076846ms after stability testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.552Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: aborting pipeline replication: peer="{Voter server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 23f6fc87-8f95-c9e5-b119-f1456ede73b7}" testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.552Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: aborting pipeline replication: peer="{Voter server-23f6fc87-8f95-c9e5-b119-f1456ede73b7 23f6fc87-8f95-c9e5-b119-f1456ede73b7}" testLoggerAdapter verbose: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.552Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: aborting pipeline replication: peer="{Voter server-6629d9bc-8f11-04c3-94ff-d919015e30db 6629d9bc-8f11-04c3-94ff-d919015e30db}" testing.go:149: server-9c1017c0-6e05-e7e8-2bd5-034147417eed: 2025-02-10T11:34:08.552Z [INFO] server-9c1017c0-6e05-e7e8-2bd5-034147417eed: aborting pipeline replication: peer="{Voter server-6629d9bc-8f11-04c3-94ff-d919015e30db 6629d9bc-8f11-04c3-94ff-d919015e30db}" --- PASS: TestRaft_TripleNode (0.27s) === RUN TestRaft_LeaderFail testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.553Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 Address:6fe55f11-8fc1-2744-7e49-1a80f4465bb7} {Suffrage:Voter ID:server-d697a267-1dcb-1c34-3e2b-52f10a826d65 Address:d697a267-1dcb-1c34-3e2b-52f10a826d65} {Suffrage:Voter ID:server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 Address:f9c2c13d-e823-6e25-0441-a5b28ca599a7}]" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.553Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 Address:6fe55f11-8fc1-2744-7e49-1a80f4465bb7} {Suffrage:Voter ID:server-d697a267-1dcb-1c34-3e2b-52f10a826d65 Address:d697a267-1dcb-1c34-3e2b-52f10a826d65} {Suffrage:Voter ID:server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 Address:f9c2c13d-e823-6e25-0441-a5b28ca599a7}]" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.553Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering follower state: follower="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Follower]" leader= testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.553Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering follower state: follower="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Follower]" leader= testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.553Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 Address:6fe55f11-8fc1-2744-7e49-1a80f4465bb7} {Suffrage:Voter ID:server-d697a267-1dcb-1c34-3e2b-52f10a826d65 Address:d697a267-1dcb-1c34-3e2b-52f10a826d65} {Suffrage:Voter ID:server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 Address:f9c2c13d-e823-6e25-0441-a5b28ca599a7}]" testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.553Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 Address:6fe55f11-8fc1-2744-7e49-1a80f4465bb7} {Suffrage:Voter ID:server-d697a267-1dcb-1c34-3e2b-52f10a826d65 Address:d697a267-1dcb-1c34-3e2b-52f10a826d65} {Suffrage:Voter ID:server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 Address:f9c2c13d-e823-6e25-0441-a5b28ca599a7}]" testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.553Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: entering follower state: follower="Node at d697a267-1dcb-1c34-3e2b-52f10a826d65 [Follower]" leader= testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.553Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: entering follower state: follower="Node at d697a267-1dcb-1c34-3e2b-52f10a826d65 [Follower]" leader= testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.553Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 Address:6fe55f11-8fc1-2744-7e49-1a80f4465bb7} {Suffrage:Voter ID:server-d697a267-1dcb-1c34-3e2b-52f10a826d65 Address:d697a267-1dcb-1c34-3e2b-52f10a826d65} {Suffrage:Voter ID:server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 Address:f9c2c13d-e823-6e25-0441-a5b28ca599a7}]" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.553Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 Address:6fe55f11-8fc1-2744-7e49-1a80f4465bb7} {Suffrage:Voter ID:server-d697a267-1dcb-1c34-3e2b-52f10a826d65 Address:d697a267-1dcb-1c34-3e2b-52f10a826d65} {Suffrage:Voter ID:server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 Address:f9c2c13d-e823-6e25-0441-a5b28ca599a7}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.553Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:08.553Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.553Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering follower state: follower="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Follower]" leader= testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.553Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering follower state: follower="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Follower]" leader= testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=2 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=2 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: election won: tally=2 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: election won: tally=2 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering leader state: leader="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Leader]" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering leader state: leader="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Leader]" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: added peer, starting replication: peer=server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: added peer, starting replication: peer=server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: added peer, starting replication: peer=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: added peer, starting replication: peer=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: pipelining replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: pipelining replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: pipelining replication: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.616Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: pipelining replication: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.721Z [INFO] stable state for Follower reached at 2025-02-10 11:34:08.616705176 +0000 UTC m=+4.985609484 (2 nodes), 63.163551ms from start of poll, 63.54209ms from cluster start. Timeout at 2025-02-10 11:34:08.721782017 +0000 UTC m=+5.090686313, 105.076829ms after stability testing.go:149: cluster: 2025-02-10T11:34:08.721Z [INFO] stable state for Follower reached at 2025-02-10 11:34:08.616705176 +0000 UTC m=+4.985609484 (2 nodes), 63.163551ms from start of poll, 63.54209ms from cluster start. Timeout at 2025-02-10 11:34:08.721782017 +0000 UTC m=+5.090686313, 105.076829ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.721Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:08.721Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.826Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.721846737 +0000 UTC m=+5.090751038 (1 nodes), 449ns from start of poll, 168.683644ms from cluster start. Timeout at 2025-02-10 11:34:08.82695071 +0000 UTC m=+5.195855060, 105.104022ms after stability testing.go:149: cluster: 2025-02-10T11:34:08.826Z [INFO] stable state for Leader reached at 2025-02-10 11:34:08.721846737 +0000 UTC m=+5.090751038 (1 nodes), 449ns from start of poll, 168.683644ms from cluster start. Timeout at 2025-02-10 11:34:08.82695071 +0000 UTC m=+5.195855060, 105.104022ms after stability raft_test.go:336: [INFO] Disconnecting Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Leader] testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.837Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.837Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.841Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.841Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.842Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to pipeline appendEntries: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="append pipeline closed" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.842Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to pipeline appendEntries: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="append pipeline closed" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.842Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: aborting pipeline replication: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.842Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: aborting pipeline replication: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:08.842Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:08.842Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.844Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to pipeline appendEntries: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="append pipeline closed" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.844Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to pipeline appendEntries: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="append pipeline closed" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.844Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: aborting pipeline replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.844Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: aborting pipeline replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.850Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.850Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.851Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.851Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.858Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.858Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.858Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.858Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.866Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.866Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.869Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.869Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.874Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.874Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.875Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.875Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.884Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact: server-id=server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 time=50.107459ms testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.884Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact: server-id=server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 time=50.107459ms testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.885Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.885Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.887Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.887Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact: server-id=server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 time=60.204678ms testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact: server-id=server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 time=60.204678ms testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact: server-id=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 time=59.270563ms testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact: server-id=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 time=59.270563ms testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact quorum of nodes, stepping down testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering follower state: follower="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Follower]" leader= testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.894Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering follower state: follower="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Follower]" leader= testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.902Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.902Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.903Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.903Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to heartbeat to: peer=d697a267-1dcb-1c34-3e2b-52f10a826d65 error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.911Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.911Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.914Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.914Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to appendEntries to: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.924Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: heartbeat timeout reached, starting election: last-leader=f9c2c13d-e823-6e25-0441-a5b28ca599a7 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.924Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: heartbeat timeout reached, starting election: last-leader=f9c2c13d-e823-6e25-0441-a5b28ca599a7 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.924Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering candidate state: node="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Candidate]" term=3 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.924Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering candidate state: node="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Candidate]" term=3 testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.924Z [WARN] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: rejecting vote request since we have a leader: from=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 leader=f9c2c13d-e823-6e25-0441-a5b28ca599a7 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.924Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to make requestVote RPC: target="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.924Z [WARN] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: rejecting vote request since we have a leader: from=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 leader=f9c2c13d-e823-6e25-0441-a5b28ca599a7 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.924Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to make requestVote RPC: target="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.946Z [WARN] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: heartbeat timeout reached, starting election: last-leader=f9c2c13d-e823-6e25-0441-a5b28ca599a7 testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.946Z [WARN] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: heartbeat timeout reached, starting election: last-leader=f9c2c13d-e823-6e25-0441-a5b28ca599a7 testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.946Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: entering candidate state: node="Node at d697a267-1dcb-1c34-3e2b-52f10a826d65 [Candidate]" term=3 testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.946Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: entering candidate state: node="Node at d697a267-1dcb-1c34-3e2b-52f10a826d65 [Candidate]" term=3 testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.946Z [ERROR] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: failed to make requestVote RPC: target="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:08.946Z [ERROR] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: failed to make requestVote RPC: target="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.946Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: duplicate requestVote for same term: term=3 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:08.946Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=3 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=3 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:08.954Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: Election timeout reached, restarting election testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: Election timeout reached, restarting election testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering candidate state: node="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Candidate]" term=4 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering candidate state: node="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Candidate]" term=4 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to make requestVote RPC: target="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to make requestVote RPC: target="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:09.020Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: entering follower state: follower="Node at d697a267-1dcb-1c34-3e2b-52f10a826d65 [Follower]" leader= testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: election won: tally=2 testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:09.020Z [INFO] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: entering follower state: follower="Node at d697a267-1dcb-1c34-3e2b-52f10a826d65 [Follower]" leader= testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: election won: tally=2 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering leader state: leader="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Leader]" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering leader state: leader="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Leader]" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: pipelining replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.020Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: pipelining replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.027Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.027Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.039Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.039Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.044Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.044Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.051Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: Election timeout reached, restarting election testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.051Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: Election timeout reached, restarting election testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.051Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=4 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.051Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=4 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.051Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.051Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.052Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.052Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.057Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.057Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.063Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.063Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.070Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to contact: server-id=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 time=50.096341ms testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.070Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to contact: server-id=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 time=50.096341ms testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.084Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.084Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to appendEntries to: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.090Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.090Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to heartbeat to: peer=f9c2c13d-e823-6e25-0441-a5b28ca599a7 error="failed to connect to peer: f9c2c13d-e823-6e25-0441-a5b28ca599a7" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: Election timeout reached, restarting election testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: Election timeout reached, restarting election testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=5 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=5 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" error="failed to connect to peer: d697a267-1dcb-1c34-3e2b-52f10a826d65" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.103Z [ERROR] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to make requestVote RPC: target="{Voter server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7 6fe55f11-8fc1-2744-7e49-1a80f4465bb7}" error="failed to connect to peer: 6fe55f11-8fc1-2744-7e49-1a80f4465bb7" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.119Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to contact: server-id=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 time=98.618623ms testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.119Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: failed to contact: server-id=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 time=98.618623ms testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.126Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.020915639 +0000 UTC m=+5.389819941 (1 nodes), 178.306236ms from start of poll, 467.752547ms from cluster start. Timeout at 2025-02-10 11:34:09.126021829 +0000 UTC m=+5.494926182, 105.106241ms after stability testing.go:149: cluster: 2025-02-10T11:34:09.126Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.020915639 +0000 UTC m=+5.389819941 (1 nodes), 178.306236ms from start of poll, 467.752547ms from cluster start. Timeout at 2025-02-10 11:34:09.126021829 +0000 UTC m=+5.494926182, 105.106241ms after stability raft_test.go:371: [INFO] Reconnecting Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate] testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.131Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: peer has newer term, stopping replication: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.131Z [ERROR] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: peer has newer term, stopping replication: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.131Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering follower state: follower="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Follower]" leader= testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.131Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering follower state: follower="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Follower]" leader= testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.131Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: aborting pipeline replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.131Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: aborting pipeline replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.163Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: Election timeout reached, restarting election testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.163Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: Election timeout reached, restarting election testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.163Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=6 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.163Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering candidate state: node="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Candidate]" term=6 testLoggerAdapter verbose: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:09.163Z [WARN] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: rejecting vote request since we have a leader: from=f9c2c13d-e823-6e25-0441-a5b28ca599a7 leader=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.163Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: rejecting vote request since our last term is greater: candidate=f9c2c13d-e823-6e25-0441-a5b28ca599a7 last-term=4 last-candidate-term=2 testing.go:149: server-d697a267-1dcb-1c34-3e2b-52f10a826d65: 2025-02-10T11:34:09.163Z [WARN] server-d697a267-1dcb-1c34-3e2b-52f10a826d65: rejecting vote request since we have a leader: from=f9c2c13d-e823-6e25-0441-a5b28ca599a7 leader=6fe55f11-8fc1-2744-7e49-1a80f4465bb7 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.163Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: rejecting vote request since our last term is greater: candidate=f9c2c13d-e823-6e25-0441-a5b28ca599a7 last-term=4 last-candidate-term=2 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering candidate state: node="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Candidate]" term=7 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering candidate state: node="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Candidate]" term=7 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: election won: tally=2 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: election won: tally=2 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering leader state: leader="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Leader]" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: entering leader state: leader="Node at 6fe55f11-8fc1-2744-7e49-1a80f4465bb7 [Leader]" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-d697a267-1dcb-1c34-3e2b-52f10a826d65 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.204Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering follower state: follower="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Follower]" leader= testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: added peer, starting replication: peer=server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.204Z [INFO] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: entering follower state: follower="Node at f9c2c13d-e823-6e25-0441-a5b28ca599a7 [Follower]" leader= testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: pipelining replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testLoggerAdapter verbose: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.204Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to get previous log: previous-index=5 last-index=3 error="log not found" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.204Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: pipelining replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testing.go:149: server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: 2025-02-10T11:34:09.204Z [WARN] server-f9c2c13d-e823-6e25-0441-a5b28ca599a7: failed to get previous log: previous-index=5 last-index=3 error="log not found" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.205Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: appendEntries rejected, sending older logs: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" next=4 testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.205Z [WARN] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: appendEntries rejected, sending older logs: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" next=4 testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.205Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: pipelining replication: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.205Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: pipelining replication: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.210Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: aborting pipeline replication: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.210Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: aborting pipeline replication: peer="{Voter server-f9c2c13d-e823-6e25-0441-a5b28ca599a7 f9c2c13d-e823-6e25-0441-a5b28ca599a7}" testLoggerAdapter verbose: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.210Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: aborting pipeline replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" testing.go:149: server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: 2025-02-10T11:34:09.210Z [INFO] server-6fe55f11-8fc1-2744-7e49-1a80f4465bb7: aborting pipeline replication: peer="{Voter server-d697a267-1dcb-1c34-3e2b-52f10a826d65 d697a267-1dcb-1c34-3e2b-52f10a826d65}" --- PASS: TestRaft_LeaderFail (0.66s) === RUN TestRaft_BehindFollower testLoggerAdapter verbose: server-407227cb-679c-1722-1648-68ec0d01f300: 2025-02-10T11:34:09.212Z [INFO] server-407227cb-679c-1722-1648-68ec0d01f300: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-407227cb-679c-1722-1648-68ec0d01f300 Address:407227cb-679c-1722-1648-68ec0d01f300} {Suffrage:Voter ID:server-0034cf17-a28c-6483-becb-3395b0525af8 Address:0034cf17-a28c-6483-becb-3395b0525af8} {Suffrage:Voter ID:server-507d33e5-8d88-8ed8-e12f-2009e6bab547 Address:507d33e5-8d88-8ed8-e12f-2009e6bab547}]" testing.go:149: server-407227cb-679c-1722-1648-68ec0d01f300: 2025-02-10T11:34:09.212Z [INFO] server-407227cb-679c-1722-1648-68ec0d01f300: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-407227cb-679c-1722-1648-68ec0d01f300 Address:407227cb-679c-1722-1648-68ec0d01f300} {Suffrage:Voter ID:server-0034cf17-a28c-6483-becb-3395b0525af8 Address:0034cf17-a28c-6483-becb-3395b0525af8} {Suffrage:Voter ID:server-507d33e5-8d88-8ed8-e12f-2009e6bab547 Address:507d33e5-8d88-8ed8-e12f-2009e6bab547}]" testLoggerAdapter verbose: server-407227cb-679c-1722-1648-68ec0d01f300: 2025-02-10T11:34:09.212Z [INFO] server-407227cb-679c-1722-1648-68ec0d01f300: entering follower state: follower="Node at 407227cb-679c-1722-1648-68ec0d01f300 [Follower]" leader= testing.go:149: server-407227cb-679c-1722-1648-68ec0d01f300: 2025-02-10T11:34:09.212Z [INFO] server-407227cb-679c-1722-1648-68ec0d01f300: entering follower state: follower="Node at 407227cb-679c-1722-1648-68ec0d01f300 [Follower]" leader= testLoggerAdapter verbose: server-0034cf17-a28c-6483-becb-3395b0525af8: 2025-02-10T11:34:09.212Z [INFO] server-0034cf17-a28c-6483-becb-3395b0525af8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-407227cb-679c-1722-1648-68ec0d01f300 Address:407227cb-679c-1722-1648-68ec0d01f300} {Suffrage:Voter ID:server-0034cf17-a28c-6483-becb-3395b0525af8 Address:0034cf17-a28c-6483-becb-3395b0525af8} {Suffrage:Voter ID:server-507d33e5-8d88-8ed8-e12f-2009e6bab547 Address:507d33e5-8d88-8ed8-e12f-2009e6bab547}]" testing.go:149: server-0034cf17-a28c-6483-becb-3395b0525af8: 2025-02-10T11:34:09.212Z [INFO] server-0034cf17-a28c-6483-becb-3395b0525af8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-407227cb-679c-1722-1648-68ec0d01f300 Address:407227cb-679c-1722-1648-68ec0d01f300} {Suffrage:Voter ID:server-0034cf17-a28c-6483-becb-3395b0525af8 Address:0034cf17-a28c-6483-becb-3395b0525af8} {Suffrage:Voter ID:server-507d33e5-8d88-8ed8-e12f-2009e6bab547 Address:507d33e5-8d88-8ed8-e12f-2009e6bab547}]" testLoggerAdapter verbose: server-0034cf17-a28c-6483-becb-3395b0525af8: 2025-02-10T11:34:09.212Z [INFO] server-0034cf17-a28c-6483-becb-3395b0525af8: entering follower state: follower="Node at 0034cf17-a28c-6483-becb-3395b0525af8 [Follower]" leader= testing.go:149: server-0034cf17-a28c-6483-becb-3395b0525af8: 2025-02-10T11:34:09.212Z [INFO] server-0034cf17-a28c-6483-becb-3395b0525af8: entering follower state: follower="Node at 0034cf17-a28c-6483-becb-3395b0525af8 [Follower]" leader= testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.212Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-407227cb-679c-1722-1648-68ec0d01f300 Address:407227cb-679c-1722-1648-68ec0d01f300} {Suffrage:Voter ID:server-0034cf17-a28c-6483-becb-3395b0525af8 Address:0034cf17-a28c-6483-becb-3395b0525af8} {Suffrage:Voter ID:server-507d33e5-8d88-8ed8-e12f-2009e6bab547 Address:507d33e5-8d88-8ed8-e12f-2009e6bab547}]" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.212Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-407227cb-679c-1722-1648-68ec0d01f300 Address:407227cb-679c-1722-1648-68ec0d01f300} {Suffrage:Voter ID:server-0034cf17-a28c-6483-becb-3395b0525af8 Address:0034cf17-a28c-6483-becb-3395b0525af8} {Suffrage:Voter ID:server-507d33e5-8d88-8ed8-e12f-2009e6bab547 Address:507d33e5-8d88-8ed8-e12f-2009e6bab547}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.212Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:09.212Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.212Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: entering follower state: follower="Node at 507d33e5-8d88-8ed8-e12f-2009e6bab547 [Follower]" leader= testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.212Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: entering follower state: follower="Node at 507d33e5-8d88-8ed8-e12f-2009e6bab547 [Follower]" leader= testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.266Z [WARN] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.266Z [WARN] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.266Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: entering candidate state: node="Node at 507d33e5-8d88-8ed8-e12f-2009e6bab547 [Candidate]" term=2 testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.266Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: entering candidate state: node="Node at 507d33e5-8d88-8ed8-e12f-2009e6bab547 [Candidate]" term=2 testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: election won: tally=2 testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: election won: tally=2 testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: entering leader state: leader="Node at 507d33e5-8d88-8ed8-e12f-2009e6bab547 [Leader]" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: entering leader state: leader="Node at 507d33e5-8d88-8ed8-e12f-2009e6bab547 [Leader]" testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: added peer, starting replication: peer=server-407227cb-679c-1722-1648-68ec0d01f300 testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: added peer, starting replication: peer=server-407227cb-679c-1722-1648-68ec0d01f300 testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: added peer, starting replication: peer=server-0034cf17-a28c-6483-becb-3395b0525af8 testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: added peer, starting replication: peer=server-0034cf17-a28c-6483-becb-3395b0525af8 testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: pipelining replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: pipelining replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: pipelining replication: peer="{Voter server-0034cf17-a28c-6483-becb-3395b0525af8 0034cf17-a28c-6483-becb-3395b0525af8}" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.267Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: pipelining replication: peer="{Voter server-0034cf17-a28c-6483-becb-3395b0525af8 0034cf17-a28c-6483-becb-3395b0525af8}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.372Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.267083972 +0000 UTC m=+5.635988274 (1 nodes), 54.165455ms from start of poll, 54.537884ms from cluster start. Timeout at 2025-02-10 11:34:09.37218561 +0000 UTC m=+5.741089960, 105.101686ms after stability testing.go:149: cluster: 2025-02-10T11:34:09.372Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.267083972 +0000 UTC m=+5.635988274 (1 nodes), 54.165455ms from start of poll, 54.537884ms from cluster start. Timeout at 2025-02-10 11:34:09.37218561 +0000 UTC m=+5.741089960, 105.101686ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.372Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:09.372Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.477Z [INFO] stable state for Follower reached at 2025-02-10 11:34:09.372343832 +0000 UTC m=+5.741248130 (2 nodes), 705ns from start of poll, 159.79774ms from cluster start. Timeout at 2025-02-10 11:34:09.477422299 +0000 UTC m=+5.846326595, 105.078465ms after stability testing.go:149: cluster: 2025-02-10T11:34:09.477Z [INFO] stable state for Follower reached at 2025-02-10 11:34:09.372343832 +0000 UTC m=+5.741248130 (2 nodes), 705ns from start of poll, 159.79774ms from cluster start. Timeout at 2025-02-10 11:34:09.477422299 +0000 UTC m=+5.846326595, 105.078465ms after stability testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.477Z [ERROR] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: failed to pipeline appendEntries: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" error="append pipeline closed" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.477Z [ERROR] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: failed to pipeline appendEntries: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" error="append pipeline closed" testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.477Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: aborting pipeline replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.477Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: aborting pipeline replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.477Z [ERROR] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: failed to appendEntries to: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" error="failed to connect to peer: 407227cb-679c-1722-1648-68ec0d01f300" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.477Z [ERROR] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: failed to appendEntries to: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" error="failed to connect to peer: 407227cb-679c-1722-1648-68ec0d01f300" raft_test.go:420: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.487Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: pipelining replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.487Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: pipelining replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.498Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:09.498Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.604Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.498957666 +0000 UTC m=+5.867861968 (1 nodes), 492ns from start of poll, 286.411578ms from cluster start. Timeout at 2025-02-10 11:34:09.604074949 +0000 UTC m=+5.972979246, 105.117278ms after stability testing.go:149: cluster: 2025-02-10T11:34:09.604Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.498957666 +0000 UTC m=+5.867861968 (1 nodes), 492ns from start of poll, 286.411578ms from cluster start. Timeout at 2025-02-10 11:34:09.604074949 +0000 UTC m=+5.972979246, 105.117278ms after stability testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.604Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: aborting pipeline replication: peer="{Voter server-0034cf17-a28c-6483-becb-3395b0525af8 0034cf17-a28c-6483-becb-3395b0525af8}" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.604Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: aborting pipeline replication: peer="{Voter server-0034cf17-a28c-6483-becb-3395b0525af8 0034cf17-a28c-6483-becb-3395b0525af8}" testLoggerAdapter verbose: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.604Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: aborting pipeline replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" testing.go:149: server-507d33e5-8d88-8ed8-e12f-2009e6bab547: 2025-02-10T11:34:09.604Z [INFO] server-507d33e5-8d88-8ed8-e12f-2009e6bab547: aborting pipeline replication: peer="{Voter server-407227cb-679c-1722-1648-68ec0d01f300 407227cb-679c-1722-1648-68ec0d01f300}" --- PASS: TestRaft_BehindFollower (0.39s) === RUN TestRaft_ApplyNonLeader testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.605Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 Address:7bbbcdca-ea19-b91e-9337-e10d3b5a4b73} {Suffrage:Voter ID:server-359cdbde-e211-094c-efe1-de0836c5a9ac Address:359cdbde-e211-094c-efe1-de0836c5a9ac} {Suffrage:Voter ID:server-6490b056-bba7-4169-5c01-b53c9086aac3 Address:6490b056-bba7-4169-5c01-b53c9086aac3}]" testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.605Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 Address:7bbbcdca-ea19-b91e-9337-e10d3b5a4b73} {Suffrage:Voter ID:server-359cdbde-e211-094c-efe1-de0836c5a9ac Address:359cdbde-e211-094c-efe1-de0836c5a9ac} {Suffrage:Voter ID:server-6490b056-bba7-4169-5c01-b53c9086aac3 Address:6490b056-bba7-4169-5c01-b53c9086aac3}]" testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.605Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: entering follower state: follower="Node at 7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 [Follower]" leader= testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.605Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: entering follower state: follower="Node at 7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 [Follower]" leader= testLoggerAdapter verbose: server-359cdbde-e211-094c-efe1-de0836c5a9ac: 2025-02-10T11:34:09.605Z [INFO] server-359cdbde-e211-094c-efe1-de0836c5a9ac: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 Address:7bbbcdca-ea19-b91e-9337-e10d3b5a4b73} {Suffrage:Voter ID:server-359cdbde-e211-094c-efe1-de0836c5a9ac Address:359cdbde-e211-094c-efe1-de0836c5a9ac} {Suffrage:Voter ID:server-6490b056-bba7-4169-5c01-b53c9086aac3 Address:6490b056-bba7-4169-5c01-b53c9086aac3}]" testing.go:149: server-359cdbde-e211-094c-efe1-de0836c5a9ac: 2025-02-10T11:34:09.605Z [INFO] server-359cdbde-e211-094c-efe1-de0836c5a9ac: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 Address:7bbbcdca-ea19-b91e-9337-e10d3b5a4b73} {Suffrage:Voter ID:server-359cdbde-e211-094c-efe1-de0836c5a9ac Address:359cdbde-e211-094c-efe1-de0836c5a9ac} {Suffrage:Voter ID:server-6490b056-bba7-4169-5c01-b53c9086aac3 Address:6490b056-bba7-4169-5c01-b53c9086aac3}]" testLoggerAdapter verbose: server-359cdbde-e211-094c-efe1-de0836c5a9ac: 2025-02-10T11:34:09.605Z [INFO] server-359cdbde-e211-094c-efe1-de0836c5a9ac: entering follower state: follower="Node at 359cdbde-e211-094c-efe1-de0836c5a9ac [Follower]" leader= testing.go:149: server-359cdbde-e211-094c-efe1-de0836c5a9ac: 2025-02-10T11:34:09.605Z [INFO] server-359cdbde-e211-094c-efe1-de0836c5a9ac: entering follower state: follower="Node at 359cdbde-e211-094c-efe1-de0836c5a9ac [Follower]" leader= testLoggerAdapter verbose: server-6490b056-bba7-4169-5c01-b53c9086aac3: 2025-02-10T11:34:09.605Z [INFO] server-6490b056-bba7-4169-5c01-b53c9086aac3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 Address:7bbbcdca-ea19-b91e-9337-e10d3b5a4b73} {Suffrage:Voter ID:server-359cdbde-e211-094c-efe1-de0836c5a9ac Address:359cdbde-e211-094c-efe1-de0836c5a9ac} {Suffrage:Voter ID:server-6490b056-bba7-4169-5c01-b53c9086aac3 Address:6490b056-bba7-4169-5c01-b53c9086aac3}]" testing.go:149: server-6490b056-bba7-4169-5c01-b53c9086aac3: 2025-02-10T11:34:09.605Z [INFO] server-6490b056-bba7-4169-5c01-b53c9086aac3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 Address:7bbbcdca-ea19-b91e-9337-e10d3b5a4b73} {Suffrage:Voter ID:server-359cdbde-e211-094c-efe1-de0836c5a9ac Address:359cdbde-e211-094c-efe1-de0836c5a9ac} {Suffrage:Voter ID:server-6490b056-bba7-4169-5c01-b53c9086aac3 Address:6490b056-bba7-4169-5c01-b53c9086aac3}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.605Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:09.605Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-6490b056-bba7-4169-5c01-b53c9086aac3: 2025-02-10T11:34:09.605Z [INFO] server-6490b056-bba7-4169-5c01-b53c9086aac3: entering follower state: follower="Node at 6490b056-bba7-4169-5c01-b53c9086aac3 [Follower]" leader= testing.go:149: server-6490b056-bba7-4169-5c01-b53c9086aac3: 2025-02-10T11:34:09.605Z [INFO] server-6490b056-bba7-4169-5c01-b53c9086aac3: entering follower state: follower="Node at 6490b056-bba7-4169-5c01-b53c9086aac3 [Follower]" leader= testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [WARN] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [WARN] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: entering candidate state: node="Node at 7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 [Candidate]" term=2 testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: entering candidate state: node="Node at 7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 [Candidate]" term=2 testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: election won: tally=2 testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: election won: tally=2 testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: entering leader state: leader="Node at 7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 [Leader]" testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: entering leader state: leader="Node at 7bbbcdca-ea19-b91e-9337-e10d3b5a4b73 [Leader]" testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: added peer, starting replication: peer=server-359cdbde-e211-094c-efe1-de0836c5a9ac testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: added peer, starting replication: peer=server-359cdbde-e211-094c-efe1-de0836c5a9ac testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: added peer, starting replication: peer=server-6490b056-bba7-4169-5c01-b53c9086aac3 testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: added peer, starting replication: peer=server-6490b056-bba7-4169-5c01-b53c9086aac3 testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: pipelining replication: peer="{Voter server-6490b056-bba7-4169-5c01-b53c9086aac3 6490b056-bba7-4169-5c01-b53c9086aac3}" testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: pipelining replication: peer="{Voter server-6490b056-bba7-4169-5c01-b53c9086aac3 6490b056-bba7-4169-5c01-b53c9086aac3}" testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: pipelining replication: peer="{Voter server-359cdbde-e211-094c-efe1-de0836c5a9ac 359cdbde-e211-094c-efe1-de0836c5a9ac}" testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.667Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: pipelining replication: peer="{Voter server-359cdbde-e211-094c-efe1-de0836c5a9ac 359cdbde-e211-094c-efe1-de0836c5a9ac}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.772Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.667647891 +0000 UTC m=+6.036552193 (1 nodes), 62.141844ms from start of poll, 62.565327ms from cluster start. Timeout at 2025-02-10 11:34:09.772720709 +0000 UTC m=+6.141625004, 105.072811ms after stability testing.go:149: cluster: 2025-02-10T11:34:09.772Z [INFO] stable state for Leader reached at 2025-02-10 11:34:09.667647891 +0000 UTC m=+6.036552193 (1 nodes), 62.141844ms from start of poll, 62.565327ms from cluster start. Timeout at 2025-02-10 11:34:09.772720709 +0000 UTC m=+6.141625004, 105.072811ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.772Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:09.772Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.877Z [INFO] stable state for Follower reached at 2025-02-10 11:34:09.772796722 +0000 UTC m=+6.141701022 (2 nodes), 703ns from start of poll, 167.714156ms from cluster start. Timeout at 2025-02-10 11:34:09.877899954 +0000 UTC m=+6.246804304, 105.103282ms after stability testing.go:149: cluster: 2025-02-10T11:34:09.877Z [INFO] stable state for Follower reached at 2025-02-10 11:34:09.772796722 +0000 UTC m=+6.141701022 (2 nodes), 703ns from start of poll, 167.714156ms from cluster start. Timeout at 2025-02-10 11:34:09.877899954 +0000 UTC m=+6.246804304, 105.103282ms after stability testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.878Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: aborting pipeline replication: peer="{Voter server-6490b056-bba7-4169-5c01-b53c9086aac3 6490b056-bba7-4169-5c01-b53c9086aac3}" testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.878Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: aborting pipeline replication: peer="{Voter server-6490b056-bba7-4169-5c01-b53c9086aac3 6490b056-bba7-4169-5c01-b53c9086aac3}" testLoggerAdapter verbose: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.878Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: aborting pipeline replication: peer="{Voter server-359cdbde-e211-094c-efe1-de0836c5a9ac 359cdbde-e211-094c-efe1-de0836c5a9ac}" testing.go:149: server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: 2025-02-10T11:34:09.878Z [INFO] server-7bbbcdca-ea19-b91e-9337-e10d3b5a4b73: aborting pipeline replication: peer="{Voter server-359cdbde-e211-094c-efe1-de0836c5a9ac 359cdbde-e211-094c-efe1-de0836c5a9ac}" --- PASS: TestRaft_ApplyNonLeader (0.27s) === RUN TestRaft_ApplyConcurrent testLoggerAdapter verbose: server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: 2025-02-10T11:34:09.879Z [INFO] server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 Address:f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0} {Suffrage:Voter ID:server-d96165b3-918a-7fad-e52b-6753fdcccd11 Address:d96165b3-918a-7fad-e52b-6753fdcccd11} {Suffrage:Voter ID:server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 Address:89c99f67-25f9-8b4e-0263-5cb1ce6de255}]" testing.go:149: server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: 2025-02-10T11:34:09.879Z [INFO] server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 Address:f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0} {Suffrage:Voter ID:server-d96165b3-918a-7fad-e52b-6753fdcccd11 Address:d96165b3-918a-7fad-e52b-6753fdcccd11} {Suffrage:Voter ID:server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 Address:89c99f67-25f9-8b4e-0263-5cb1ce6de255}]" testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:09.879Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 Address:f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0} {Suffrage:Voter ID:server-d96165b3-918a-7fad-e52b-6753fdcccd11 Address:d96165b3-918a-7fad-e52b-6753fdcccd11} {Suffrage:Voter ID:server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 Address:89c99f67-25f9-8b4e-0263-5cb1ce6de255}]" testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:09.879Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 Address:f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0} {Suffrage:Voter ID:server-d96165b3-918a-7fad-e52b-6753fdcccd11 Address:d96165b3-918a-7fad-e52b-6753fdcccd11} {Suffrage:Voter ID:server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 Address:89c99f67-25f9-8b4e-0263-5cb1ce6de255}]" testLoggerAdapter verbose: server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: 2025-02-10T11:34:09.879Z [INFO] server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 Address:f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0} {Suffrage:Voter ID:server-d96165b3-918a-7fad-e52b-6753fdcccd11 Address:d96165b3-918a-7fad-e52b-6753fdcccd11} {Suffrage:Voter ID:server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 Address:89c99f67-25f9-8b4e-0263-5cb1ce6de255}]" testing.go:149: server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: 2025-02-10T11:34:09.879Z [INFO] server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 Address:f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0} {Suffrage:Voter ID:server-d96165b3-918a-7fad-e52b-6753fdcccd11 Address:d96165b3-918a-7fad-e52b-6753fdcccd11} {Suffrage:Voter ID:server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 Address:89c99f67-25f9-8b4e-0263-5cb1ce6de255}]" testLoggerAdapter verbose: server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: 2025-02-10T11:34:09.879Z [INFO] server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: entering follower state: follower="Node at f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 [Follower]" leader= testing.go:149: server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: 2025-02-10T11:34:09.879Z [INFO] server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0: entering follower state: follower="Node at f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 [Follower]" leader= testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:09.879Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: entering follower state: follower="Node at d96165b3-918a-7fad-e52b-6753fdcccd11 [Follower]" leader= testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:09.879Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: entering follower state: follower="Node at d96165b3-918a-7fad-e52b-6753fdcccd11 [Follower]" leader= testLoggerAdapter verbose: cluster: 2025-02-10T11:34:09.879Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:09.879Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: 2025-02-10T11:34:09.880Z [INFO] server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: entering follower state: follower="Node at 89c99f67-25f9-8b4e-0263-5cb1ce6de255 [Follower]" leader= testing.go:149: server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: 2025-02-10T11:34:09.880Z [INFO] server-89c99f67-25f9-8b4e-0263-5cb1ce6de255: entering follower state: follower="Node at 89c99f67-25f9-8b4e-0263-5cb1ce6de255 [Follower]" leader= testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [WARN] server-d96165b3-918a-7fad-e52b-6753fdcccd11: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [WARN] server-d96165b3-918a-7fad-e52b-6753fdcccd11: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: entering candidate state: node="Node at d96165b3-918a-7fad-e52b-6753fdcccd11 [Candidate]" term=2 testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: entering candidate state: node="Node at d96165b3-918a-7fad-e52b-6753fdcccd11 [Candidate]" term=2 testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: election won: tally=2 testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: election won: tally=2 testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: entering leader state: leader="Node at d96165b3-918a-7fad-e52b-6753fdcccd11 [Leader]" testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: entering leader state: leader="Node at d96165b3-918a-7fad-e52b-6753fdcccd11 [Leader]" testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: added peer, starting replication: peer=server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: added peer, starting replication: peer=server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: added peer, starting replication: peer=server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: added peer, starting replication: peer=server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: pipelining replication: peer="{Voter server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0}" testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: pipelining replication: peer="{Voter server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0}" testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: pipelining replication: peer="{Voter server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 89c99f67-25f9-8b4e-0263-5cb1ce6de255}" testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.048Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: pipelining replication: peer="{Voter server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 89c99f67-25f9-8b4e-0263-5cb1ce6de255}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.253Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.048169172 +0000 UTC m=+6.417073472 (1 nodes), 168.124617ms from start of poll, 169.18335ms from cluster start. Timeout at 2025-02-10 11:34:10.253270139 +0000 UTC m=+6.622174492, 205.10102ms after stability testing.go:149: cluster: 2025-02-10T11:34:10.253Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.048169172 +0000 UTC m=+6.417073472 (1 nodes), 168.124617ms from start of poll, 169.18335ms from cluster start. Timeout at 2025-02-10 11:34:10.253270139 +0000 UTC m=+6.622174492, 205.10102ms after stability testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.264Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: aborting pipeline replication: peer="{Voter server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 89c99f67-25f9-8b4e-0263-5cb1ce6de255}" testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.264Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: aborting pipeline replication: peer="{Voter server-89c99f67-25f9-8b4e-0263-5cb1ce6de255 89c99f67-25f9-8b4e-0263-5cb1ce6de255}" testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.264Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: aborting pipeline replication: peer="{Voter server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0}" testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.264Z [INFO] server-d96165b3-918a-7fad-e52b-6753fdcccd11: aborting pipeline replication: peer="{Voter server-f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0}" testLoggerAdapter verbose: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.315Z [ERROR] server-d96165b3-918a-7fad-e52b-6753fdcccd11: failed to heartbeat to: peer=f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 error="command timed out" testing.go:149: server-d96165b3-918a-7fad-e52b-6753fdcccd11: 2025-02-10T11:34:10.315Z [ERROR] server-d96165b3-918a-7fad-e52b-6753fdcccd11: failed to heartbeat to: peer=f80a1c08-7986-b2ef-56a9-bc6e00b4c5d0 error="command timed out" --- PASS: TestRaft_ApplyConcurrent (0.44s) === RUN TestRaft_ApplyConcurrent_Timeout testLoggerAdapter verbose: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.315Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2d169bdb-3864-ac4f-f013-4c94a790a8a0 Address:2d169bdb-3864-ac4f-f013-4c94a790a8a0}]" testing.go:149: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.315Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2d169bdb-3864-ac4f-f013-4c94a790a8a0 Address:2d169bdb-3864-ac4f-f013-4c94a790a8a0}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.316Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:10.316Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.316Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: entering follower state: follower="Node at 2d169bdb-3864-ac4f-f013-4c94a790a8a0 [Follower]" leader= testing.go:149: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.316Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: entering follower state: follower="Node at 2d169bdb-3864-ac4f-f013-4c94a790a8a0 [Follower]" leader= testLoggerAdapter verbose: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [WARN] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [WARN] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: entering candidate state: node="Node at 2d169bdb-3864-ac4f-f013-4c94a790a8a0 [Candidate]" term=2 testing.go:149: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: entering candidate state: node="Node at 2d169bdb-3864-ac4f-f013-4c94a790a8a0 [Candidate]" term=2 testLoggerAdapter verbose: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: election won: tally=1 testing.go:149: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: election won: tally=1 testLoggerAdapter verbose: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: entering leader state: leader="Node at 2d169bdb-3864-ac4f-f013-4c94a790a8a0 [Leader]" testing.go:149: server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: 2025-02-10T11:34:10.431Z [INFO] server-2d169bdb-3864-ac4f-f013-4c94a790a8a0: entering leader state: leader="Node at 2d169bdb-3864-ac4f-f013-4c94a790a8a0 [Leader]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.632Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.431892344 +0000 UTC m=+6.800796710 (1 nodes), 115.843764ms from start of poll, 116.007607ms from cluster start. Timeout at 2025-02-10 11:34:10.632965142 +0000 UTC m=+7.001869448, 201.072738ms after stability testing.go:149: cluster: 2025-02-10T11:34:10.632Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.431892344 +0000 UTC m=+6.800796710 (1 nodes), 115.843764ms from start of poll, 116.007607ms from cluster start. Timeout at 2025-02-10 11:34:10.632965142 +0000 UTC m=+7.001869448, 201.072738ms after stability --- PASS: TestRaft_ApplyConcurrent_Timeout (0.32s) === RUN TestRaft_JoinNode testLoggerAdapter verbose: server-543b908b-aed1-0232-f10d-2d27521cd86e: 2025-02-10T11:34:10.633Z [INFO] server-543b908b-aed1-0232-f10d-2d27521cd86e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-543b908b-aed1-0232-f10d-2d27521cd86e Address:543b908b-aed1-0232-f10d-2d27521cd86e} {Suffrage:Voter ID:server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e Address:bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e}]" testing.go:149: server-543b908b-aed1-0232-f10d-2d27521cd86e: 2025-02-10T11:34:10.633Z [INFO] server-543b908b-aed1-0232-f10d-2d27521cd86e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-543b908b-aed1-0232-f10d-2d27521cd86e Address:543b908b-aed1-0232-f10d-2d27521cd86e} {Suffrage:Voter ID:server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e Address:bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e}]" testLoggerAdapter verbose: server-543b908b-aed1-0232-f10d-2d27521cd86e: 2025-02-10T11:34:10.633Z [INFO] server-543b908b-aed1-0232-f10d-2d27521cd86e: entering follower state: follower="Node at 543b908b-aed1-0232-f10d-2d27521cd86e [Follower]" leader= testing.go:149: server-543b908b-aed1-0232-f10d-2d27521cd86e: 2025-02-10T11:34:10.633Z [INFO] server-543b908b-aed1-0232-f10d-2d27521cd86e: entering follower state: follower="Node at 543b908b-aed1-0232-f10d-2d27521cd86e [Follower]" leader= testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.633Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-543b908b-aed1-0232-f10d-2d27521cd86e Address:543b908b-aed1-0232-f10d-2d27521cd86e} {Suffrage:Voter ID:server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e Address:bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e}]" testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.633Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-543b908b-aed1-0232-f10d-2d27521cd86e Address:543b908b-aed1-0232-f10d-2d27521cd86e} {Suffrage:Voter ID:server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e Address:bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e}]" testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.633Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: entering follower state: follower="Node at bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e [Follower]" leader= testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.633Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: entering follower state: follower="Node at bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e [Follower]" leader= testLoggerAdapter verbose: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.634Z [INFO] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: initial configuration: index=0 servers=[] testing.go:149: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.634Z [INFO] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.634Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.634Z [INFO] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: entering follower state: follower="Node at a125ff63-dc6d-ddfe-592a-7f0b1854e881 [Follower]" leader= testing.go:149: cluster: 2025-02-10T11:34:10.634Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.634Z [INFO] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: entering follower state: follower="Node at a125ff63-dc6d-ddfe-592a-7f0b1854e881 [Follower]" leader= testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [WARN] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [WARN] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: entering candidate state: node="Node at bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e [Candidate]" term=2 testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: entering candidate state: node="Node at bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e [Candidate]" term=2 testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: election won: tally=2 testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: election won: tally=2 testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: entering leader state: leader="Node at bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e [Leader]" testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: entering leader state: leader="Node at bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e [Leader]" testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: added peer, starting replication: peer=server-543b908b-aed1-0232-f10d-2d27521cd86e testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: added peer, starting replication: peer=server-543b908b-aed1-0232-f10d-2d27521cd86e testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: pipelining replication: peer="{Voter server-543b908b-aed1-0232-f10d-2d27521cd86e 543b908b-aed1-0232-f10d-2d27521cd86e}" testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.692Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: pipelining replication: peer="{Voter server-543b908b-aed1-0232-f10d-2d27521cd86e 543b908b-aed1-0232-f10d-2d27521cd86e}" testLoggerAdapter verbose: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.701Z [WARN] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: no known peers, aborting election testing.go:149: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.701Z [WARN] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: no known peers, aborting election testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.797Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.692601454 +0000 UTC m=+7.061505758 (1 nodes), 58.466846ms from start of poll, 59.044767ms from cluster start. Timeout at 2025-02-10 11:34:10.797704606 +0000 UTC m=+7.166608952, 105.103194ms after stability testing.go:149: cluster: 2025-02-10T11:34:10.797Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.692601454 +0000 UTC m=+7.061505758 (1 nodes), 58.466846ms from start of poll, 59.044767ms from cluster start. Timeout at 2025-02-10 11:34:10.797704606 +0000 UTC m=+7.166608952, 105.103194ms after stability testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.797Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: updating configuration: command=AddStaging server-id=server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 server-addr=a125ff63-dc6d-ddfe-592a-7f0b1854e881 servers="[{Suffrage:Voter ID:server-543b908b-aed1-0232-f10d-2d27521cd86e Address:543b908b-aed1-0232-f10d-2d27521cd86e} {Suffrage:Voter ID:server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e Address:bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e} {Suffrage:Voter ID:server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 Address:a125ff63-dc6d-ddfe-592a-7f0b1854e881}]" testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.797Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: updating configuration: command=AddStaging server-id=server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 server-addr=a125ff63-dc6d-ddfe-592a-7f0b1854e881 servers="[{Suffrage:Voter ID:server-543b908b-aed1-0232-f10d-2d27521cd86e Address:543b908b-aed1-0232-f10d-2d27521cd86e} {Suffrage:Voter ID:server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e Address:bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e} {Suffrage:Voter ID:server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 Address:a125ff63-dc6d-ddfe-592a-7f0b1854e881}]" testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.798Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: added peer, starting replication: peer=server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.798Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: added peer, starting replication: peer=server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 testLoggerAdapter verbose: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.798Z [WARN] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: 2025-02-10T11:34:10.798Z [WARN] server-a125ff63-dc6d-ddfe-592a-7f0b1854e881: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.798Z [WARN] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: appendEntries rejected, sending older logs: peer="{Voter server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 a125ff63-dc6d-ddfe-592a-7f0b1854e881}" next=1 testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.798Z [WARN] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: appendEntries rejected, sending older logs: peer="{Voter server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 a125ff63-dc6d-ddfe-592a-7f0b1854e881}" next=1 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.798Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:10.798Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.798Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: pipelining replication: peer="{Voter server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 a125ff63-dc6d-ddfe-592a-7f0b1854e881}" testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.798Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: pipelining replication: peer="{Voter server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 a125ff63-dc6d-ddfe-592a-7f0b1854e881}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.903Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.798185654 +0000 UTC m=+7.167089955 (1 nodes), 378ns from start of poll, 164.628964ms from cluster start. Timeout at 2025-02-10 11:34:10.903258966 +0000 UTC m=+7.272163259, 105.073304ms after stability testing.go:149: cluster: 2025-02-10T11:34:10.903Z [INFO] stable state for Leader reached at 2025-02-10 11:34:10.798185654 +0000 UTC m=+7.167089955 (1 nodes), 378ns from start of poll, 164.628964ms from cluster start. Timeout at 2025-02-10 11:34:10.903258966 +0000 UTC m=+7.272163259, 105.073304ms after stability testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.903Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: aborting pipeline replication: peer="{Voter server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 a125ff63-dc6d-ddfe-592a-7f0b1854e881}" testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.903Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: aborting pipeline replication: peer="{Voter server-a125ff63-dc6d-ddfe-592a-7f0b1854e881 a125ff63-dc6d-ddfe-592a-7f0b1854e881}" testLoggerAdapter verbose: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.903Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: aborting pipeline replication: peer="{Voter server-543b908b-aed1-0232-f10d-2d27521cd86e 543b908b-aed1-0232-f10d-2d27521cd86e}" testing.go:149: server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: 2025-02-10T11:34:10.903Z [INFO] server-bcfc8e4a-05d0-5c4d-adce-b1a181e5dc9e: aborting pipeline replication: peer="{Voter server-543b908b-aed1-0232-f10d-2d27521cd86e 543b908b-aed1-0232-f10d-2d27521cd86e}" --- PASS: TestRaft_JoinNode (0.27s) === RUN TestRaft_JoinNode_ConfigStore testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:10.904Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff6b7497-89ec-f2d7-c906-c5218d0fc273 Address:ff6b7497-89ec-f2d7-c906-c5218d0fc273}]" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:10.904Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff6b7497-89ec-f2d7-c906-c5218d0fc273 Address:ff6b7497-89ec-f2d7-c906-c5218d0fc273}]" testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:10.904Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: entering follower state: follower="Node at ff6b7497-89ec-f2d7-c906-c5218d0fc273 [Follower]" leader= testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:10.904Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: entering follower state: follower="Node at ff6b7497-89ec-f2d7-c906-c5218d0fc273 [Follower]" leader= testLoggerAdapter verbose: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:10.904Z [INFO] server-011d2ffb-116f-044e-01a7-99293021b44a: initial configuration: index=0 servers=[] testing.go:149: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:10.904Z [INFO] server-011d2ffb-116f-044e-01a7-99293021b44a: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:10.904Z [INFO] server-011d2ffb-116f-044e-01a7-99293021b44a: entering follower state: follower="Node at 011d2ffb-116f-044e-01a7-99293021b44a [Follower]" leader= testing.go:149: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:10.904Z [INFO] server-011d2ffb-116f-044e-01a7-99293021b44a: entering follower state: follower="Node at 011d2ffb-116f-044e-01a7-99293021b44a [Follower]" leader= testLoggerAdapter verbose: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:10.904Z [INFO] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: initial configuration: index=0 servers=[] testing.go:149: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:10.904Z [INFO] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2025-02-10T11:34:10.904Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:10.904Z [INFO] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: entering follower state: follower="Node at 80c460e7-ade7-622a-cbb2-4ecf276387c2 [Follower]" leader= testing.go:149: cluster: 2025-02-10T11:34:10.904Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:10.904Z [INFO] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: entering follower state: follower="Node at 80c460e7-ade7-622a-cbb2-4ecf276387c2 [Follower]" leader= testLoggerAdapter verbose: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:10.976Z [WARN] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: no known peers, aborting election testing.go:149: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:10.976Z [WARN] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: no known peers, aborting election testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [WARN] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [WARN] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: entering candidate state: node="Node at ff6b7497-89ec-f2d7-c906-c5218d0fc273 [Candidate]" term=2 testLoggerAdapter verbose: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:11.003Z [WARN] server-011d2ffb-116f-044e-01a7-99293021b44a: no known peers, aborting election testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: entering candidate state: node="Node at ff6b7497-89ec-f2d7-c906-c5218d0fc273 [Candidate]" term=2 testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: election won: tally=1 testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: election won: tally=1 testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: entering leader state: leader="Node at ff6b7497-89ec-f2d7-c906-c5218d0fc273 [Leader]" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.003Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: entering leader state: leader="Node at ff6b7497-89ec-f2d7-c906-c5218d0fc273 [Leader]" testing.go:149: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:11.003Z [WARN] server-011d2ffb-116f-044e-01a7-99293021b44a: no known peers, aborting election testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.108Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.003244339 +0000 UTC m=+7.372148670 (1 nodes), 98.520108ms from start of poll, 99.127421ms from cluster start. Timeout at 2025-02-10 11:34:11.108322021 +0000 UTC m=+7.477226317, 105.077647ms after stability testing.go:149: cluster: 2025-02-10T11:34:11.108Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.003244339 +0000 UTC m=+7.372148670 (1 nodes), 98.520108ms from start of poll, 99.127421ms from cluster start. Timeout at 2025-02-10 11:34:11.108322021 +0000 UTC m=+7.477226317, 105.077647ms after stability testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: updating configuration: command=AddStaging server-id=server-011d2ffb-116f-044e-01a7-99293021b44a server-addr=011d2ffb-116f-044e-01a7-99293021b44a servers="[{Suffrage:Voter ID:server-ff6b7497-89ec-f2d7-c906-c5218d0fc273 Address:ff6b7497-89ec-f2d7-c906-c5218d0fc273} {Suffrage:Voter ID:server-011d2ffb-116f-044e-01a7-99293021b44a Address:011d2ffb-116f-044e-01a7-99293021b44a}]" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: updating configuration: command=AddStaging server-id=server-011d2ffb-116f-044e-01a7-99293021b44a server-addr=011d2ffb-116f-044e-01a7-99293021b44a servers="[{Suffrage:Voter ID:server-ff6b7497-89ec-f2d7-c906-c5218d0fc273 Address:ff6b7497-89ec-f2d7-c906-c5218d0fc273} {Suffrage:Voter ID:server-011d2ffb-116f-044e-01a7-99293021b44a Address:011d2ffb-116f-044e-01a7-99293021b44a}]" testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: added peer, starting replication: peer=server-011d2ffb-116f-044e-01a7-99293021b44a testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: added peer, starting replication: peer=server-011d2ffb-116f-044e-01a7-99293021b44a testLoggerAdapter verbose: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:11.108Z [WARN] server-011d2ffb-116f-044e-01a7-99293021b44a: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-011d2ffb-116f-044e-01a7-99293021b44a: 2025-02-10T11:34:11.108Z [WARN] server-011d2ffb-116f-044e-01a7-99293021b44a: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [WARN] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: appendEntries rejected, sending older logs: peer="{Voter server-011d2ffb-116f-044e-01a7-99293021b44a 011d2ffb-116f-044e-01a7-99293021b44a}" next=1 testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [WARN] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: appendEntries rejected, sending older logs: peer="{Voter server-011d2ffb-116f-044e-01a7-99293021b44a 011d2ffb-116f-044e-01a7-99293021b44a}" next=1 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.108Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:11.108Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: pipelining replication: peer="{Voter server-011d2ffb-116f-044e-01a7-99293021b44a 011d2ffb-116f-044e-01a7-99293021b44a}" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.108Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: pipelining replication: peer="{Voter server-011d2ffb-116f-044e-01a7-99293021b44a 011d2ffb-116f-044e-01a7-99293021b44a}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.213Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.10865151 +0000 UTC m=+7.477555836 (1 nodes), 669ns from start of poll, 204.534587ms from cluster start. Timeout at 2025-02-10 11:34:11.213719642 +0000 UTC m=+7.582623939, 105.068103ms after stability testing.go:149: cluster: 2025-02-10T11:34:11.213Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.10865151 +0000 UTC m=+7.477555836 (1 nodes), 669ns from start of poll, 204.534587ms from cluster start. Timeout at 2025-02-10 11:34:11.213719642 +0000 UTC m=+7.582623939, 105.068103ms after stability testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.213Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: updating configuration: command=AddStaging server-id=server-80c460e7-ade7-622a-cbb2-4ecf276387c2 server-addr=80c460e7-ade7-622a-cbb2-4ecf276387c2 servers="[{Suffrage:Voter ID:server-ff6b7497-89ec-f2d7-c906-c5218d0fc273 Address:ff6b7497-89ec-f2d7-c906-c5218d0fc273} {Suffrage:Voter ID:server-011d2ffb-116f-044e-01a7-99293021b44a Address:011d2ffb-116f-044e-01a7-99293021b44a} {Suffrage:Voter ID:server-80c460e7-ade7-622a-cbb2-4ecf276387c2 Address:80c460e7-ade7-622a-cbb2-4ecf276387c2}]" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.213Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: updating configuration: command=AddStaging server-id=server-80c460e7-ade7-622a-cbb2-4ecf276387c2 server-addr=80c460e7-ade7-622a-cbb2-4ecf276387c2 servers="[{Suffrage:Voter ID:server-ff6b7497-89ec-f2d7-c906-c5218d0fc273 Address:ff6b7497-89ec-f2d7-c906-c5218d0fc273} {Suffrage:Voter ID:server-011d2ffb-116f-044e-01a7-99293021b44a Address:011d2ffb-116f-044e-01a7-99293021b44a} {Suffrage:Voter ID:server-80c460e7-ade7-622a-cbb2-4ecf276387c2 Address:80c460e7-ade7-622a-cbb2-4ecf276387c2}]" testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.213Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: added peer, starting replication: peer=server-80c460e7-ade7-622a-cbb2-4ecf276387c2 testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.213Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: added peer, starting replication: peer=server-80c460e7-ade7-622a-cbb2-4ecf276387c2 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.214Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:11.214Z [WARN] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: failed to get previous log: previous-index=4 last-index=0 error="log not found" testing.go:149: cluster: 2025-02-10T11:34:11.214Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-80c460e7-ade7-622a-cbb2-4ecf276387c2: 2025-02-10T11:34:11.214Z [WARN] server-80c460e7-ade7-622a-cbb2-4ecf276387c2: failed to get previous log: previous-index=4 last-index=0 error="log not found" testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.214Z [WARN] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: appendEntries rejected, sending older logs: peer="{Voter server-80c460e7-ade7-622a-cbb2-4ecf276387c2 80c460e7-ade7-622a-cbb2-4ecf276387c2}" next=1 testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.214Z [WARN] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: appendEntries rejected, sending older logs: peer="{Voter server-80c460e7-ade7-622a-cbb2-4ecf276387c2 80c460e7-ade7-622a-cbb2-4ecf276387c2}" next=1 testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.214Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: pipelining replication: peer="{Voter server-80c460e7-ade7-622a-cbb2-4ecf276387c2 80c460e7-ade7-622a-cbb2-4ecf276387c2}" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.214Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: pipelining replication: peer="{Voter server-80c460e7-ade7-622a-cbb2-4ecf276387c2 80c460e7-ade7-622a-cbb2-4ecf276387c2}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.319Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.214047749 +0000 UTC m=+7.582952048 (1 nodes), 537ns from start of poll, 309.930799ms from cluster start. Timeout at 2025-02-10 11:34:11.319120564 +0000 UTC m=+7.688024859, 105.072811ms after stability testing.go:149: cluster: 2025-02-10T11:34:11.319Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.214047749 +0000 UTC m=+7.582952048 (1 nodes), 537ns from start of poll, 309.930799ms from cluster start. Timeout at 2025-02-10 11:34:11.319120564 +0000 UTC m=+7.688024859, 105.072811ms after stability testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.319Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: aborting pipeline replication: peer="{Voter server-80c460e7-ade7-622a-cbb2-4ecf276387c2 80c460e7-ade7-622a-cbb2-4ecf276387c2}" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.319Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: aborting pipeline replication: peer="{Voter server-80c460e7-ade7-622a-cbb2-4ecf276387c2 80c460e7-ade7-622a-cbb2-4ecf276387c2}" testLoggerAdapter verbose: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.319Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: aborting pipeline replication: peer="{Voter server-011d2ffb-116f-044e-01a7-99293021b44a 011d2ffb-116f-044e-01a7-99293021b44a}" testing.go:149: server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: 2025-02-10T11:34:11.319Z [INFO] server-ff6b7497-89ec-f2d7-c906-c5218d0fc273: aborting pipeline replication: peer="{Voter server-011d2ffb-116f-044e-01a7-99293021b44a 011d2ffb-116f-044e-01a7-99293021b44a}" --- PASS: TestRaft_JoinNode_ConfigStore (0.42s) === RUN TestRaft_RemoveFollower testLoggerAdapter verbose: server-6575fec4-7ef2-093e-b4ed-8ee28300c899: 2025-02-10T11:34:11.320Z [INFO] server-6575fec4-7ef2-093e-b4ed-8ee28300c899: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6575fec4-7ef2-093e-b4ed-8ee28300c899 Address:6575fec4-7ef2-093e-b4ed-8ee28300c899} {Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testing.go:149: server-6575fec4-7ef2-093e-b4ed-8ee28300c899: 2025-02-10T11:34:11.320Z [INFO] server-6575fec4-7ef2-093e-b4ed-8ee28300c899: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6575fec4-7ef2-093e-b4ed-8ee28300c899 Address:6575fec4-7ef2-093e-b4ed-8ee28300c899} {Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testLoggerAdapter verbose: server-6575fec4-7ef2-093e-b4ed-8ee28300c899: 2025-02-10T11:34:11.320Z [INFO] server-6575fec4-7ef2-093e-b4ed-8ee28300c899: entering follower state: follower="Node at 6575fec4-7ef2-093e-b4ed-8ee28300c899 [Follower]" leader= testing.go:149: server-6575fec4-7ef2-093e-b4ed-8ee28300c899: 2025-02-10T11:34:11.320Z [INFO] server-6575fec4-7ef2-093e-b4ed-8ee28300c899: entering follower state: follower="Node at 6575fec4-7ef2-093e-b4ed-8ee28300c899 [Follower]" leader= testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.320Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6575fec4-7ef2-093e-b4ed-8ee28300c899 Address:6575fec4-7ef2-093e-b4ed-8ee28300c899} {Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.320Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6575fec4-7ef2-093e-b4ed-8ee28300c899 Address:6575fec4-7ef2-093e-b4ed-8ee28300c899} {Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.320Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: entering follower state: follower="Node at 69b264f1-d42c-ea84-71c4-5fdbd600eed1 [Follower]" leader= testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.320Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: entering follower state: follower="Node at 69b264f1-d42c-ea84-71c4-5fdbd600eed1 [Follower]" leader= testLoggerAdapter verbose: server-ddef9a16-de0b-679b-739d-e326a2103501: 2025-02-10T11:34:11.320Z [INFO] server-ddef9a16-de0b-679b-739d-e326a2103501: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6575fec4-7ef2-093e-b4ed-8ee28300c899 Address:6575fec4-7ef2-093e-b4ed-8ee28300c899} {Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testing.go:149: server-ddef9a16-de0b-679b-739d-e326a2103501: 2025-02-10T11:34:11.320Z [INFO] server-ddef9a16-de0b-679b-739d-e326a2103501: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6575fec4-7ef2-093e-b4ed-8ee28300c899 Address:6575fec4-7ef2-093e-b4ed-8ee28300c899} {Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.320Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:11.320Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-ddef9a16-de0b-679b-739d-e326a2103501: 2025-02-10T11:34:11.320Z [INFO] server-ddef9a16-de0b-679b-739d-e326a2103501: entering follower state: follower="Node at ddef9a16-de0b-679b-739d-e326a2103501 [Follower]" leader= testing.go:149: server-ddef9a16-de0b-679b-739d-e326a2103501: 2025-02-10T11:34:11.320Z [INFO] server-ddef9a16-de0b-679b-739d-e326a2103501: entering follower state: follower="Node at ddef9a16-de0b-679b-739d-e326a2103501 [Follower]" leader= testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [WARN] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [WARN] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: entering candidate state: node="Node at 69b264f1-d42c-ea84-71c4-5fdbd600eed1 [Candidate]" term=2 testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: entering candidate state: node="Node at 69b264f1-d42c-ea84-71c4-5fdbd600eed1 [Candidate]" term=2 testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: election won: tally=2 testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: election won: tally=2 testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: entering leader state: leader="Node at 69b264f1-d42c-ea84-71c4-5fdbd600eed1 [Leader]" testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: entering leader state: leader="Node at 69b264f1-d42c-ea84-71c4-5fdbd600eed1 [Leader]" testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: added peer, starting replication: peer=server-6575fec4-7ef2-093e-b4ed-8ee28300c899 testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: added peer, starting replication: peer=server-6575fec4-7ef2-093e-b4ed-8ee28300c899 testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: added peer, starting replication: peer=server-ddef9a16-de0b-679b-739d-e326a2103501 testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: added peer, starting replication: peer=server-ddef9a16-de0b-679b-739d-e326a2103501 testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: pipelining replication: peer="{Voter server-6575fec4-7ef2-093e-b4ed-8ee28300c899 6575fec4-7ef2-093e-b4ed-8ee28300c899}" testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: pipelining replication: peer="{Voter server-6575fec4-7ef2-093e-b4ed-8ee28300c899 6575fec4-7ef2-093e-b4ed-8ee28300c899}" testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: pipelining replication: peer="{Voter server-ddef9a16-de0b-679b-739d-e326a2103501 ddef9a16-de0b-679b-739d-e326a2103501}" testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.377Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: pipelining replication: peer="{Voter server-ddef9a16-de0b-679b-739d-e326a2103501 ddef9a16-de0b-679b-739d-e326a2103501}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.482Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.377400848 +0000 UTC m=+7.746305151 (1 nodes), 56.76796ms from start of poll, 57.165786ms from cluster start. Timeout at 2025-02-10 11:34:11.482474141 +0000 UTC m=+7.851378437, 105.073286ms after stability testing.go:149: cluster: 2025-02-10T11:34:11.482Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.377400848 +0000 UTC m=+7.746305151 (1 nodes), 56.76796ms from start of poll, 57.165786ms from cluster start. Timeout at 2025-02-10 11:34:11.482474141 +0000 UTC m=+7.851378437, 105.073286ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.487Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:11.487Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.592Z [INFO] stable state for Follower reached at 2025-02-10 11:34:11.487656043 +0000 UTC m=+7.856560342 (2 nodes), 837ns from start of poll, 167.420977ms from cluster start. Timeout at 2025-02-10 11:34:11.592728302 +0000 UTC m=+7.961632602, 105.07226ms after stability testing.go:149: cluster: 2025-02-10T11:34:11.592Z [INFO] stable state for Follower reached at 2025-02-10 11:34:11.487656043 +0000 UTC m=+7.856560342 (2 nodes), 837ns from start of poll, 167.420977ms from cluster start. Timeout at 2025-02-10 11:34:11.592728302 +0000 UTC m=+7.961632602, 105.07226ms after stability testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.592Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: updating configuration: command=RemoveServer server-id=server-6575fec4-7ef2-093e-b4ed-8ee28300c899 server-addr= servers="[{Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.592Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: updating configuration: command=RemoveServer server-id=server-6575fec4-7ef2-093e-b4ed-8ee28300c899 server-addr= servers="[{Suffrage:Voter ID:server-69b264f1-d42c-ea84-71c4-5fdbd600eed1 Address:69b264f1-d42c-ea84-71c4-5fdbd600eed1} {Suffrage:Voter ID:server-ddef9a16-de0b-679b-739d-e326a2103501 Address:ddef9a16-de0b-679b-739d-e326a2103501}]" testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.592Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: removed peer, stopping replication: peer=server-6575fec4-7ef2-093e-b4ed-8ee28300c899 last-index=3 testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.592Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: removed peer, stopping replication: peer=server-6575fec4-7ef2-093e-b4ed-8ee28300c899 last-index=3 testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.592Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: aborting pipeline replication: peer="{Voter server-6575fec4-7ef2-093e-b4ed-8ee28300c899 6575fec4-7ef2-093e-b4ed-8ee28300c899}" testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.592Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: aborting pipeline replication: peer="{Voter server-6575fec4-7ef2-093e-b4ed-8ee28300c899 6575fec4-7ef2-093e-b4ed-8ee28300c899}" testLoggerAdapter verbose: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.698Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: aborting pipeline replication: peer="{Voter server-ddef9a16-de0b-679b-739d-e326a2103501 ddef9a16-de0b-679b-739d-e326a2103501}" testing.go:149: server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: 2025-02-10T11:34:11.698Z [INFO] server-69b264f1-d42c-ea84-71c4-5fdbd600eed1: aborting pipeline replication: peer="{Voter server-ddef9a16-de0b-679b-739d-e326a2103501 ddef9a16-de0b-679b-739d-e326a2103501}" --- PASS: TestRaft_RemoveFollower (0.38s) === RUN TestRaft_RemoveLeader testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.699Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-15effd90-5407-7b7f-7502-c7f4035cc10d Address:15effd90-5407-7b7f-7502-c7f4035cc10d} {Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.699Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-15effd90-5407-7b7f-7502-c7f4035cc10d Address:15effd90-5407-7b7f-7502-c7f4035cc10d} {Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.699Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: entering follower state: follower="Node at 15effd90-5407-7b7f-7502-c7f4035cc10d [Follower]" leader= testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.699Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: entering follower state: follower="Node at 15effd90-5407-7b7f-7502-c7f4035cc10d [Follower]" leader= testLoggerAdapter verbose: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:11.699Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-15effd90-5407-7b7f-7502-c7f4035cc10d Address:15effd90-5407-7b7f-7502-c7f4035cc10d} {Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testing.go:149: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:11.699Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-15effd90-5407-7b7f-7502-c7f4035cc10d Address:15effd90-5407-7b7f-7502-c7f4035cc10d} {Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testLoggerAdapter verbose: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:11.699Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: entering follower state: follower="Node at e8a89705-86f4-a5a9-9a7b-ad099c23e969 [Follower]" leader= testing.go:149: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:11.699Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: entering follower state: follower="Node at e8a89705-86f4-a5a9-9a7b-ad099c23e969 [Follower]" leader= testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:11.699Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-15effd90-5407-7b7f-7502-c7f4035cc10d Address:15effd90-5407-7b7f-7502-c7f4035cc10d} {Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:11.699Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-15effd90-5407-7b7f-7502-c7f4035cc10d Address:15effd90-5407-7b7f-7502-c7f4035cc10d} {Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.699Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:11.699Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:11.699Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering follower state: follower="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Follower]" leader= testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:11.699Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering follower state: follower="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Follower]" leader= testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [WARN] server-15effd90-5407-7b7f-7502-c7f4035cc10d: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [WARN] server-15effd90-5407-7b7f-7502-c7f4035cc10d: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: entering candidate state: node="Node at 15effd90-5407-7b7f-7502-c7f4035cc10d [Candidate]" term=2 testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: entering candidate state: node="Node at 15effd90-5407-7b7f-7502-c7f4035cc10d [Candidate]" term=2 testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: election won: tally=2 testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: election won: tally=2 testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: entering leader state: leader="Node at 15effd90-5407-7b7f-7502-c7f4035cc10d [Leader]" testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: entering leader state: leader="Node at 15effd90-5407-7b7f-7502-c7f4035cc10d [Leader]" testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: added peer, starting replication: peer=server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: added peer, starting replication: peer=server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: added peer, starting replication: peer=server-274c9ffb-318d-1625-25c6-f6e8d57b8223 testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: added peer, starting replication: peer=server-274c9ffb-318d-1625-25c6-f6e8d57b8223 testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: pipelining replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: pipelining replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: pipelining replication: peer="{Voter server-274c9ffb-318d-1625-25c6-f6e8d57b8223 274c9ffb-318d-1625-25c6-f6e8d57b8223}" testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.752Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: pipelining replication: peer="{Voter server-274c9ffb-318d-1625-25c6-f6e8d57b8223 274c9ffb-318d-1625-25c6-f6e8d57b8223}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.857Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.752470057 +0000 UTC m=+8.121374358 (1 nodes), 52.941826ms from start of poll, 53.340587ms from cluster start. Timeout at 2025-02-10 11:34:11.857558136 +0000 UTC m=+8.226462512, 105.088154ms after stability testing.go:149: cluster: 2025-02-10T11:34:11.857Z [INFO] stable state for Leader reached at 2025-02-10 11:34:11.752470057 +0000 UTC m=+8.121374358 (1 nodes), 52.941826ms from start of poll, 53.340587ms from cluster start. Timeout at 2025-02-10 11:34:11.857558136 +0000 UTC m=+8.226462512, 105.088154ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.862Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:11.862Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:11.967Z [INFO] stable state for Follower reached at 2025-02-10 11:34:11.862811007 +0000 UTC m=+8.231715307 (2 nodes), 597ns from start of poll, 163.681536ms from cluster start. Timeout at 2025-02-10 11:34:11.967883422 +0000 UTC m=+8.336787713, 105.072406ms after stability testing.go:149: cluster: 2025-02-10T11:34:11.967Z [INFO] stable state for Follower reached at 2025-02-10 11:34:11.862811007 +0000 UTC m=+8.231715307 (2 nodes), 597ns from start of poll, 163.681536ms from cluster start. Timeout at 2025-02-10 11:34:11.967883422 +0000 UTC m=+8.336787713, 105.072406ms after stability testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.967Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: updating configuration: command=RemoveServer server-id=server-15effd90-5407-7b7f-7502-c7f4035cc10d server-addr= servers="[{Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.967Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: updating configuration: command=RemoveServer server-id=server-15effd90-5407-7b7f-7502-c7f4035cc10d server-addr= servers="[{Suffrage:Voter ID:server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 Address:e8a89705-86f4-a5a9-9a7b-ad099c23e969} {Suffrage:Voter ID:server-274c9ffb-318d-1625-25c6-f6e8d57b8223 Address:274c9ffb-318d-1625-25c6-f6e8d57b8223}]" testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.968Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: removed ourself, shutting down testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.968Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: removed ourself, shutting down testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.968Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: aborting pipeline replication: peer="{Voter server-274c9ffb-318d-1625-25c6-f6e8d57b8223 274c9ffb-318d-1625-25c6-f6e8d57b8223}" testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.968Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: aborting pipeline replication: peer="{Voter server-274c9ffb-318d-1625-25c6-f6e8d57b8223 274c9ffb-318d-1625-25c6-f6e8d57b8223}" testLoggerAdapter verbose: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.968Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: aborting pipeline replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" testing.go:149: server-15effd90-5407-7b7f-7502-c7f4035cc10d: 2025-02-10T11:34:11.968Z [INFO] server-15effd90-5407-7b7f-7502-c7f4035cc10d: aborting pipeline replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.034Z [WARN] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: heartbeat timeout reached, starting election: last-leader=15effd90-5407-7b7f-7502-c7f4035cc10d testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.034Z [WARN] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: heartbeat timeout reached, starting election: last-leader=15effd90-5407-7b7f-7502-c7f4035cc10d testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.034Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering candidate state: node="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Candidate]" term=3 testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.034Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering candidate state: node="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Candidate]" term=3 testLoggerAdapter verbose: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.034Z [WARN] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: rejecting vote request since we have a leader: from=274c9ffb-318d-1625-25c6-f6e8d57b8223 leader=15effd90-5407-7b7f-7502-c7f4035cc10d testing.go:149: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.034Z [WARN] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: rejecting vote request since we have a leader: from=274c9ffb-318d-1625-25c6-f6e8d57b8223 leader=15effd90-5407-7b7f-7502-c7f4035cc10d testLoggerAdapter verbose: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.076Z [WARN] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: heartbeat timeout reached, starting election: last-leader=15effd90-5407-7b7f-7502-c7f4035cc10d testing.go:149: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.076Z [WARN] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: heartbeat timeout reached, starting election: last-leader=15effd90-5407-7b7f-7502-c7f4035cc10d testLoggerAdapter verbose: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.077Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: entering candidate state: node="Node at e8a89705-86f4-a5a9-9a7b-ad099c23e969 [Candidate]" term=3 testing.go:149: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.077Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: entering candidate state: node="Node at e8a89705-86f4-a5a9-9a7b-ad099c23e969 [Candidate]" term=3 testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.077Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: duplicate requestVote for same term: term=3 testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.077Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [WARN] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: Election timeout reached, restarting election testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [WARN] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: Election timeout reached, restarting election testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering candidate state: node="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Candidate]" term=4 testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering candidate state: node="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Candidate]" term=4 testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: election won: tally=2 testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: election won: tally=2 testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering leader state: leader="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Leader]" testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: entering leader state: leader="Node at 274c9ffb-318d-1625-25c6-f6e8d57b8223 [Leader]" testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: added peer, starting replication: peer=server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 testLoggerAdapter verbose: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.116Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: entering follower state: follower="Node at e8a89705-86f4-a5a9-9a7b-ad099c23e969 [Follower]" leader= testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: added peer, starting replication: peer=server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 testing.go:149: server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: 2025-02-10T11:34:12.116Z [INFO] server-e8a89705-86f4-a5a9-9a7b-ad099c23e969: entering follower state: follower="Node at e8a89705-86f4-a5a9-9a7b-ad099c23e969 [Follower]" leader= testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: pipelining replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.116Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: pipelining replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.178Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:12.178Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.283Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.178291735 +0000 UTC m=+8.547196035 (1 nodes), 21.937µs from start of poll, 479.162264ms from cluster start. Timeout at 2025-02-10 11:34:12.283361249 +0000 UTC m=+8.652265545, 105.06951ms after stability testing.go:149: cluster: 2025-02-10T11:34:12.283Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.178291735 +0000 UTC m=+8.547196035 (1 nodes), 21.937µs from start of poll, 479.162264ms from cluster start. Timeout at 2025-02-10 11:34:12.283361249 +0000 UTC m=+8.652265545, 105.06951ms after stability testLoggerAdapter verbose: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.283Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: aborting pipeline replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" testing.go:149: server-274c9ffb-318d-1625-25c6-f6e8d57b8223: 2025-02-10T11:34:12.283Z [INFO] server-274c9ffb-318d-1625-25c6-f6e8d57b8223: aborting pipeline replication: peer="{Voter server-e8a89705-86f4-a5a9-9a7b-ad099c23e969 e8a89705-86f4-a5a9-9a7b-ad099c23e969}" --- PASS: TestRaft_RemoveLeader (0.59s) === RUN TestRaft_RemoveLeader_NoShutdown testLoggerAdapter verbose: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.284Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd} {Suffrage:Voter ID:server-7387e957-3920-f465-8f70-f69cf7046b0c Address:7387e957-3920-f465-8f70-f69cf7046b0c}]" testing.go:149: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.284Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd} {Suffrage:Voter ID:server-7387e957-3920-f465-8f70-f69cf7046b0c Address:7387e957-3920-f465-8f70-f69cf7046b0c}]" testLoggerAdapter verbose: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.284Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: entering follower state: follower="Node at e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 [Follower]" leader= testing.go:149: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.284Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: entering follower state: follower="Node at e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 [Follower]" leader= testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.284Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd} {Suffrage:Voter ID:server-7387e957-3920-f465-8f70-f69cf7046b0c Address:7387e957-3920-f465-8f70-f69cf7046b0c}]" testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.284Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd} {Suffrage:Voter ID:server-7387e957-3920-f465-8f70-f69cf7046b0c Address:7387e957-3920-f465-8f70-f69cf7046b0c}]" testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.284Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering follower state: follower="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Follower]" leader= testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.284Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering follower state: follower="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Follower]" leader= testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.284Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd} {Suffrage:Voter ID:server-7387e957-3920-f465-8f70-f69cf7046b0c Address:7387e957-3920-f465-8f70-f69cf7046b0c}]" testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.284Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd} {Suffrage:Voter ID:server-7387e957-3920-f465-8f70-f69cf7046b0c Address:7387e957-3920-f465-8f70-f69cf7046b0c}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.284Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:12.284Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.284Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering follower state: follower="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Follower]" leader= testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.284Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering follower state: follower="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Follower]" leader= testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [WARN] server-7387e957-3920-f465-8f70-f69cf7046b0c: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [WARN] server-7387e957-3920-f465-8f70-f69cf7046b0c: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering candidate state: node="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Candidate]" term=2 testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering candidate state: node="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Candidate]" term=2 testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: election won: tally=2 testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: election won: tally=2 testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering leader state: leader="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Leader]" testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering leader state: leader="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Leader]" testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: added peer, starting replication: peer=server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: added peer, starting replication: peer=server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: added peer, starting replication: peer=server-65e22077-d0d6-575f-959b-7c218aa329fd testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: added peer, starting replication: peer=server-65e22077-d0d6-575f-959b-7c218aa329fd testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: pipelining replication: peer="{Voter server-65e22077-d0d6-575f-959b-7c218aa329fd 65e22077-d0d6-575f-959b-7c218aa329fd}" testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: pipelining replication: peer="{Voter server-65e22077-d0d6-575f-959b-7c218aa329fd 65e22077-d0d6-575f-959b-7c218aa329fd}" testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: pipelining replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.346Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: pipelining replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.451Z [INFO] stable state for Follower reached at 2025-02-10 11:34:12.346255138 +0000 UTC m=+8.715159439 (2 nodes), 61.331252ms from start of poll, 61.715667ms from cluster start. Timeout at 2025-02-10 11:34:12.451327724 +0000 UTC m=+8.820232018, 105.072579ms after stability testing.go:149: cluster: 2025-02-10T11:34:12.451Z [INFO] stable state for Follower reached at 2025-02-10 11:34:12.346255138 +0000 UTC m=+8.715159439 (2 nodes), 61.331252ms from start of poll, 61.715667ms from cluster start. Timeout at 2025-02-10 11:34:12.451327724 +0000 UTC m=+8.820232018, 105.072579ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.451Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:12.451Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.556Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.451408698 +0000 UTC m=+8.820312998 (1 nodes), 492ns from start of poll, 166.869226ms from cluster start. Timeout at 2025-02-10 11:34:12.556488313 +0000 UTC m=+8.925392610, 105.079612ms after stability testing.go:149: cluster: 2025-02-10T11:34:12.556Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.451408698 +0000 UTC m=+8.820312998 (1 nodes), 492ns from start of poll, 166.869226ms from cluster start. Timeout at 2025-02-10 11:34:12.556488313 +0000 UTC m=+8.925392610, 105.079612ms after stability testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: updating configuration: command=RemoveServer server-id=server-7387e957-3920-f465-8f70-f69cf7046b0c server-addr= servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd}]" testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: updating configuration: command=RemoveServer server-id=server-7387e957-3920-f465-8f70-f69cf7046b0c server-addr= servers="[{Suffrage:Voter ID:server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 Address:e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44} {Suffrage:Voter ID:server-65e22077-d0d6-575f-959b-7c218aa329fd Address:65e22077-d0d6-575f-959b-7c218aa329fd}]" testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: removed ourself, transitioning to follower testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: removed ourself, transitioning to follower testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering follower state: follower="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Follower]" leader= testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: entering follower state: follower="Node at 7387e957-3920-f465-8f70-f69cf7046b0c [Follower]" leader= testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: aborting pipeline replication: peer="{Voter server-65e22077-d0d6-575f-959b-7c218aa329fd 65e22077-d0d6-575f-959b-7c218aa329fd}" testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: aborting pipeline replication: peer="{Voter server-65e22077-d0d6-575f-959b-7c218aa329fd 65e22077-d0d6-575f-959b-7c218aa329fd}" testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: aborting pipeline replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.557Z [INFO] server-7387e957-3920-f465-8f70-f69cf7046b0c: aborting pipeline replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.623Z [WARN] server-65e22077-d0d6-575f-959b-7c218aa329fd: heartbeat timeout reached, starting election: last-leader=7387e957-3920-f465-8f70-f69cf7046b0c testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.623Z [WARN] server-65e22077-d0d6-575f-959b-7c218aa329fd: heartbeat timeout reached, starting election: last-leader=7387e957-3920-f465-8f70-f69cf7046b0c testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.623Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering candidate state: node="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Candidate]" term=3 testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.623Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering candidate state: node="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Candidate]" term=3 testLoggerAdapter verbose: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.623Z [WARN] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: rejecting vote request since we have a leader: from=65e22077-d0d6-575f-959b-7c218aa329fd leader=7387e957-3920-f465-8f70-f69cf7046b0c testing.go:149: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.623Z [WARN] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: rejecting vote request since we have a leader: from=65e22077-d0d6-575f-959b-7c218aa329fd leader=7387e957-3920-f465-8f70-f69cf7046b0c testLoggerAdapter verbose: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.651Z [WARN] server-7387e957-3920-f465-8f70-f69cf7046b0c: not part of stable configuration, aborting election testing.go:149: server-7387e957-3920-f465-8f70-f69cf7046b0c: 2025-02-10T11:34:12.651Z [WARN] server-7387e957-3920-f465-8f70-f69cf7046b0c: not part of stable configuration, aborting election testLoggerAdapter verbose: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.655Z [WARN] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: heartbeat timeout reached, starting election: last-leader=7387e957-3920-f465-8f70-f69cf7046b0c testing.go:149: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.655Z [WARN] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: heartbeat timeout reached, starting election: last-leader=7387e957-3920-f465-8f70-f69cf7046b0c testLoggerAdapter verbose: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.655Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: entering candidate state: node="Node at e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 [Candidate]" term=3 testing.go:149: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.655Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: entering candidate state: node="Node at e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 [Candidate]" term=3 testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.655Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: duplicate requestVote for same term: term=3 testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.655Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.662Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:12.662Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [WARN] server-65e22077-d0d6-575f-959b-7c218aa329fd: Election timeout reached, restarting election testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [WARN] server-65e22077-d0d6-575f-959b-7c218aa329fd: Election timeout reached, restarting election testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering candidate state: node="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Candidate]" term=4 testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering candidate state: node="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Candidate]" term=4 testLoggerAdapter verbose: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.695Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: entering follower state: follower="Node at e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 [Follower]" leader= testing.go:149: server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: 2025-02-10T11:34:12.695Z [INFO] server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44: entering follower state: follower="Node at e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 [Follower]" leader= testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: election won: tally=2 testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: election won: tally=2 testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering leader state: leader="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Leader]" testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: entering leader state: leader="Node at 65e22077-d0d6-575f-959b-7c218aa329fd [Leader]" testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: added peer, starting replication: peer=server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: added peer, starting replication: peer=server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: pipelining replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.695Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: pipelining replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.801Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.695883237 +0000 UTC m=+9.064787555 (1 nodes), 33.261908ms from start of poll, 411.343783ms from cluster start. Timeout at 2025-02-10 11:34:12.80098997 +0000 UTC m=+9.169894323, 105.106768ms after stability testing.go:149: cluster: 2025-02-10T11:34:12.801Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.695883237 +0000 UTC m=+9.064787555 (1 nodes), 33.261908ms from start of poll, 411.343783ms from cluster start. Timeout at 2025-02-10 11:34:12.80098997 +0000 UTC m=+9.169894323, 105.106768ms after stability testLoggerAdapter verbose: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.906Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: aborting pipeline replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" testing.go:149: server-65e22077-d0d6-575f-959b-7c218aa329fd: 2025-02-10T11:34:12.906Z [INFO] server-65e22077-d0d6-575f-959b-7c218aa329fd: aborting pipeline replication: peer="{Voter server-e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44 e1cc21d9-3c6a-ff00-3645-c3f4e3f73b44}" --- PASS: TestRaft_RemoveLeader_NoShutdown (0.62s) === RUN TestRaft_RemoveFollower_SplitCluster testLoggerAdapter verbose: server-24c3003c-4c9c-025e-ff01-255bebc31000: 2025-02-10T11:34:12.907Z [INFO] server-24c3003c-4c9c-025e-ff01-255bebc31000: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testing.go:149: server-24c3003c-4c9c-025e-ff01-255bebc31000: 2025-02-10T11:34:12.907Z [INFO] server-24c3003c-4c9c-025e-ff01-255bebc31000: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testLoggerAdapter verbose: server-24c3003c-4c9c-025e-ff01-255bebc31000: 2025-02-10T11:34:12.907Z [INFO] server-24c3003c-4c9c-025e-ff01-255bebc31000: entering follower state: follower="Node at 24c3003c-4c9c-025e-ff01-255bebc31000 [Follower]" leader= testing.go:149: server-24c3003c-4c9c-025e-ff01-255bebc31000: 2025-02-10T11:34:12.907Z [INFO] server-24c3003c-4c9c-025e-ff01-255bebc31000: entering follower state: follower="Node at 24c3003c-4c9c-025e-ff01-255bebc31000 [Follower]" leader= testLoggerAdapter verbose: server-60141061-f7a9-4547-c487-7876db3d60b5: 2025-02-10T11:34:12.907Z [INFO] server-60141061-f7a9-4547-c487-7876db3d60b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testing.go:149: server-60141061-f7a9-4547-c487-7876db3d60b5: 2025-02-10T11:34:12.907Z [INFO] server-60141061-f7a9-4547-c487-7876db3d60b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testLoggerAdapter verbose: server-60141061-f7a9-4547-c487-7876db3d60b5: 2025-02-10T11:34:12.907Z [INFO] server-60141061-f7a9-4547-c487-7876db3d60b5: entering follower state: follower="Node at 60141061-f7a9-4547-c487-7876db3d60b5 [Follower]" leader= testing.go:149: server-60141061-f7a9-4547-c487-7876db3d60b5: 2025-02-10T11:34:12.907Z [INFO] server-60141061-f7a9-4547-c487-7876db3d60b5: entering follower state: follower="Node at 60141061-f7a9-4547-c487-7876db3d60b5 [Follower]" leader= testLoggerAdapter verbose: server-b9fab61a-5861-d93f-09b6-920663dfde4d: 2025-02-10T11:34:12.907Z [INFO] server-b9fab61a-5861-d93f-09b6-920663dfde4d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testing.go:149: server-b9fab61a-5861-d93f-09b6-920663dfde4d: 2025-02-10T11:34:12.907Z [INFO] server-b9fab61a-5861-d93f-09b6-920663dfde4d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testLoggerAdapter verbose: server-b9fab61a-5861-d93f-09b6-920663dfde4d: 2025-02-10T11:34:12.907Z [INFO] server-b9fab61a-5861-d93f-09b6-920663dfde4d: entering follower state: follower="Node at b9fab61a-5861-d93f-09b6-920663dfde4d [Follower]" leader= testing.go:149: server-b9fab61a-5861-d93f-09b6-920663dfde4d: 2025-02-10T11:34:12.907Z [INFO] server-b9fab61a-5861-d93f-09b6-920663dfde4d: entering follower state: follower="Node at b9fab61a-5861-d93f-09b6-920663dfde4d [Follower]" leader= testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.907Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.907Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-b9fab61a-5861-d93f-09b6-920663dfde4d Address:b9fab61a-5861-d93f-09b6-920663dfde4d} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:12.907Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:12.907Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.907Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering follower state: follower="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Follower]" leader= testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.907Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering follower state: follower="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Follower]" leader= testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering candidate state: node="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Candidate]" term=2 testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering candidate state: node="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Candidate]" term=2 testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: election won: tally=3 testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: election won: tally=3 testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering leader state: leader="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Leader]" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering leader state: leader="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Leader]" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: added peer, starting replication: peer=server-24c3003c-4c9c-025e-ff01-255bebc31000 testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: added peer, starting replication: peer=server-24c3003c-4c9c-025e-ff01-255bebc31000 testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: added peer, starting replication: peer=server-60141061-f7a9-4547-c487-7876db3d60b5 testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: added peer, starting replication: peer=server-60141061-f7a9-4547-c487-7876db3d60b5 testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: added peer, starting replication: peer=server-b9fab61a-5861-d93f-09b6-920663dfde4d testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: added peer, starting replication: peer=server-b9fab61a-5861-d93f-09b6-920663dfde4d testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: pipelining replication: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: pipelining replication: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: pipelining replication: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: pipelining replication: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: pipelining replication: peer="{Voter server-b9fab61a-5861-d93f-09b6-920663dfde4d b9fab61a-5861-d93f-09b6-920663dfde4d}" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:12.968Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: pipelining replication: peer="{Voter server-b9fab61a-5861-d93f-09b6-920663dfde4d b9fab61a-5861-d93f-09b6-920663dfde4d}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.073Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.968328085 +0000 UTC m=+9.337232383 (1 nodes), 60.540029ms from start of poll, 61.052877ms from cluster start. Timeout at 2025-02-10 11:34:13.073436846 +0000 UTC m=+9.442341252, 105.108869ms after stability testing.go:149: cluster: 2025-02-10T11:34:13.073Z [INFO] stable state for Leader reached at 2025-02-10 11:34:12.968328085 +0000 UTC m=+9.337232383 (1 nodes), 60.540029ms from start of poll, 61.052877ms from cluster start. Timeout at 2025-02-10 11:34:13.073436846 +0000 UTC m=+9.442341252, 105.108869ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.178Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:13.178Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.283Z [INFO] stable state for Follower reached at 2025-02-10 11:34:13.178653742 +0000 UTC m=+9.547558043 (3 nodes), 804ns from start of poll, 271.378537ms from cluster start. Timeout at 2025-02-10 11:34:13.283754621 +0000 UTC m=+9.652658973, 105.10093ms after stability testing.go:149: cluster: 2025-02-10T11:34:13.283Z [INFO] stable state for Follower reached at 2025-02-10 11:34:13.178653742 +0000 UTC m=+9.547558043 (3 nodes), 804ns from start of poll, 271.378537ms from cluster start. Timeout at 2025-02-10 11:34:13.283754621 +0000 UTC m=+9.652658973, 105.10093ms after stability testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.283Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: updating configuration: command=RemoveServer server-id=server-b9fab61a-5861-d93f-09b6-920663dfde4d server-addr= servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.283Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: updating configuration: command=RemoveServer server-id=server-b9fab61a-5861-d93f-09b6-920663dfde4d server-addr= servers="[{Suffrage:Voter ID:server-24c3003c-4c9c-025e-ff01-255bebc31000 Address:24c3003c-4c9c-025e-ff01-255bebc31000} {Suffrage:Voter ID:server-60141061-f7a9-4547-c487-7876db3d60b5 Address:60141061-f7a9-4547-c487-7876db3d60b5} {Suffrage:Voter ID:server-d687b9b3-9467-cd46-cea1-c8f0e48c612b Address:d687b9b3-9467-cd46-cea1-c8f0e48c612b}]" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: removed peer, stopping replication: peer=server-b9fab61a-5861-d93f-09b6-920663dfde4d last-index=3 testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: removed peer, stopping replication: peer=server-b9fab61a-5861-d93f-09b6-920663dfde4d last-index=3 testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to pipeline appendEntries: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="append pipeline closed" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to pipeline appendEntries: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="append pipeline closed" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: aborting pipeline replication: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: aborting pipeline replication: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: aborting pipeline replication: peer="{Voter server-b9fab61a-5861-d93f-09b6-920663dfde4d b9fab61a-5861-d93f-09b6-920663dfde4d}" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: aborting pipeline replication: peer="{Voter server-b9fab61a-5861-d93f-09b6-920663dfde4d b9fab61a-5861-d93f-09b6-920663dfde4d}" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to pipeline appendEntries: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="append pipeline closed" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to pipeline appendEntries: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="append pipeline closed" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: aborting pipeline replication: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.284Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: aborting pipeline replication: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.290Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.290Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.291Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=24c3003c-4c9c-025e-ff01-255bebc31000 error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.291Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=24c3003c-4c9c-025e-ff01-255bebc31000 error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.291Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=60141061-f7a9-4547-c487-7876db3d60b5 error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.291Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=60141061-f7a9-4547-c487-7876db3d60b5 error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.291Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.291Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.306Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=60141061-f7a9-4547-c487-7876db3d60b5 error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.306Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=60141061-f7a9-4547-c487-7876db3d60b5 error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.307Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.307Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.307Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=24c3003c-4c9c-025e-ff01-255bebc31000 error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.307Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=24c3003c-4c9c-025e-ff01-255bebc31000 error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.309Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.309Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.323Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=60141061-f7a9-4547-c487-7876db3d60b5 error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.323Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=60141061-f7a9-4547-c487-7876db3d60b5 error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.325Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=24c3003c-4c9c-025e-ff01-255bebc31000 error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.325Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to heartbeat to: peer=24c3003c-4c9c-025e-ff01-255bebc31000 error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.326Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.326Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.327Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.327Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.331Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact: server-id=server-24c3003c-4c9c-025e-ff01-255bebc31000 time=50.064048ms testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.331Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact: server-id=server-24c3003c-4c9c-025e-ff01-255bebc31000 time=50.064048ms testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact: server-id=server-24c3003c-4c9c-025e-ff01-255bebc31000 time=60.213892ms testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact: server-id=server-24c3003c-4c9c-025e-ff01-255bebc31000 time=60.213892ms testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact: server-id=server-60141061-f7a9-4547-c487-7876db3d60b5 time=60.107557ms testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact: server-id=server-60141061-f7a9-4547-c487-7876db3d60b5 time=60.107557ms testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact quorum of nodes, stepping down testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [WARN] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering follower state: follower="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Follower]" leader= testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [INFO] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: entering follower state: follower="Node at d687b9b3-9467-cd46-cea1-c8f0e48c612b [Follower]" leader= testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-24c3003c-4c9c-025e-ff01-255bebc31000 24c3003c-4c9c-025e-ff01-255bebc31000}" error="failed to connect to peer: 24c3003c-4c9c-025e-ff01-255bebc31000" testLoggerAdapter verbose: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" testing.go:149: server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: 2025-02-10T11:34:13.342Z [ERROR] server-d687b9b3-9467-cd46-cea1-c8f0e48c612b: failed to appendEntries to: peer="{Voter server-60141061-f7a9-4547-c487-7876db3d60b5 60141061-f7a9-4547-c487-7876db3d60b5}" error="failed to connect to peer: 60141061-f7a9-4547-c487-7876db3d60b5" --- PASS: TestRaft_RemoveFollower_SplitCluster (0.44s) === RUN TestRaft_AddKnownPeer testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.343Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.343Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.343Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: entering follower state: follower="Node at acdc1783-00f7-1b47-6362-2ca2fa448643 [Follower]" leader= testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.343Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: entering follower state: follower="Node at acdc1783-00f7-1b47-6362-2ca2fa448643 [Follower]" leader= testLoggerAdapter verbose: server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: 2025-02-10T11:34:13.343Z [INFO] server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testing.go:149: server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: 2025-02-10T11:34:13.343Z [INFO] server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testLoggerAdapter verbose: server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: 2025-02-10T11:34:13.343Z [INFO] server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: entering follower state: follower="Node at 5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 [Follower]" leader= testing.go:149: server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: 2025-02-10T11:34:13.343Z [INFO] server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3: entering follower state: follower="Node at 5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 [Follower]" leader= testLoggerAdapter verbose: server-92905cb5-0269-dc14-2d78-8df63d45a327: 2025-02-10T11:34:13.343Z [INFO] server-92905cb5-0269-dc14-2d78-8df63d45a327: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testing.go:149: server-92905cb5-0269-dc14-2d78-8df63d45a327: 2025-02-10T11:34:13.343Z [INFO] server-92905cb5-0269-dc14-2d78-8df63d45a327: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.343Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:13.343Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-92905cb5-0269-dc14-2d78-8df63d45a327: 2025-02-10T11:34:13.343Z [INFO] server-92905cb5-0269-dc14-2d78-8df63d45a327: entering follower state: follower="Node at 92905cb5-0269-dc14-2d78-8df63d45a327 [Follower]" leader= testing.go:149: server-92905cb5-0269-dc14-2d78-8df63d45a327: 2025-02-10T11:34:13.343Z [INFO] server-92905cb5-0269-dc14-2d78-8df63d45a327: entering follower state: follower="Node at 92905cb5-0269-dc14-2d78-8df63d45a327 [Follower]" leader= testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [WARN] server-acdc1783-00f7-1b47-6362-2ca2fa448643: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [WARN] server-acdc1783-00f7-1b47-6362-2ca2fa448643: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: entering candidate state: node="Node at acdc1783-00f7-1b47-6362-2ca2fa448643 [Candidate]" term=2 testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: entering candidate state: node="Node at acdc1783-00f7-1b47-6362-2ca2fa448643 [Candidate]" term=2 testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: election won: tally=2 testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: election won: tally=2 testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: entering leader state: leader="Node at acdc1783-00f7-1b47-6362-2ca2fa448643 [Leader]" testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: entering leader state: leader="Node at acdc1783-00f7-1b47-6362-2ca2fa448643 [Leader]" testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: added peer, starting replication: peer=server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: added peer, starting replication: peer=server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: added peer, starting replication: peer=server-92905cb5-0269-dc14-2d78-8df63d45a327 testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: added peer, starting replication: peer=server-92905cb5-0269-dc14-2d78-8df63d45a327 testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: pipelining replication: peer="{Voter server-92905cb5-0269-dc14-2d78-8df63d45a327 92905cb5-0269-dc14-2d78-8df63d45a327}" testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: pipelining replication: peer="{Voter server-92905cb5-0269-dc14-2d78-8df63d45a327 92905cb5-0269-dc14-2d78-8df63d45a327}" testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: pipelining replication: peer="{Voter server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 5b6dfd96-4124-b06c-cfaa-25e1a189b1e3}" testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.403Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: pipelining replication: peer="{Voter server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 5b6dfd96-4124-b06c-cfaa-25e1a189b1e3}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.508Z [INFO] stable state for Leader reached at 2025-02-10 11:34:13.403541963 +0000 UTC m=+9.772446264 (1 nodes), 59.655121ms from start of poll, 59.997976ms from cluster start. Timeout at 2025-02-10 11:34:13.508635198 +0000 UTC m=+9.877539555, 105.093291ms after stability testing.go:149: cluster: 2025-02-10T11:34:13.508Z [INFO] stable state for Leader reached at 2025-02-10 11:34:13.403541963 +0000 UTC m=+9.772446264 (1 nodes), 59.655121ms from start of poll, 59.997976ms from cluster start. Timeout at 2025-02-10 11:34:13.508635198 +0000 UTC m=+9.877539555, 105.093291ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.508Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:13.508Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.613Z [INFO] stable state for Follower reached at 2025-02-10 11:34:13.508829808 +0000 UTC m=+9.877734171 (2 nodes), 1.484µs from start of poll, 165.285883ms from cluster start. Timeout at 2025-02-10 11:34:13.613903936 +0000 UTC m=+9.982808233, 105.074062ms after stability testing.go:149: cluster: 2025-02-10T11:34:13.613Z [INFO] stable state for Follower reached at 2025-02-10 11:34:13.508829808 +0000 UTC m=+9.877734171 (2 nodes), 1.484µs from start of poll, 165.285883ms from cluster start. Timeout at 2025-02-10 11:34:13.613903936 +0000 UTC m=+9.982808233, 105.074062ms after stability testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.613Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: updating configuration: command=AddStaging server-id=server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 server-addr=5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.613Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: updating configuration: command=AddStaging server-id=server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 server-addr=5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 servers="[{Suffrage:Voter ID:server-acdc1783-00f7-1b47-6362-2ca2fa448643 Address:acdc1783-00f7-1b47-6362-2ca2fa448643} {Suffrage:Voter ID:server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 Address:5b6dfd96-4124-b06c-cfaa-25e1a189b1e3} {Suffrage:Voter ID:server-92905cb5-0269-dc14-2d78-8df63d45a327 Address:92905cb5-0269-dc14-2d78-8df63d45a327}]" testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.614Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: aborting pipeline replication: peer="{Voter server-92905cb5-0269-dc14-2d78-8df63d45a327 92905cb5-0269-dc14-2d78-8df63d45a327}" testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.614Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: aborting pipeline replication: peer="{Voter server-92905cb5-0269-dc14-2d78-8df63d45a327 92905cb5-0269-dc14-2d78-8df63d45a327}" testLoggerAdapter verbose: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.614Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: aborting pipeline replication: peer="{Voter server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 5b6dfd96-4124-b06c-cfaa-25e1a189b1e3}" testing.go:149: server-acdc1783-00f7-1b47-6362-2ca2fa448643: 2025-02-10T11:34:13.614Z [INFO] server-acdc1783-00f7-1b47-6362-2ca2fa448643: aborting pipeline replication: peer="{Voter server-5b6dfd96-4124-b06c-cfaa-25e1a189b1e3 5b6dfd96-4124-b06c-cfaa-25e1a189b1e3}" --- PASS: TestRaft_AddKnownPeer (0.27s) === RUN TestRaft_RemoveUnknownPeer testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.615Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.615Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.615Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: entering follower state: follower="Node at 48ffc5e2-1ffc-220c-f30d-2687d1530cee [Follower]" leader= testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.615Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: entering follower state: follower="Node at 48ffc5e2-1ffc-220c-f30d-2687d1530cee [Follower]" leader= testLoggerAdapter verbose: server-baee4eee-a15f-a1a6-ad90-e59dcde42748: 2025-02-10T11:34:13.615Z [INFO] server-baee4eee-a15f-a1a6-ad90-e59dcde42748: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testing.go:149: server-baee4eee-a15f-a1a6-ad90-e59dcde42748: 2025-02-10T11:34:13.615Z [INFO] server-baee4eee-a15f-a1a6-ad90-e59dcde42748: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testLoggerAdapter verbose: server-baee4eee-a15f-a1a6-ad90-e59dcde42748: 2025-02-10T11:34:13.615Z [INFO] server-baee4eee-a15f-a1a6-ad90-e59dcde42748: entering follower state: follower="Node at baee4eee-a15f-a1a6-ad90-e59dcde42748 [Follower]" leader= testing.go:149: server-baee4eee-a15f-a1a6-ad90-e59dcde42748: 2025-02-10T11:34:13.615Z [INFO] server-baee4eee-a15f-a1a6-ad90-e59dcde42748: entering follower state: follower="Node at baee4eee-a15f-a1a6-ad90-e59dcde42748 [Follower]" leader= testLoggerAdapter verbose: server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: 2025-02-10T11:34:13.615Z [INFO] server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testing.go:149: server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: 2025-02-10T11:34:13.615Z [INFO] server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.615Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:13.615Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: 2025-02-10T11:34:13.615Z [INFO] server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: entering follower state: follower="Node at 210968fe-7ad3-bad6-5d58-7b08e51e49eb [Follower]" leader= testing.go:149: server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: 2025-02-10T11:34:13.615Z [INFO] server-210968fe-7ad3-bad6-5d58-7b08e51e49eb: entering follower state: follower="Node at 210968fe-7ad3-bad6-5d58-7b08e51e49eb [Follower]" leader= testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [WARN] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [WARN] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: entering candidate state: node="Node at 48ffc5e2-1ffc-220c-f30d-2687d1530cee [Candidate]" term=2 testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: entering candidate state: node="Node at 48ffc5e2-1ffc-220c-f30d-2687d1530cee [Candidate]" term=2 testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: election won: tally=2 testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: election won: tally=2 testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: entering leader state: leader="Node at 48ffc5e2-1ffc-220c-f30d-2687d1530cee [Leader]" testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: entering leader state: leader="Node at 48ffc5e2-1ffc-220c-f30d-2687d1530cee [Leader]" testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: added peer, starting replication: peer=server-baee4eee-a15f-a1a6-ad90-e59dcde42748 testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: added peer, starting replication: peer=server-baee4eee-a15f-a1a6-ad90-e59dcde42748 testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: added peer, starting replication: peer=server-210968fe-7ad3-bad6-5d58-7b08e51e49eb testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: added peer, starting replication: peer=server-210968fe-7ad3-bad6-5d58-7b08e51e49eb testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: pipelining replication: peer="{Voter server-baee4eee-a15f-a1a6-ad90-e59dcde42748 baee4eee-a15f-a1a6-ad90-e59dcde42748}" testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: pipelining replication: peer="{Voter server-baee4eee-a15f-a1a6-ad90-e59dcde42748 baee4eee-a15f-a1a6-ad90-e59dcde42748}" testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: pipelining replication: peer="{Voter server-210968fe-7ad3-bad6-5d58-7b08e51e49eb 210968fe-7ad3-bad6-5d58-7b08e51e49eb}" testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.680Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: pipelining replication: peer="{Voter server-210968fe-7ad3-bad6-5d58-7b08e51e49eb 210968fe-7ad3-bad6-5d58-7b08e51e49eb}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.785Z [INFO] stable state for Leader reached at 2025-02-10 11:34:13.680623236 +0000 UTC m=+10.049527602 (1 nodes), 65.154056ms from start of poll, 65.529091ms from cluster start. Timeout at 2025-02-10 11:34:13.785697837 +0000 UTC m=+10.154602134, 105.074532ms after stability testing.go:149: cluster: 2025-02-10T11:34:13.785Z [INFO] stable state for Leader reached at 2025-02-10 11:34:13.680623236 +0000 UTC m=+10.049527602 (1 nodes), 65.154056ms from start of poll, 65.529091ms from cluster start. Timeout at 2025-02-10 11:34:13.785697837 +0000 UTC m=+10.154602134, 105.074532ms after stability testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.785Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: updating configuration: command=RemoveServer server-id=abd03c33-e653-c776-70e3-9a7c59d3c0f1 server-addr= servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.785Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: updating configuration: command=RemoveServer server-id=abd03c33-e653-c776-70e3-9a7c59d3c0f1 server-addr= servers="[{Suffrage:Voter ID:server-48ffc5e2-1ffc-220c-f30d-2687d1530cee Address:48ffc5e2-1ffc-220c-f30d-2687d1530cee} {Suffrage:Voter ID:server-baee4eee-a15f-a1a6-ad90-e59dcde42748 Address:baee4eee-a15f-a1a6-ad90-e59dcde42748} {Suffrage:Voter ID:server-210968fe-7ad3-bad6-5d58-7b08e51e49eb Address:210968fe-7ad3-bad6-5d58-7b08e51e49eb}]" testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.785Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: aborting pipeline replication: peer="{Voter server-210968fe-7ad3-bad6-5d58-7b08e51e49eb 210968fe-7ad3-bad6-5d58-7b08e51e49eb}" testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.785Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: aborting pipeline replication: peer="{Voter server-210968fe-7ad3-bad6-5d58-7b08e51e49eb 210968fe-7ad3-bad6-5d58-7b08e51e49eb}" testLoggerAdapter verbose: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.785Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: aborting pipeline replication: peer="{Voter server-baee4eee-a15f-a1a6-ad90-e59dcde42748 baee4eee-a15f-a1a6-ad90-e59dcde42748}" testing.go:149: server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: 2025-02-10T11:34:13.785Z [INFO] server-48ffc5e2-1ffc-220c-f30d-2687d1530cee: aborting pipeline replication: peer="{Voter server-baee4eee-a15f-a1a6-ad90-e59dcde42748 baee4eee-a15f-a1a6-ad90-e59dcde42748}" --- PASS: TestRaft_RemoveUnknownPeer (0.17s) === RUN TestRaft_SnapshotRestore testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.786Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-852cefc1-8fe8-1a27-ad1e-037457a37f9b Address:852cefc1-8fe8-1a27-ad1e-037457a37f9b}]" testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.786Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-852cefc1-8fe8-1a27-ad1e-037457a37f9b Address:852cefc1-8fe8-1a27-ad1e-037457a37f9b}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.786Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:13.786Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.786Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: entering follower state: follower="Node at 852cefc1-8fe8-1a27-ad1e-037457a37f9b [Follower]" leader= testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.786Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: entering follower state: follower="Node at 852cefc1-8fe8-1a27-ad1e-037457a37f9b [Follower]" leader= testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [WARN] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [WARN] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: entering candidate state: node="Node at 852cefc1-8fe8-1a27-ad1e-037457a37f9b [Candidate]" term=2 testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: entering candidate state: node="Node at 852cefc1-8fe8-1a27-ad1e-037457a37f9b [Candidate]" term=2 testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: election won: tally=1 testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: election won: tally=1 testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: entering leader state: leader="Node at 852cefc1-8fe8-1a27-ad1e-037457a37f9b [Leader]" testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.853Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: entering leader state: leader="Node at 852cefc1-8fe8-1a27-ad1e-037457a37f9b [Leader]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.958Z [INFO] stable state for Leader reached at 2025-02-10 11:34:13.853767807 +0000 UTC m=+10.222672108 (1 nodes), 66.965135ms from start of poll, 67.11233ms from cluster start. Timeout at 2025-02-10 11:34:13.958866691 +0000 UTC m=+10.327771077, 105.098969ms after stability testing.go:149: cluster: 2025-02-10T11:34:13.958Z [INFO] stable state for Leader reached at 2025-02-10 11:34:13.853767807 +0000 UTC m=+10.222672108 (1 nodes), 66.965135ms from start of poll, 67.11233ms from cluster start. Timeout at 2025-02-10 11:34:13.958866691 +0000 UTC m=+10.327771077, 105.098969ms after stability testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.959Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: starting snapshot up to: index=102 testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.959Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: starting snapshot up to: index=102 testing.go:153: 2025-02-10T11:34:13.959Z [INFO] creating new snapshot: path=/tmp/raft381616389/snapshots/2-102-1739187253959.tmp testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.964Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: compacting logs: from=1 to=92 testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.964Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.964Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: snapshot complete up to: index=102 testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.964Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: snapshot complete up to: index=102 testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.965Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: restored from snapshot: id=2-102-1739187253959 testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.965Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: restored from snapshot: id=2-102-1739187253959 testLoggerAdapter verbose: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.965Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-852cefc1-8fe8-1a27-ad1e-037457a37f9b Address:852cefc1-8fe8-1a27-ad1e-037457a37f9b}]" testing.go:149: server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: 2025-02-10T11:34:13.965Z [INFO] server-852cefc1-8fe8-1a27-ad1e-037457a37f9b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-852cefc1-8fe8-1a27-ad1e-037457a37f9b Address:852cefc1-8fe8-1a27-ad1e-037457a37f9b}]" --- PASS: TestRaft_SnapshotRestore (0.18s) === RUN TestRaft_NoRestoreOnStart testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:13.965Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-93cd1f0a-625e-f564-dce2-07799daf38b2 Address:93cd1f0a-625e-f564-dce2-07799daf38b2}]" testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:13.965Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-93cd1f0a-625e-f564-dce2-07799daf38b2 Address:93cd1f0a-625e-f564-dce2-07799daf38b2}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:13.965Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:13.965Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:13.965Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering follower state: follower="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Follower]" leader= testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:13.965Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering follower state: follower="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Follower]" leader= testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [WARN] server-93cd1f0a-625e-f564-dce2-07799daf38b2: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [WARN] server-93cd1f0a-625e-f564-dce2-07799daf38b2: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering candidate state: node="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Candidate]" term=2 testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering candidate state: node="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Candidate]" term=2 testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: election won: tally=1 testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: election won: tally=1 testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering leader state: leader="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Leader]" testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.034Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering leader state: leader="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Leader]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.139Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.034202323 +0000 UTC m=+10.403106624 (1 nodes), 68.403527ms from start of poll, 68.545916ms from cluster start. Timeout at 2025-02-10 11:34:14.139279954 +0000 UTC m=+10.508184247, 105.077623ms after stability testing.go:149: cluster: 2025-02-10T11:34:14.139Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.034202323 +0000 UTC m=+10.403106624 (1 nodes), 68.403527ms from start of poll, 68.545916ms from cluster start. Timeout at 2025-02-10 11:34:14.139279954 +0000 UTC m=+10.508184247, 105.077623ms after stability testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.139Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: starting snapshot up to: index=102 testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.139Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: starting snapshot up to: index=102 testing.go:153: 2025-02-10T11:34:14.140Z [INFO] creating new snapshot: path=/tmp/raft315923839/snapshots/2-102-1739187254140.tmp testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.144Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: compacting logs: from=1 to=92 testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.144Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.144Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: snapshot complete up to: index=102 testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.144Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: snapshot complete up to: index=102 testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.144Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-93cd1f0a-625e-f564-dce2-07799daf38b2 Address:93cd1f0a-625e-f564-dce2-07799daf38b2}]" testing.go:149: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.144Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-93cd1f0a-625e-f564-dce2-07799daf38b2 Address:93cd1f0a-625e-f564-dce2-07799daf38b2}]" --- PASS: TestRaft_NoRestoreOnStart (0.18s) === RUN TestRaft_SnapshotRestore_PeerChange raft_test.go:1026: Disabled unreliable test --- SKIP: TestRaft_SnapshotRestore_PeerChange (0.00s) === RUN TestRaft_AutoSnapshot testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.144Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering follower state: follower="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Follower]" leader= raft_test.go:1132: Disabled unreliable test --- SKIP: TestRaft_AutoSnapshot (0.00s) === RUN TestRaft_UserSnapshot testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.144Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e603ad55-fc4f-b668-115f-550176ee85e2 Address:e603ad55-fc4f-b668-115f-550176ee85e2}]" testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.144Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e603ad55-fc4f-b668-115f-550176ee85e2 Address:e603ad55-fc4f-b668-115f-550176ee85e2}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.144Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:14.144Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.144Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: entering follower state: follower="Node at e603ad55-fc4f-b668-115f-550176ee85e2 [Follower]" leader= testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.144Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: entering follower state: follower="Node at e603ad55-fc4f-b668-115f-550176ee85e2 [Follower]" leader= testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [WARN] server-e603ad55-fc4f-b668-115f-550176ee85e2: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [WARN] server-e603ad55-fc4f-b668-115f-550176ee85e2: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: entering candidate state: node="Node at e603ad55-fc4f-b668-115f-550176ee85e2 [Candidate]" term=2 testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: entering candidate state: node="Node at e603ad55-fc4f-b668-115f-550176ee85e2 [Candidate]" term=2 testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: election won: tally=1 testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: election won: tally=1 testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: entering leader state: leader="Node at e603ad55-fc4f-b668-115f-550176ee85e2 [Leader]" testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.203Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: entering leader state: leader="Node at e603ad55-fc4f-b668-115f-550176ee85e2 [Leader]" testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.208Z [WARN] server-93cd1f0a-625e-f564-dce2-07799daf38b2: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.208Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering candidate state: node="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Candidate]" term=3 testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.208Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: election won: tally=1 testLoggerAdapter verbose: server-93cd1f0a-625e-f564-dce2-07799daf38b2: 2025-02-10T11:34:14.208Z [INFO] server-93cd1f0a-625e-f564-dce2-07799daf38b2: entering leader state: leader="Node at 93cd1f0a-625e-f564-dce2-07799daf38b2 [Leader]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.308Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.203387579 +0000 UTC m=+10.572291945 (1 nodes), 58.529487ms from start of poll, 58.685738ms from cluster start. Timeout at 2025-02-10 11:34:14.308406517 +0000 UTC m=+10.677310986, 105.019041ms after stability testing.go:149: cluster: 2025-02-10T11:34:14.308Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.203387579 +0000 UTC m=+10.572291945 (1 nodes), 58.529487ms from start of poll, 58.685738ms from cluster start. Timeout at 2025-02-10 11:34:14.308406517 +0000 UTC m=+10.677310986, 105.019041ms after stability testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.308Z [ERROR] server-e603ad55-fc4f-b668-115f-550176ee85e2: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.308Z [ERROR] server-e603ad55-fc4f-b668-115f-550176ee85e2: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.308Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: starting snapshot up to: index=12 testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.308Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: starting snapshot up to: index=12 testing.go:153: 2025-02-10T11:34:14.308Z [INFO] creating new snapshot: path=/tmp/raft019808265/snapshots/2-12-1739187254308.tmp testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.312Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: compacting logs: from=1 to=2 testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.312Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: compacting logs: from=1 to=2 testLoggerAdapter verbose: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.312Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: snapshot complete up to: index=12 testing.go:149: server-e603ad55-fc4f-b668-115f-550176ee85e2: 2025-02-10T11:34:14.312Z [INFO] server-e603ad55-fc4f-b668-115f-550176ee85e2: snapshot complete up to: index=12 --- PASS: TestRaft_UserSnapshot (0.17s) === RUN TestRaft_UserRestore testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.312Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e39bd5e-9334-fe43-6354-f78b50d41f05 Address:8e39bd5e-9334-fe43-6354-f78b50d41f05} {Suffrage:Voter ID:server-d8248c37-3322-d35d-4ed6-2d9614664faf Address:d8248c37-3322-d35d-4ed6-2d9614664faf} {Suffrage:Voter ID:server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb Address:b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}]" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.312Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e39bd5e-9334-fe43-6354-f78b50d41f05 Address:8e39bd5e-9334-fe43-6354-f78b50d41f05} {Suffrage:Voter ID:server-d8248c37-3322-d35d-4ed6-2d9614664faf Address:d8248c37-3322-d35d-4ed6-2d9614664faf} {Suffrage:Voter ID:server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb Address:b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}]" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.312Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: entering follower state: follower="Node at 8e39bd5e-9334-fe43-6354-f78b50d41f05 [Follower]" leader= testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.312Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: entering follower state: follower="Node at 8e39bd5e-9334-fe43-6354-f78b50d41f05 [Follower]" leader= testLoggerAdapter verbose: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.313Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e39bd5e-9334-fe43-6354-f78b50d41f05 Address:8e39bd5e-9334-fe43-6354-f78b50d41f05} {Suffrage:Voter ID:server-d8248c37-3322-d35d-4ed6-2d9614664faf Address:d8248c37-3322-d35d-4ed6-2d9614664faf} {Suffrage:Voter ID:server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb Address:b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}]" testing.go:149: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.313Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e39bd5e-9334-fe43-6354-f78b50d41f05 Address:8e39bd5e-9334-fe43-6354-f78b50d41f05} {Suffrage:Voter ID:server-d8248c37-3322-d35d-4ed6-2d9614664faf Address:d8248c37-3322-d35d-4ed6-2d9614664faf} {Suffrage:Voter ID:server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb Address:b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}]" testLoggerAdapter verbose: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.313Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: entering follower state: follower="Node at d8248c37-3322-d35d-4ed6-2d9614664faf [Follower]" leader= testing.go:149: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.313Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: entering follower state: follower="Node at d8248c37-3322-d35d-4ed6-2d9614664faf [Follower]" leader= testLoggerAdapter verbose: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.313Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e39bd5e-9334-fe43-6354-f78b50d41f05 Address:8e39bd5e-9334-fe43-6354-f78b50d41f05} {Suffrage:Voter ID:server-d8248c37-3322-d35d-4ed6-2d9614664faf Address:d8248c37-3322-d35d-4ed6-2d9614664faf} {Suffrage:Voter ID:server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb Address:b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}]" testing.go:149: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.313Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e39bd5e-9334-fe43-6354-f78b50d41f05 Address:8e39bd5e-9334-fe43-6354-f78b50d41f05} {Suffrage:Voter ID:server-d8248c37-3322-d35d-4ed6-2d9614664faf Address:d8248c37-3322-d35d-4ed6-2d9614664faf} {Suffrage:Voter ID:server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb Address:b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.313Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:14.313Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.313Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: entering follower state: follower="Node at b76869c8-0eeb-cb7f-c033-48ec73fbfeeb [Follower]" leader= testing.go:149: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.313Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: entering follower state: follower="Node at b76869c8-0eeb-cb7f-c033-48ec73fbfeeb [Follower]" leader= testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [WARN] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [WARN] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: entering candidate state: node="Node at 8e39bd5e-9334-fe43-6354-f78b50d41f05 [Candidate]" term=2 testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: entering candidate state: node="Node at 8e39bd5e-9334-fe43-6354-f78b50d41f05 [Candidate]" term=2 testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: election won: tally=2 testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: election won: tally=2 testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: entering leader state: leader="Node at 8e39bd5e-9334-fe43-6354-f78b50d41f05 [Leader]" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: entering leader state: leader="Node at 8e39bd5e-9334-fe43-6354-f78b50d41f05 [Leader]" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: added peer, starting replication: peer=server-d8248c37-3322-d35d-4ed6-2d9614664faf testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: added peer, starting replication: peer=server-d8248c37-3322-d35d-4ed6-2d9614664faf testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: added peer, starting replication: peer=server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: added peer, starting replication: peer=server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.375Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.480Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.375469175 +0000 UTC m=+10.744373476 (1 nodes), 62.252329ms from start of poll, 62.646306ms from cluster start. Timeout at 2025-02-10 11:34:14.480541566 +0000 UTC m=+10.849445886, 105.07241ms after stability testing.go:149: cluster: 2025-02-10T11:34:14.480Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.375469175 +0000 UTC m=+10.744373476 (1 nodes), 62.252329ms from start of poll, 62.646306ms from cluster start. Timeout at 2025-02-10 11:34:14.480541566 +0000 UTC m=+10.849445886, 105.07241ms after stability testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.480Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: starting snapshot up to: index=12 testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.480Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: starting snapshot up to: index=12 testing.go:153: 2025-02-10T11:34:14.480Z [INFO] creating new snapshot: path=/tmp/raft529662243/snapshots/2-12-1739187254480.tmp testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.484Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: snapshot complete up to: index=12 testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.484Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: snapshot complete up to: index=12 testing.go:153: 2025-02-10T11:34:14.484Z [INFO] creating new snapshot: path=/tmp/raft529662243/snapshots/2-23-1739187254484.tmp testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.487Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: copied to local snapshot: bytes=71 testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.487Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: restored user snapshot: index=1 testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: restored user snapshot: index=1 testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.488Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.496Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.496Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:14.496Z [INFO] creating new snapshot: path=/tmp/raft031717197/snapshots/2-23-1739187254496.tmp testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.497Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.497Z [ERROR] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:14.497Z [INFO] creating new snapshot: path=/tmp/raft407137543/snapshots/2-23-1739187254497.tmp testLoggerAdapter verbose: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.501Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: copied to local snapshot: bytes=71 testing.go:149: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.501Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.501Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: Installed remote snapshot testing.go:149: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.501Z [INFO] server-d8248c37-3322-d35d-4ed6-2d9614664faf: Installed remote snapshot testLoggerAdapter verbose: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.501Z [WARN] server-d8248c37-3322-d35d-4ed6-2d9614664faf: skipping application of old log: index=22 testing.go:149: server-d8248c37-3322-d35d-4ed6-2d9614664faf: 2025-02-10T11:34:14.501Z [WARN] server-d8248c37-3322-d35d-4ed6-2d9614664faf: skipping application of old log: index=22 testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.501Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.501Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testLoggerAdapter verbose: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.501Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: copied to local snapshot: bytes=71 testing.go:149: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.501Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.501Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: Installed remote snapshot testing.go:149: server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: 2025-02-10T11:34:14.501Z [INFO] server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb: Installed remote snapshot testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.501Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.501Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: pipelining replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.516Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.516Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-b76869c8-0eeb-cb7f-c033-48ec73fbfeeb b76869c8-0eeb-cb7f-c033-48ec73fbfeeb}" testLoggerAdapter verbose: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.516Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testing.go:149: server-8e39bd5e-9334-fe43-6354-f78b50d41f05: 2025-02-10T11:34:14.516Z [INFO] server-8e39bd5e-9334-fe43-6354-f78b50d41f05: aborting pipeline replication: peer="{Voter server-d8248c37-3322-d35d-4ed6-2d9614664faf d8248c37-3322-d35d-4ed6-2d9614664faf}" testLoggerAdapter verbose: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.518Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-77142aee-2979-634e-7a43-baea0fa60b72 Address:77142aee-2979-634e-7a43-baea0fa60b72} {Suffrage:Voter ID:server-cc1ddb75-ee67-a616-1d79-141e839e4263 Address:cc1ddb75-ee67-a616-1d79-141e839e4263} {Suffrage:Voter ID:server-7073b9b5-b424-61bd-522e-811dbc4994b5 Address:7073b9b5-b424-61bd-522e-811dbc4994b5}]" testing.go:149: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.518Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-77142aee-2979-634e-7a43-baea0fa60b72 Address:77142aee-2979-634e-7a43-baea0fa60b72} {Suffrage:Voter ID:server-cc1ddb75-ee67-a616-1d79-141e839e4263 Address:cc1ddb75-ee67-a616-1d79-141e839e4263} {Suffrage:Voter ID:server-7073b9b5-b424-61bd-522e-811dbc4994b5 Address:7073b9b5-b424-61bd-522e-811dbc4994b5}]" testLoggerAdapter verbose: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.518Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: entering follower state: follower="Node at 77142aee-2979-634e-7a43-baea0fa60b72 [Follower]" leader= testing.go:149: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.518Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: entering follower state: follower="Node at 77142aee-2979-634e-7a43-baea0fa60b72 [Follower]" leader= testLoggerAdapter verbose: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.518Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-77142aee-2979-634e-7a43-baea0fa60b72 Address:77142aee-2979-634e-7a43-baea0fa60b72} {Suffrage:Voter ID:server-cc1ddb75-ee67-a616-1d79-141e839e4263 Address:cc1ddb75-ee67-a616-1d79-141e839e4263} {Suffrage:Voter ID:server-7073b9b5-b424-61bd-522e-811dbc4994b5 Address:7073b9b5-b424-61bd-522e-811dbc4994b5}]" testing.go:149: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.518Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-77142aee-2979-634e-7a43-baea0fa60b72 Address:77142aee-2979-634e-7a43-baea0fa60b72} {Suffrage:Voter ID:server-cc1ddb75-ee67-a616-1d79-141e839e4263 Address:cc1ddb75-ee67-a616-1d79-141e839e4263} {Suffrage:Voter ID:server-7073b9b5-b424-61bd-522e-811dbc4994b5 Address:7073b9b5-b424-61bd-522e-811dbc4994b5}]" testLoggerAdapter verbose: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.518Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: entering follower state: follower="Node at cc1ddb75-ee67-a616-1d79-141e839e4263 [Follower]" leader= testing.go:149: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.518Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: entering follower state: follower="Node at cc1ddb75-ee67-a616-1d79-141e839e4263 [Follower]" leader= testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.518Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-77142aee-2979-634e-7a43-baea0fa60b72 Address:77142aee-2979-634e-7a43-baea0fa60b72} {Suffrage:Voter ID:server-cc1ddb75-ee67-a616-1d79-141e839e4263 Address:cc1ddb75-ee67-a616-1d79-141e839e4263} {Suffrage:Voter ID:server-7073b9b5-b424-61bd-522e-811dbc4994b5 Address:7073b9b5-b424-61bd-522e-811dbc4994b5}]" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.518Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-77142aee-2979-634e-7a43-baea0fa60b72 Address:77142aee-2979-634e-7a43-baea0fa60b72} {Suffrage:Voter ID:server-cc1ddb75-ee67-a616-1d79-141e839e4263 Address:cc1ddb75-ee67-a616-1d79-141e839e4263} {Suffrage:Voter ID:server-7073b9b5-b424-61bd-522e-811dbc4994b5 Address:7073b9b5-b424-61bd-522e-811dbc4994b5}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.518Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:14.518Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.518Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: entering follower state: follower="Node at 7073b9b5-b424-61bd-522e-811dbc4994b5 [Follower]" leader= testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.518Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: entering follower state: follower="Node at 7073b9b5-b424-61bd-522e-811dbc4994b5 [Follower]" leader= testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [WARN] server-7073b9b5-b424-61bd-522e-811dbc4994b5: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [WARN] server-7073b9b5-b424-61bd-522e-811dbc4994b5: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: entering candidate state: node="Node at 7073b9b5-b424-61bd-522e-811dbc4994b5 [Candidate]" term=2 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: entering candidate state: node="Node at 7073b9b5-b424-61bd-522e-811dbc4994b5 [Candidate]" term=2 testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: election won: tally=2 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: election won: tally=2 testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: entering leader state: leader="Node at 7073b9b5-b424-61bd-522e-811dbc4994b5 [Leader]" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: entering leader state: leader="Node at 7073b9b5-b424-61bd-522e-811dbc4994b5 [Leader]" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: added peer, starting replication: peer=server-77142aee-2979-634e-7a43-baea0fa60b72 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: added peer, starting replication: peer=server-77142aee-2979-634e-7a43-baea0fa60b72 testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: added peer, starting replication: peer=server-cc1ddb75-ee67-a616-1d79-141e839e4263 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: added peer, starting replication: peer=server-cc1ddb75-ee67-a616-1d79-141e839e4263 testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.576Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.681Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.576774433 +0000 UTC m=+10.945678732 (1 nodes), 58.284644ms from start of poll, 58.664008ms from cluster start. Timeout at 2025-02-10 11:34:14.681846899 +0000 UTC m=+11.050751195, 105.072463ms after stability testing.go:149: cluster: 2025-02-10T11:34:14.681Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.576774433 +0000 UTC m=+10.945678732 (1 nodes), 58.284644ms from start of poll, 58.664008ms from cluster start. Timeout at 2025-02-10 11:34:14.681846899 +0000 UTC m=+11.050751195, 105.072463ms after stability testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.682Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: starting snapshot up to: index=12 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.682Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: starting snapshot up to: index=12 testing.go:153: 2025-02-10T11:34:14.682Z [INFO] creating new snapshot: path=/tmp/raft777606293/snapshots/2-12-1739187254682.tmp testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.686Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: snapshot complete up to: index=12 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.686Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: snapshot complete up to: index=12 testing.go:153: 2025-02-10T11:34:14.686Z [INFO] creating new snapshot: path=/tmp/raft777606293/snapshots/2-23-1739187254686.tmp testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.689Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: copied to local snapshot: bytes=71 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.689Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: restored user snapshot: index=1 testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: restored user snapshot: index=1 testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.690Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.696Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.696Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:14.696Z [INFO] creating new snapshot: path=/tmp/raft683169489/snapshots/2-23-1739187254696.tmp testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.697Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.697Z [ERROR] server-7073b9b5-b424-61bd-522e-811dbc4994b5: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:14.697Z [INFO] creating new snapshot: path=/tmp/raft856493867/snapshots/2-23-1739187254697.tmp testLoggerAdapter verbose: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.700Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: copied to local snapshot: bytes=71 testing.go:149: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.700Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.700Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: Installed remote snapshot testing.go:149: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.700Z [INFO] server-77142aee-2979-634e-7a43-baea0fa60b72: Installed remote snapshot testLoggerAdapter verbose: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.700Z [WARN] server-77142aee-2979-634e-7a43-baea0fa60b72: skipping application of old log: index=22 testing.go:149: server-77142aee-2979-634e-7a43-baea0fa60b72: 2025-02-10T11:34:14.700Z [WARN] server-77142aee-2979-634e-7a43-baea0fa60b72: skipping application of old log: index=22 testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.700Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.700Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testLoggerAdapter verbose: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.701Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: copied to local snapshot: bytes=71 testing.go:149: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.701Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.701Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: Installed remote snapshot testing.go:149: server-cc1ddb75-ee67-a616-1d79-141e839e4263: 2025-02-10T11:34:14.701Z [INFO] server-cc1ddb75-ee67-a616-1d79-141e839e4263: Installed remote snapshot testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.701Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.701Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: pipelining replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.716Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.716Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-cc1ddb75-ee67-a616-1d79-141e839e4263 cc1ddb75-ee67-a616-1d79-141e839e4263}" testLoggerAdapter verbose: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.716Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testing.go:149: server-7073b9b5-b424-61bd-522e-811dbc4994b5: 2025-02-10T11:34:14.716Z [INFO] server-7073b9b5-b424-61bd-522e-811dbc4994b5: aborting pipeline replication: peer="{Voter server-77142aee-2979-634e-7a43-baea0fa60b72 77142aee-2979-634e-7a43-baea0fa60b72}" testLoggerAdapter verbose: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.717Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 Address:90c0e5c1-c290-77a3-9bb1-82eedf965d36} {Suffrage:Voter ID:server-70230d0f-0411-338a-1dac-ec8f9531cd81 Address:70230d0f-0411-338a-1dac-ec8f9531cd81} {Suffrage:Voter ID:server-e84355b1-6bed-c60e-79ae-a49db53f455b Address:e84355b1-6bed-c60e-79ae-a49db53f455b}]" testing.go:149: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.717Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 Address:90c0e5c1-c290-77a3-9bb1-82eedf965d36} {Suffrage:Voter ID:server-70230d0f-0411-338a-1dac-ec8f9531cd81 Address:70230d0f-0411-338a-1dac-ec8f9531cd81} {Suffrage:Voter ID:server-e84355b1-6bed-c60e-79ae-a49db53f455b Address:e84355b1-6bed-c60e-79ae-a49db53f455b}]" testLoggerAdapter verbose: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.717Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: entering follower state: follower="Node at 90c0e5c1-c290-77a3-9bb1-82eedf965d36 [Follower]" leader= testing.go:149: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.717Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: entering follower state: follower="Node at 90c0e5c1-c290-77a3-9bb1-82eedf965d36 [Follower]" leader= testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.717Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 Address:90c0e5c1-c290-77a3-9bb1-82eedf965d36} {Suffrage:Voter ID:server-70230d0f-0411-338a-1dac-ec8f9531cd81 Address:70230d0f-0411-338a-1dac-ec8f9531cd81} {Suffrage:Voter ID:server-e84355b1-6bed-c60e-79ae-a49db53f455b Address:e84355b1-6bed-c60e-79ae-a49db53f455b}]" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.717Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 Address:90c0e5c1-c290-77a3-9bb1-82eedf965d36} {Suffrage:Voter ID:server-70230d0f-0411-338a-1dac-ec8f9531cd81 Address:70230d0f-0411-338a-1dac-ec8f9531cd81} {Suffrage:Voter ID:server-e84355b1-6bed-c60e-79ae-a49db53f455b Address:e84355b1-6bed-c60e-79ae-a49db53f455b}]" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.717Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: entering follower state: follower="Node at 70230d0f-0411-338a-1dac-ec8f9531cd81 [Follower]" leader= testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.717Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: entering follower state: follower="Node at 70230d0f-0411-338a-1dac-ec8f9531cd81 [Follower]" leader= testLoggerAdapter verbose: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.717Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 Address:90c0e5c1-c290-77a3-9bb1-82eedf965d36} {Suffrage:Voter ID:server-70230d0f-0411-338a-1dac-ec8f9531cd81 Address:70230d0f-0411-338a-1dac-ec8f9531cd81} {Suffrage:Voter ID:server-e84355b1-6bed-c60e-79ae-a49db53f455b Address:e84355b1-6bed-c60e-79ae-a49db53f455b}]" testing.go:149: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.717Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 Address:90c0e5c1-c290-77a3-9bb1-82eedf965d36} {Suffrage:Voter ID:server-70230d0f-0411-338a-1dac-ec8f9531cd81 Address:70230d0f-0411-338a-1dac-ec8f9531cd81} {Suffrage:Voter ID:server-e84355b1-6bed-c60e-79ae-a49db53f455b Address:e84355b1-6bed-c60e-79ae-a49db53f455b}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.717Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:14.717Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.717Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: entering follower state: follower="Node at e84355b1-6bed-c60e-79ae-a49db53f455b [Follower]" leader= testing.go:149: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.717Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: entering follower state: follower="Node at e84355b1-6bed-c60e-79ae-a49db53f455b [Follower]" leader= testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [WARN] server-70230d0f-0411-338a-1dac-ec8f9531cd81: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [WARN] server-70230d0f-0411-338a-1dac-ec8f9531cd81: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: entering candidate state: node="Node at 70230d0f-0411-338a-1dac-ec8f9531cd81 [Candidate]" term=2 testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: entering candidate state: node="Node at 70230d0f-0411-338a-1dac-ec8f9531cd81 [Candidate]" term=2 testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: election won: tally=2 testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: election won: tally=2 testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: entering leader state: leader="Node at 70230d0f-0411-338a-1dac-ec8f9531cd81 [Leader]" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: entering leader state: leader="Node at 70230d0f-0411-338a-1dac-ec8f9531cd81 [Leader]" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: added peer, starting replication: peer=server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: added peer, starting replication: peer=server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: added peer, starting replication: peer=server-e84355b1-6bed-c60e-79ae-a49db53f455b testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: added peer, starting replication: peer=server-e84355b1-6bed-c60e-79ae-a49db53f455b testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.805Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.910Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.805272425 +0000 UTC m=+11.174176733 (1 nodes), 87.600354ms from start of poll, 87.975102ms from cluster start. Timeout at 2025-02-10 11:34:14.910348228 +0000 UTC m=+11.279252525, 105.075792ms after stability testing.go:149: cluster: 2025-02-10T11:34:14.910Z [INFO] stable state for Leader reached at 2025-02-10 11:34:14.805272425 +0000 UTC m=+11.174176733 (1 nodes), 87.600354ms from start of poll, 87.975102ms from cluster start. Timeout at 2025-02-10 11:34:14.910348228 +0000 UTC m=+11.279252525, 105.075792ms after stability testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.910Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: starting snapshot up to: index=12 testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.910Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: starting snapshot up to: index=12 testing.go:153: 2025-02-10T11:34:14.910Z [INFO] creating new snapshot: path=/tmp/raft193024153/snapshots/2-12-1739187254910.tmp testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.914Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: snapshot complete up to: index=12 testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.914Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: snapshot complete up to: index=12 testing.go:153: 2025-02-10T11:34:14.914Z [INFO] creating new snapshot: path=/tmp/raft193024153/snapshots/2-23-1739187254914.tmp testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: copied to local snapshot: bytes=71 testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: restored user snapshot: index=1 testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: restored user snapshot: index=1 testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.918Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.925Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.925Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:14.925Z [INFO] creating new snapshot: path=/tmp/raft729165199/snapshots/2-23-1739187254925.tmp testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.927Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.927Z [ERROR] server-70230d0f-0411-338a-1dac-ec8f9531cd81: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:14.928Z [INFO] creating new snapshot: path=/tmp/raft913532979/snapshots/2-23-1739187254928.tmp testLoggerAdapter verbose: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.928Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: copied to local snapshot: bytes=71 testing.go:149: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.928Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.928Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: Installed remote snapshot testing.go:149: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.928Z [INFO] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: Installed remote snapshot testLoggerAdapter verbose: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.928Z [WARN] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: skipping application of old log: index=22 testing.go:149: server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: 2025-02-10T11:34:14.928Z [WARN] server-90c0e5c1-c290-77a3-9bb1-82eedf965d36: skipping application of old log: index=22 testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.928Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.928Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testLoggerAdapter verbose: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.931Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: copied to local snapshot: bytes=71 testing.go:149: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.931Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.931Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: Installed remote snapshot testing.go:149: server-e84355b1-6bed-c60e-79ae-a49db53f455b: 2025-02-10T11:34:14.931Z [INFO] server-e84355b1-6bed-c60e-79ae-a49db53f455b: Installed remote snapshot testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.931Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.931Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: pipelining replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.944Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.944Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-e84355b1-6bed-c60e-79ae-a49db53f455b e84355b1-6bed-c60e-79ae-a49db53f455b}" testLoggerAdapter verbose: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.944Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testing.go:149: server-70230d0f-0411-338a-1dac-ec8f9531cd81: 2025-02-10T11:34:14.944Z [INFO] server-70230d0f-0411-338a-1dac-ec8f9531cd81: aborting pipeline replication: peer="{Voter server-90c0e5c1-c290-77a3-9bb1-82eedf965d36 90c0e5c1-c290-77a3-9bb1-82eedf965d36}" testLoggerAdapter verbose: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:14.945Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c47cd624-b2c4-794d-fb56-76673ad2ec12 Address:c47cd624-b2c4-794d-fb56-76673ad2ec12} {Suffrage:Voter ID:server-5e1ed750-a47f-50bf-4865-fd3b3e23a726 Address:5e1ed750-a47f-50bf-4865-fd3b3e23a726} {Suffrage:Voter ID:server-8fb74d80-05b8-90f9-e4e9-a17f5500abee Address:8fb74d80-05b8-90f9-e4e9-a17f5500abee}]" testing.go:149: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:14.945Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c47cd624-b2c4-794d-fb56-76673ad2ec12 Address:c47cd624-b2c4-794d-fb56-76673ad2ec12} {Suffrage:Voter ID:server-5e1ed750-a47f-50bf-4865-fd3b3e23a726 Address:5e1ed750-a47f-50bf-4865-fd3b3e23a726} {Suffrage:Voter ID:server-8fb74d80-05b8-90f9-e4e9-a17f5500abee Address:8fb74d80-05b8-90f9-e4e9-a17f5500abee}]" testLoggerAdapter verbose: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:14.945Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: entering follower state: follower="Node at c47cd624-b2c4-794d-fb56-76673ad2ec12 [Follower]" leader= testing.go:149: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:14.945Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: entering follower state: follower="Node at c47cd624-b2c4-794d-fb56-76673ad2ec12 [Follower]" leader= testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:14.945Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c47cd624-b2c4-794d-fb56-76673ad2ec12 Address:c47cd624-b2c4-794d-fb56-76673ad2ec12} {Suffrage:Voter ID:server-5e1ed750-a47f-50bf-4865-fd3b3e23a726 Address:5e1ed750-a47f-50bf-4865-fd3b3e23a726} {Suffrage:Voter ID:server-8fb74d80-05b8-90f9-e4e9-a17f5500abee Address:8fb74d80-05b8-90f9-e4e9-a17f5500abee}]" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:14.945Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c47cd624-b2c4-794d-fb56-76673ad2ec12 Address:c47cd624-b2c4-794d-fb56-76673ad2ec12} {Suffrage:Voter ID:server-5e1ed750-a47f-50bf-4865-fd3b3e23a726 Address:5e1ed750-a47f-50bf-4865-fd3b3e23a726} {Suffrage:Voter ID:server-8fb74d80-05b8-90f9-e4e9-a17f5500abee Address:8fb74d80-05b8-90f9-e4e9-a17f5500abee}]" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:14.945Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: entering follower state: follower="Node at 5e1ed750-a47f-50bf-4865-fd3b3e23a726 [Follower]" leader= testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:14.945Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: entering follower state: follower="Node at 5e1ed750-a47f-50bf-4865-fd3b3e23a726 [Follower]" leader= testLoggerAdapter verbose: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:14.945Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c47cd624-b2c4-794d-fb56-76673ad2ec12 Address:c47cd624-b2c4-794d-fb56-76673ad2ec12} {Suffrage:Voter ID:server-5e1ed750-a47f-50bf-4865-fd3b3e23a726 Address:5e1ed750-a47f-50bf-4865-fd3b3e23a726} {Suffrage:Voter ID:server-8fb74d80-05b8-90f9-e4e9-a17f5500abee Address:8fb74d80-05b8-90f9-e4e9-a17f5500abee}]" testing.go:149: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:14.945Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c47cd624-b2c4-794d-fb56-76673ad2ec12 Address:c47cd624-b2c4-794d-fb56-76673ad2ec12} {Suffrage:Voter ID:server-5e1ed750-a47f-50bf-4865-fd3b3e23a726 Address:5e1ed750-a47f-50bf-4865-fd3b3e23a726} {Suffrage:Voter ID:server-8fb74d80-05b8-90f9-e4e9-a17f5500abee Address:8fb74d80-05b8-90f9-e4e9-a17f5500abee}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:14.945Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:14.945Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:14.945Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: entering follower state: follower="Node at 8fb74d80-05b8-90f9-e4e9-a17f5500abee [Follower]" leader= testing.go:149: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:14.945Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: entering follower state: follower="Node at 8fb74d80-05b8-90f9-e4e9-a17f5500abee [Follower]" leader= testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [WARN] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [WARN] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: entering candidate state: node="Node at 5e1ed750-a47f-50bf-4865-fd3b3e23a726 [Candidate]" term=2 testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: entering candidate state: node="Node at 5e1ed750-a47f-50bf-4865-fd3b3e23a726 [Candidate]" term=2 testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: election won: tally=2 testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: election won: tally=2 testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: entering leader state: leader="Node at 5e1ed750-a47f-50bf-4865-fd3b3e23a726 [Leader]" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: entering leader state: leader="Node at 5e1ed750-a47f-50bf-4865-fd3b3e23a726 [Leader]" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: added peer, starting replication: peer=server-c47cd624-b2c4-794d-fb56-76673ad2ec12 testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: added peer, starting replication: peer=server-c47cd624-b2c4-794d-fb56-76673ad2ec12 testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: added peer, starting replication: peer=server-8fb74d80-05b8-90f9-e4e9-a17f5500abee testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: added peer, starting replication: peer=server-8fb74d80-05b8-90f9-e4e9-a17f5500abee testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.028Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.133Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.028574236 +0000 UTC m=+11.397478553 (1 nodes), 82.727978ms from start of poll, 83.110354ms from cluster start. Timeout at 2025-02-10 11:34:15.133646678 +0000 UTC m=+11.502550974, 105.072421ms after stability testing.go:149: cluster: 2025-02-10T11:34:15.133Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.028574236 +0000 UTC m=+11.397478553 (1 nodes), 82.727978ms from start of poll, 83.110354ms from cluster start. Timeout at 2025-02-10 11:34:15.133646678 +0000 UTC m=+11.502550974, 105.072421ms after stability testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.133Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: starting snapshot up to: index=12 testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.133Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: starting snapshot up to: index=12 testing.go:153: 2025-02-10T11:34:15.133Z [INFO] creating new snapshot: path=/tmp/raft960678167/snapshots/2-12-1739187255133.tmp testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.138Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: snapshot complete up to: index=12 testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.138Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: snapshot complete up to: index=12 testing.go:153: 2025-02-10T11:34:15.138Z [INFO] creating new snapshot: path=/tmp/raft960678167/snapshots/2-113-1739187255138.tmp testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: copied to local snapshot: bytes=71 testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: restored user snapshot: index=1 testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: restored user snapshot: index=1 testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.141Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.147Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.147Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:15.147Z [INFO] creating new snapshot: path=/tmp/raft984029025/snapshots/2-113-1739187255147.tmp testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.150Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.150Z [ERROR] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:15.150Z [INFO] creating new snapshot: path=/tmp/raft320489181/snapshots/2-113-1739187255150.tmp testLoggerAdapter verbose: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:15.150Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: copied to local snapshot: bytes=71 testing.go:149: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:15.150Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:15.150Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: Installed remote snapshot testing.go:149: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:15.150Z [INFO] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: Installed remote snapshot testLoggerAdapter verbose: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:15.150Z [WARN] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: skipping application of old log: index=22 testing.go:149: server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: 2025-02-10T11:34:15.150Z [WARN] server-8fb74d80-05b8-90f9-e4e9-a17f5500abee: skipping application of old log: index=22 testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.150Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.150Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testLoggerAdapter verbose: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:15.153Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: copied to local snapshot: bytes=71 testing.go:149: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:15.153Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:15.153Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: Installed remote snapshot testing.go:149: server-c47cd624-b2c4-794d-fb56-76673ad2ec12: 2025-02-10T11:34:15.153Z [INFO] server-c47cd624-b2c4-794d-fb56-76673ad2ec12: Installed remote snapshot testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.153Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.153Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: pipelining replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.166Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.166Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-8fb74d80-05b8-90f9-e4e9-a17f5500abee 8fb74d80-05b8-90f9-e4e9-a17f5500abee}" testLoggerAdapter verbose: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.166Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testing.go:149: server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: 2025-02-10T11:34:15.166Z [INFO] server-5e1ed750-a47f-50bf-4865-fd3b3e23a726: aborting pipeline replication: peer="{Voter server-c47cd624-b2c4-794d-fb56-76673ad2ec12 c47cd624-b2c4-794d-fb56-76673ad2ec12}" testLoggerAdapter verbose: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.167Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e61a1425-2820-0d76-ef22-b9252688f802 Address:e61a1425-2820-0d76-ef22-b9252688f802} {Suffrage:Voter ID:server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a Address:0e8b1fa8-63d7-7647-a9bb-65a1b951d56a} {Suffrage:Voter ID:server-27d2e67c-b3a2-be67-1885-adfbaf815739 Address:27d2e67c-b3a2-be67-1885-adfbaf815739}]" testing.go:149: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.167Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e61a1425-2820-0d76-ef22-b9252688f802 Address:e61a1425-2820-0d76-ef22-b9252688f802} {Suffrage:Voter ID:server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a Address:0e8b1fa8-63d7-7647-a9bb-65a1b951d56a} {Suffrage:Voter ID:server-27d2e67c-b3a2-be67-1885-adfbaf815739 Address:27d2e67c-b3a2-be67-1885-adfbaf815739}]" testLoggerAdapter verbose: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.167Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: entering follower state: follower="Node at e61a1425-2820-0d76-ef22-b9252688f802 [Follower]" leader= testing.go:149: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.167Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: entering follower state: follower="Node at e61a1425-2820-0d76-ef22-b9252688f802 [Follower]" leader= testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.168Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e61a1425-2820-0d76-ef22-b9252688f802 Address:e61a1425-2820-0d76-ef22-b9252688f802} {Suffrage:Voter ID:server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a Address:0e8b1fa8-63d7-7647-a9bb-65a1b951d56a} {Suffrage:Voter ID:server-27d2e67c-b3a2-be67-1885-adfbaf815739 Address:27d2e67c-b3a2-be67-1885-adfbaf815739}]" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.168Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e61a1425-2820-0d76-ef22-b9252688f802 Address:e61a1425-2820-0d76-ef22-b9252688f802} {Suffrage:Voter ID:server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a Address:0e8b1fa8-63d7-7647-a9bb-65a1b951d56a} {Suffrage:Voter ID:server-27d2e67c-b3a2-be67-1885-adfbaf815739 Address:27d2e67c-b3a2-be67-1885-adfbaf815739}]" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.168Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: entering follower state: follower="Node at 0e8b1fa8-63d7-7647-a9bb-65a1b951d56a [Follower]" leader= testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.168Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: entering follower state: follower="Node at 0e8b1fa8-63d7-7647-a9bb-65a1b951d56a [Follower]" leader= testLoggerAdapter verbose: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.168Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e61a1425-2820-0d76-ef22-b9252688f802 Address:e61a1425-2820-0d76-ef22-b9252688f802} {Suffrage:Voter ID:server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a Address:0e8b1fa8-63d7-7647-a9bb-65a1b951d56a} {Suffrage:Voter ID:server-27d2e67c-b3a2-be67-1885-adfbaf815739 Address:27d2e67c-b3a2-be67-1885-adfbaf815739}]" testing.go:149: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.168Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e61a1425-2820-0d76-ef22-b9252688f802 Address:e61a1425-2820-0d76-ef22-b9252688f802} {Suffrage:Voter ID:server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a Address:0e8b1fa8-63d7-7647-a9bb-65a1b951d56a} {Suffrage:Voter ID:server-27d2e67c-b3a2-be67-1885-adfbaf815739 Address:27d2e67c-b3a2-be67-1885-adfbaf815739}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.168Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:15.168Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.168Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: entering follower state: follower="Node at 27d2e67c-b3a2-be67-1885-adfbaf815739 [Follower]" leader= testing.go:149: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.168Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: entering follower state: follower="Node at 27d2e67c-b3a2-be67-1885-adfbaf815739 [Follower]" leader= testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [WARN] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [WARN] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: entering candidate state: node="Node at 0e8b1fa8-63d7-7647-a9bb-65a1b951d56a [Candidate]" term=2 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: entering candidate state: node="Node at 0e8b1fa8-63d7-7647-a9bb-65a1b951d56a [Candidate]" term=2 testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: election won: tally=2 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: election won: tally=2 testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: entering leader state: leader="Node at 0e8b1fa8-63d7-7647-a9bb-65a1b951d56a [Leader]" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: entering leader state: leader="Node at 0e8b1fa8-63d7-7647-a9bb-65a1b951d56a [Leader]" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: added peer, starting replication: peer=server-e61a1425-2820-0d76-ef22-b9252688f802 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: added peer, starting replication: peer=server-e61a1425-2820-0d76-ef22-b9252688f802 testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: added peer, starting replication: peer=server-27d2e67c-b3a2-be67-1885-adfbaf815739 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: added peer, starting replication: peer=server-27d2e67c-b3a2-be67-1885-adfbaf815739 testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.233Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.338Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.233682874 +0000 UTC m=+11.602587174 (1 nodes), 65.439297ms from start of poll, 65.91897ms from cluster start. Timeout at 2025-02-10 11:34:15.338757709 +0000 UTC m=+11.707662006, 105.074832ms after stability testing.go:149: cluster: 2025-02-10T11:34:15.338Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.233682874 +0000 UTC m=+11.602587174 (1 nodes), 65.439297ms from start of poll, 65.91897ms from cluster start. Timeout at 2025-02-10 11:34:15.338757709 +0000 UTC m=+11.707662006, 105.074832ms after stability testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.338Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: starting snapshot up to: index=12 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.338Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: starting snapshot up to: index=12 testing.go:153: 2025-02-10T11:34:15.338Z [INFO] creating new snapshot: path=/tmp/raft142705189/snapshots/2-12-1739187255338.tmp testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.342Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: snapshot complete up to: index=12 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.342Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: snapshot complete up to: index=12 testing.go:153: 2025-02-10T11:34:15.343Z [INFO] creating new snapshot: path=/tmp/raft142705189/snapshots/2-1013-1739187255343.tmp testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.346Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: copied to local snapshot: bytes=71 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.346Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.346Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: restored user snapshot: index=1 testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.346Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: restored user snapshot: index=1 testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.347Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.352Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.352Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:15.353Z [INFO] creating new snapshot: path=/tmp/raft168087099/snapshots/2-1013-1739187255353.tmp testLoggerAdapter verbose: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.356Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: copied to local snapshot: bytes=71 testing.go:149: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.356Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.356Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: Installed remote snapshot testing.go:149: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.356Z [INFO] server-e61a1425-2820-0d76-ef22-b9252688f802: Installed remote snapshot testLoggerAdapter verbose: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.356Z [WARN] server-e61a1425-2820-0d76-ef22-b9252688f802: skipping application of old log: index=1013 testing.go:149: server-e61a1425-2820-0d76-ef22-b9252688f802: 2025-02-10T11:34:15.356Z [WARN] server-e61a1425-2820-0d76-ef22-b9252688f802: skipping application of old log: index=1013 testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.356Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.356Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.356Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.356Z [ERROR] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:15.357Z [INFO] creating new snapshot: path=/tmp/raft984375199/snapshots/2-1013-1739187255357.tmp testLoggerAdapter verbose: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.360Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: copied to local snapshot: bytes=71 testing.go:149: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.360Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.360Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: Installed remote snapshot testing.go:149: server-27d2e67c-b3a2-be67-1885-adfbaf815739: 2025-02-10T11:34:15.360Z [INFO] server-27d2e67c-b3a2-be67-1885-adfbaf815739: Installed remote snapshot testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.360Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.360Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: pipelining replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.372Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.372Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-27d2e67c-b3a2-be67-1885-adfbaf815739 27d2e67c-b3a2-be67-1885-adfbaf815739}" testLoggerAdapter verbose: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.372Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testing.go:149: server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: 2025-02-10T11:34:15.372Z [INFO] server-0e8b1fa8-63d7-7647-a9bb-65a1b951d56a: aborting pipeline replication: peer="{Voter server-e61a1425-2820-0d76-ef22-b9252688f802 e61a1425-2820-0d76-ef22-b9252688f802}" testLoggerAdapter verbose: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.373Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9460d8c4-1a97-026d-d752-fe77c5640676 Address:9460d8c4-1a97-026d-d752-fe77c5640676} {Suffrage:Voter ID:server-f3e90ecb-61fa-597c-f955-b9aa253138ca Address:f3e90ecb-61fa-597c-f955-b9aa253138ca} {Suffrage:Voter ID:server-e25eecab-c323-3c4f-b52e-031f09fec682 Address:e25eecab-c323-3c4f-b52e-031f09fec682}]" testing.go:149: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.373Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9460d8c4-1a97-026d-d752-fe77c5640676 Address:9460d8c4-1a97-026d-d752-fe77c5640676} {Suffrage:Voter ID:server-f3e90ecb-61fa-597c-f955-b9aa253138ca Address:f3e90ecb-61fa-597c-f955-b9aa253138ca} {Suffrage:Voter ID:server-e25eecab-c323-3c4f-b52e-031f09fec682 Address:e25eecab-c323-3c4f-b52e-031f09fec682}]" testLoggerAdapter verbose: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.373Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: entering follower state: follower="Node at 9460d8c4-1a97-026d-d752-fe77c5640676 [Follower]" leader= testing.go:149: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.373Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: entering follower state: follower="Node at 9460d8c4-1a97-026d-d752-fe77c5640676 [Follower]" leader= testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.373Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9460d8c4-1a97-026d-d752-fe77c5640676 Address:9460d8c4-1a97-026d-d752-fe77c5640676} {Suffrage:Voter ID:server-f3e90ecb-61fa-597c-f955-b9aa253138ca Address:f3e90ecb-61fa-597c-f955-b9aa253138ca} {Suffrage:Voter ID:server-e25eecab-c323-3c4f-b52e-031f09fec682 Address:e25eecab-c323-3c4f-b52e-031f09fec682}]" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.373Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9460d8c4-1a97-026d-d752-fe77c5640676 Address:9460d8c4-1a97-026d-d752-fe77c5640676} {Suffrage:Voter ID:server-f3e90ecb-61fa-597c-f955-b9aa253138ca Address:f3e90ecb-61fa-597c-f955-b9aa253138ca} {Suffrage:Voter ID:server-e25eecab-c323-3c4f-b52e-031f09fec682 Address:e25eecab-c323-3c4f-b52e-031f09fec682}]" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.373Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: entering follower state: follower="Node at f3e90ecb-61fa-597c-f955-b9aa253138ca [Follower]" leader= testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.373Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: entering follower state: follower="Node at f3e90ecb-61fa-597c-f955-b9aa253138ca [Follower]" leader= testLoggerAdapter verbose: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.373Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9460d8c4-1a97-026d-d752-fe77c5640676 Address:9460d8c4-1a97-026d-d752-fe77c5640676} {Suffrage:Voter ID:server-f3e90ecb-61fa-597c-f955-b9aa253138ca Address:f3e90ecb-61fa-597c-f955-b9aa253138ca} {Suffrage:Voter ID:server-e25eecab-c323-3c4f-b52e-031f09fec682 Address:e25eecab-c323-3c4f-b52e-031f09fec682}]" testing.go:149: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.373Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9460d8c4-1a97-026d-d752-fe77c5640676 Address:9460d8c4-1a97-026d-d752-fe77c5640676} {Suffrage:Voter ID:server-f3e90ecb-61fa-597c-f955-b9aa253138ca Address:f3e90ecb-61fa-597c-f955-b9aa253138ca} {Suffrage:Voter ID:server-e25eecab-c323-3c4f-b52e-031f09fec682 Address:e25eecab-c323-3c4f-b52e-031f09fec682}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.373Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:15.373Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.373Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: entering follower state: follower="Node at e25eecab-c323-3c4f-b52e-031f09fec682 [Follower]" leader= testing.go:149: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.373Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: entering follower state: follower="Node at e25eecab-c323-3c4f-b52e-031f09fec682 [Follower]" leader= testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [WARN] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [WARN] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: entering candidate state: node="Node at f3e90ecb-61fa-597c-f955-b9aa253138ca [Candidate]" term=2 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: entering candidate state: node="Node at f3e90ecb-61fa-597c-f955-b9aa253138ca [Candidate]" term=2 testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: election won: tally=2 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: election won: tally=2 testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: entering leader state: leader="Node at f3e90ecb-61fa-597c-f955-b9aa253138ca [Leader]" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: entering leader state: leader="Node at f3e90ecb-61fa-597c-f955-b9aa253138ca [Leader]" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: added peer, starting replication: peer=server-9460d8c4-1a97-026d-d752-fe77c5640676 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: added peer, starting replication: peer=server-9460d8c4-1a97-026d-d752-fe77c5640676 testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: added peer, starting replication: peer=server-e25eecab-c323-3c4f-b52e-031f09fec682 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: added peer, starting replication: peer=server-e25eecab-c323-3c4f-b52e-031f09fec682 testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.425Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.530Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.425722993 +0000 UTC m=+11.794627294 (1 nodes), 52.122775ms from start of poll, 52.48789ms from cluster start. Timeout at 2025-02-10 11:34:15.530744918 +0000 UTC m=+11.899649216, 105.021922ms after stability testing.go:149: cluster: 2025-02-10T11:34:15.530Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.425722993 +0000 UTC m=+11.794627294 (1 nodes), 52.122775ms from start of poll, 52.48789ms from cluster start. Timeout at 2025-02-10 11:34:15.530744918 +0000 UTC m=+11.899649216, 105.021922ms after stability testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.530Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: starting snapshot up to: index=12 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.530Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: starting snapshot up to: index=12 testing.go:153: 2025-02-10T11:34:15.531Z [INFO] creating new snapshot: path=/tmp/raft973796675/snapshots/2-12-1739187255531.tmp testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.534Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: snapshot complete up to: index=12 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.534Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: snapshot complete up to: index=12 testing.go:153: 2025-02-10T11:34:15.535Z [INFO] creating new snapshot: path=/tmp/raft973796675/snapshots/2-10013-1739187255535.tmp testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: copied to local snapshot: bytes=71 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: restored user snapshot: index=1 testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: restored user snapshot: index=1 testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.538Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.546Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.546Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:15.546Z [INFO] creating new snapshot: path=/tmp/raft692088105/snapshots/2-10013-1739187255546.tmp testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.547Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.547Z [ERROR] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: failed to get log: index=23 error="log not found" testing.go:153: 2025-02-10T11:34:15.547Z [INFO] creating new snapshot: path=/tmp/raft878192237/snapshots/2-10013-1739187255547.tmp testLoggerAdapter verbose: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.551Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: copied to local snapshot: bytes=71 testing.go:149: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.551Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.551Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: Installed remote snapshot testing.go:149: server-9460d8c4-1a97-026d-d752-fe77c5640676: 2025-02-10T11:34:15.551Z [INFO] server-9460d8c4-1a97-026d-d752-fe77c5640676: Installed remote snapshot testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.551Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.551Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" testLoggerAdapter verbose: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.551Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: copied to local snapshot: bytes=71 testing.go:149: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.551Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.552Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: Installed remote snapshot testing.go:149: server-e25eecab-c323-3c4f-b52e-031f09fec682: 2025-02-10T11:34:15.552Z [INFO] server-e25eecab-c323-3c4f-b52e-031f09fec682: Installed remote snapshot testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.552Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.552Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: pipelining replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.567Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.567Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-e25eecab-c323-3c4f-b52e-031f09fec682 e25eecab-c323-3c4f-b52e-031f09fec682}" testLoggerAdapter verbose: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.567Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" testing.go:149: server-f3e90ecb-61fa-597c-f955-b9aa253138ca: 2025-02-10T11:34:15.567Z [INFO] server-f3e90ecb-61fa-597c-f955-b9aa253138ca: aborting pipeline replication: peer="{Voter server-9460d8c4-1a97-026d-d752-fe77c5640676 9460d8c4-1a97-026d-d752-fe77c5640676}" --- PASS: TestRaft_UserRestore (1.26s) === RUN TestRaft_SendSnapshotFollower testLoggerAdapter verbose: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.568Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db9355e2-3f83-6a49-22a3-3b602f6e05ba Address:db9355e2-3f83-6a49-22a3-3b602f6e05ba} {Suffrage:Voter ID:server-cf86f427-6a60-2994-9d2a-f75de2e456ad Address:cf86f427-6a60-2994-9d2a-f75de2e456ad} {Suffrage:Voter ID:server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 Address:2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}]" testing.go:149: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.568Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db9355e2-3f83-6a49-22a3-3b602f6e05ba Address:db9355e2-3f83-6a49-22a3-3b602f6e05ba} {Suffrage:Voter ID:server-cf86f427-6a60-2994-9d2a-f75de2e456ad Address:cf86f427-6a60-2994-9d2a-f75de2e456ad} {Suffrage:Voter ID:server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 Address:2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}]" testLoggerAdapter verbose: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.568Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: entering follower state: follower="Node at db9355e2-3f83-6a49-22a3-3b602f6e05ba [Follower]" leader= testing.go:149: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.568Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: entering follower state: follower="Node at db9355e2-3f83-6a49-22a3-3b602f6e05ba [Follower]" leader= testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.568Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db9355e2-3f83-6a49-22a3-3b602f6e05ba Address:db9355e2-3f83-6a49-22a3-3b602f6e05ba} {Suffrage:Voter ID:server-cf86f427-6a60-2994-9d2a-f75de2e456ad Address:cf86f427-6a60-2994-9d2a-f75de2e456ad} {Suffrage:Voter ID:server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 Address:2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}]" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.568Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db9355e2-3f83-6a49-22a3-3b602f6e05ba Address:db9355e2-3f83-6a49-22a3-3b602f6e05ba} {Suffrage:Voter ID:server-cf86f427-6a60-2994-9d2a-f75de2e456ad Address:cf86f427-6a60-2994-9d2a-f75de2e456ad} {Suffrage:Voter ID:server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 Address:2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}]" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.568Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: entering follower state: follower="Node at cf86f427-6a60-2994-9d2a-f75de2e456ad [Follower]" leader= testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.568Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: entering follower state: follower="Node at cf86f427-6a60-2994-9d2a-f75de2e456ad [Follower]" leader= testLoggerAdapter verbose: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.568Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db9355e2-3f83-6a49-22a3-3b602f6e05ba Address:db9355e2-3f83-6a49-22a3-3b602f6e05ba} {Suffrage:Voter ID:server-cf86f427-6a60-2994-9d2a-f75de2e456ad Address:cf86f427-6a60-2994-9d2a-f75de2e456ad} {Suffrage:Voter ID:server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 Address:2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}]" testing.go:149: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.568Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db9355e2-3f83-6a49-22a3-3b602f6e05ba Address:db9355e2-3f83-6a49-22a3-3b602f6e05ba} {Suffrage:Voter ID:server-cf86f427-6a60-2994-9d2a-f75de2e456ad Address:cf86f427-6a60-2994-9d2a-f75de2e456ad} {Suffrage:Voter ID:server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 Address:2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.568Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:15.568Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.568Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: entering follower state: follower="Node at 2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 [Follower]" leader= testing.go:149: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.568Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: entering follower state: follower="Node at 2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 [Follower]" leader= testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [WARN] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [WARN] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: entering candidate state: node="Node at cf86f427-6a60-2994-9d2a-f75de2e456ad [Candidate]" term=2 testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: entering candidate state: node="Node at cf86f427-6a60-2994-9d2a-f75de2e456ad [Candidate]" term=2 testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: election won: tally=2 testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: election won: tally=2 testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: entering leader state: leader="Node at cf86f427-6a60-2994-9d2a-f75de2e456ad [Leader]" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: entering leader state: leader="Node at cf86f427-6a60-2994-9d2a-f75de2e456ad [Leader]" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: added peer, starting replication: peer=server-db9355e2-3f83-6a49-22a3-3b602f6e05ba testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: added peer, starting replication: peer=server-db9355e2-3f83-6a49-22a3-3b602f6e05ba testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: added peer, starting replication: peer=server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: added peer, starting replication: peer=server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: pipelining replication: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: pipelining replication: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: pipelining replication: peer="{Voter server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.636Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: pipelining replication: peer="{Voter server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.741Z [INFO] stable state for Follower reached at 2025-02-10 11:34:15.636228634 +0000 UTC m=+12.005132950 (2 nodes), 67.272781ms from start of poll, 67.652741ms from cluster start. Timeout at 2025-02-10 11:34:15.741305528 +0000 UTC m=+12.110209822, 105.076872ms after stability testing.go:149: cluster: 2025-02-10T11:34:15.741Z [INFO] stable state for Follower reached at 2025-02-10 11:34:15.636228634 +0000 UTC m=+12.005132950 (2 nodes), 67.272781ms from start of poll, 67.652741ms from cluster start. Timeout at 2025-02-10 11:34:15.741305528 +0000 UTC m=+12.110209822, 105.076872ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.741Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:15.741Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.846Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.741376153 +0000 UTC m=+12.110280458 (1 nodes), 461ns from start of poll, 172.800249ms from cluster start. Timeout at 2025-02-10 11:34:15.846451158 +0000 UTC m=+12.215355454, 105.074996ms after stability testing.go:149: cluster: 2025-02-10T11:34:15.846Z [INFO] stable state for Leader reached at 2025-02-10 11:34:15.741376153 +0000 UTC m=+12.110280458 (1 nodes), 461ns from start of poll, 172.800249ms from cluster start. Timeout at 2025-02-10 11:34:15.846451158 +0000 UTC m=+12.215355454, 105.074996ms after stability testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.846Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to pipeline appendEntries: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="append pipeline closed" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.846Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to pipeline appendEntries: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="append pipeline closed" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.846Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: aborting pipeline replication: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.846Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: aborting pipeline replication: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.846Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to appendEntries to: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.846Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to appendEntries to: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" raft_test.go:1320: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.847Z [ERROR] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.847Z [ERROR] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.847Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: starting snapshot up to: index=102 testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.847Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: starting snapshot up to: index=102 testing.go:153: 2025-02-10T11:34:15.847Z [INFO] creating new snapshot: path=/tmp/raft659035121/snapshots/2-102-1739187255847.tmp testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.851Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to heartbeat to: peer=db9355e2-3f83-6a49-22a3-3b602f6e05ba error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.851Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to heartbeat to: peer=db9355e2-3f83-6a49-22a3-3b602f6e05ba error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.857Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to appendEntries to: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.857Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to appendEntries to: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.867Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to appendEntries to: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.867Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to appendEntries to: peer="{Voter server-db9355e2-3f83-6a49-22a3-3b602f6e05ba db9355e2-3f83-6a49-22a3-3b602f6e05ba}" error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.871Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to heartbeat to: peer=db9355e2-3f83-6a49-22a3-3b602f6e05ba error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.871Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to heartbeat to: peer=db9355e2-3f83-6a49-22a3-3b602f6e05ba error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.887Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to heartbeat to: peer=db9355e2-3f83-6a49-22a3-3b602f6e05ba error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.887Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to heartbeat to: peer=db9355e2-3f83-6a49-22a3-3b602f6e05ba error="failed to connect to peer: db9355e2-3f83-6a49-22a3-3b602f6e05ba" testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.890Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: compacting logs: from=1 to=92 testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.890Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.890Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: snapshot complete up to: index=102 testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.890Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: snapshot complete up to: index=102 testLoggerAdapter verbose: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.890Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: starting snapshot up to: index=102 testing.go:149: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.890Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: starting snapshot up to: index=102 testing.go:153: 2025-02-10T11:34:15.890Z [INFO] creating new snapshot: path=/tmp/raft652765515/snapshots/2-102-1739187255890.tmp testLoggerAdapter verbose: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.893Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: compacting logs: from=1 to=92 testing.go:149: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.893Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.893Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: snapshot complete up to: index=102 testing.go:149: server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: 2025-02-10T11:34:15.893Z [INFO] server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47: snapshot complete up to: index=102 testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.896Z [WARN] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to contact: server-id=server-db9355e2-3f83-6a49-22a3-3b602f6e05ba time=50.10504ms testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.896Z [WARN] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to contact: server-id=server-db9355e2-3f83-6a49-22a3-3b602f6e05ba time=50.10504ms testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.897Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to get log: index=2 error="log not found" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.897Z [ERROR] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: failed to get log: index=2 error="log not found" testing.go:153: 2025-02-10T11:34:15.898Z [INFO] creating new snapshot: path=/tmp/raft056120615/snapshots/2-102-1739187255898.tmp testLoggerAdapter verbose: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.901Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: copied to local snapshot: bytes=693 testing.go:149: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.901Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: copied to local snapshot: bytes=693 testLoggerAdapter verbose: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.901Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: Installed remote snapshot testing.go:149: server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: 2025-02-10T11:34:15.901Z [INFO] server-db9355e2-3f83-6a49-22a3-3b602f6e05ba: Installed remote snapshot testLoggerAdapter verbose: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.903Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: aborting pipeline replication: peer="{Voter server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}" testing.go:149: server-cf86f427-6a60-2994-9d2a-f75de2e456ad: 2025-02-10T11:34:15.903Z [INFO] server-cf86f427-6a60-2994-9d2a-f75de2e456ad: aborting pipeline replication: peer="{Voter server-2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47 2cc1a1cc-cc0e-e376-718a-d54a1d9c7d47}" --- PASS: TestRaft_SendSnapshotFollower (0.34s) === RUN TestRaft_SendSnapshotAndLogsFollower testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.905Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-180b40c2-ef69-a3ce-ffbb-997856cc17cc Address:180b40c2-ef69-a3ce-ffbb-997856cc17cc} {Suffrage:Voter ID:server-a80fefcc-48e7-f68a-31a7-2757e3746482 Address:a80fefcc-48e7-f68a-31a7-2757e3746482} {Suffrage:Voter ID:server-030319e1-fd9e-1785-6008-7d4008060bfe Address:030319e1-fd9e-1785-6008-7d4008060bfe}]" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.905Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-180b40c2-ef69-a3ce-ffbb-997856cc17cc Address:180b40c2-ef69-a3ce-ffbb-997856cc17cc} {Suffrage:Voter ID:server-a80fefcc-48e7-f68a-31a7-2757e3746482 Address:a80fefcc-48e7-f68a-31a7-2757e3746482} {Suffrage:Voter ID:server-030319e1-fd9e-1785-6008-7d4008060bfe Address:030319e1-fd9e-1785-6008-7d4008060bfe}]" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.905Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: entering follower state: follower="Node at 180b40c2-ef69-a3ce-ffbb-997856cc17cc [Follower]" leader= testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.905Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: entering follower state: follower="Node at 180b40c2-ef69-a3ce-ffbb-997856cc17cc [Follower]" leader= testLoggerAdapter verbose: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:15.905Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-180b40c2-ef69-a3ce-ffbb-997856cc17cc Address:180b40c2-ef69-a3ce-ffbb-997856cc17cc} {Suffrage:Voter ID:server-a80fefcc-48e7-f68a-31a7-2757e3746482 Address:a80fefcc-48e7-f68a-31a7-2757e3746482} {Suffrage:Voter ID:server-030319e1-fd9e-1785-6008-7d4008060bfe Address:030319e1-fd9e-1785-6008-7d4008060bfe}]" testing.go:149: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:15.905Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-180b40c2-ef69-a3ce-ffbb-997856cc17cc Address:180b40c2-ef69-a3ce-ffbb-997856cc17cc} {Suffrage:Voter ID:server-a80fefcc-48e7-f68a-31a7-2757e3746482 Address:a80fefcc-48e7-f68a-31a7-2757e3746482} {Suffrage:Voter ID:server-030319e1-fd9e-1785-6008-7d4008060bfe Address:030319e1-fd9e-1785-6008-7d4008060bfe}]" testLoggerAdapter verbose: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:15.905Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: entering follower state: follower="Node at a80fefcc-48e7-f68a-31a7-2757e3746482 [Follower]" leader= testLoggerAdapter verbose: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:15.905Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-180b40c2-ef69-a3ce-ffbb-997856cc17cc Address:180b40c2-ef69-a3ce-ffbb-997856cc17cc} {Suffrage:Voter ID:server-a80fefcc-48e7-f68a-31a7-2757e3746482 Address:a80fefcc-48e7-f68a-31a7-2757e3746482} {Suffrage:Voter ID:server-030319e1-fd9e-1785-6008-7d4008060bfe Address:030319e1-fd9e-1785-6008-7d4008060bfe}]" testing.go:149: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:15.905Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: entering follower state: follower="Node at a80fefcc-48e7-f68a-31a7-2757e3746482 [Follower]" leader= testing.go:149: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:15.905Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-180b40c2-ef69-a3ce-ffbb-997856cc17cc Address:180b40c2-ef69-a3ce-ffbb-997856cc17cc} {Suffrage:Voter ID:server-a80fefcc-48e7-f68a-31a7-2757e3746482 Address:a80fefcc-48e7-f68a-31a7-2757e3746482} {Suffrage:Voter ID:server-030319e1-fd9e-1785-6008-7d4008060bfe Address:030319e1-fd9e-1785-6008-7d4008060bfe}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:15.906Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:15.906Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:15.906Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: entering follower state: follower="Node at 030319e1-fd9e-1785-6008-7d4008060bfe [Follower]" leader= testing.go:149: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:15.906Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: entering follower state: follower="Node at 030319e1-fd9e-1785-6008-7d4008060bfe [Follower]" leader= testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [WARN] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [WARN] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: entering candidate state: node="Node at 180b40c2-ef69-a3ce-ffbb-997856cc17cc [Candidate]" term=2 testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: entering candidate state: node="Node at 180b40c2-ef69-a3ce-ffbb-997856cc17cc [Candidate]" term=2 testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: election won: tally=2 testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: election won: tally=2 testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: entering leader state: leader="Node at 180b40c2-ef69-a3ce-ffbb-997856cc17cc [Leader]" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: entering leader state: leader="Node at 180b40c2-ef69-a3ce-ffbb-997856cc17cc [Leader]" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: added peer, starting replication: peer=server-a80fefcc-48e7-f68a-31a7-2757e3746482 testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: added peer, starting replication: peer=server-a80fefcc-48e7-f68a-31a7-2757e3746482 testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: added peer, starting replication: peer=server-030319e1-fd9e-1785-6008-7d4008060bfe testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: added peer, starting replication: peer=server-030319e1-fd9e-1785-6008-7d4008060bfe testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: pipelining replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: pipelining replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: pipelining replication: peer="{Voter server-030319e1-fd9e-1785-6008-7d4008060bfe 030319e1-fd9e-1785-6008-7d4008060bfe}" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:15.957Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: pipelining replication: peer="{Voter server-030319e1-fd9e-1785-6008-7d4008060bfe 030319e1-fd9e-1785-6008-7d4008060bfe}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.062Z [INFO] stable state for Follower reached at 2025-02-10 11:34:15.957408874 +0000 UTC m=+12.326313201 (2 nodes), 51.349421ms from start of poll, 52.032449ms from cluster start. Timeout at 2025-02-10 11:34:16.06248323 +0000 UTC m=+12.431387526, 105.074325ms after stability testing.go:149: cluster: 2025-02-10T11:34:16.062Z [INFO] stable state for Follower reached at 2025-02-10 11:34:15.957408874 +0000 UTC m=+12.326313201 (2 nodes), 51.349421ms from start of poll, 52.032449ms from cluster start. Timeout at 2025-02-10 11:34:16.06248323 +0000 UTC m=+12.431387526, 105.074325ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.062Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:16.062Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.167Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.062560361 +0000 UTC m=+12.431464670 (1 nodes), 537ns from start of poll, 157.183918ms from cluster start. Timeout at 2025-02-10 11:34:16.167663939 +0000 UTC m=+12.536568292, 105.103622ms after stability testing.go:149: cluster: 2025-02-10T11:34:16.167Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.062560361 +0000 UTC m=+12.431464670 (1 nodes), 537ns from start of poll, 157.183918ms from cluster start. Timeout at 2025-02-10 11:34:16.167663939 +0000 UTC m=+12.536568292, 105.103622ms after stability testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.167Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to pipeline appendEntries: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" error="append pipeline closed" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.167Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to pipeline appendEntries: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" error="append pipeline closed" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: aborting pipeline replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: aborting pipeline replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to appendEntries to: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" error="failed to connect to peer: a80fefcc-48e7-f68a-31a7-2757e3746482" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to appendEntries to: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" error="failed to connect to peer: a80fefcc-48e7-f68a-31a7-2757e3746482" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to heartbeat to: peer=a80fefcc-48e7-f68a-31a7-2757e3746482 error="failed to connect to peer: a80fefcc-48e7-f68a-31a7-2757e3746482" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to heartbeat to: peer=a80fefcc-48e7-f68a-31a7-2757e3746482 error="failed to connect to peer: a80fefcc-48e7-f68a-31a7-2757e3746482" raft_test.go:1362: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: starting snapshot up to: index=102 testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.168Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: starting snapshot up to: index=102 testing.go:153: 2025-02-10T11:34:16.168Z [INFO] creating new snapshot: path=/tmp/raft508936117/snapshots/2-102-1739187256168.tmp testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.172Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: compacting logs: from=1 to=92 testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.172Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.172Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: snapshot complete up to: index=102 testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.172Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: snapshot complete up to: index=102 testLoggerAdapter verbose: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:16.172Z [ERROR] server-a80fefcc-48e7-f68a-31a7-2757e3746482: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:16.172Z [ERROR] server-a80fefcc-48e7-f68a-31a7-2757e3746482: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:16.172Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: starting snapshot up to: index=99 testing.go:149: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:16.172Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: starting snapshot up to: index=99 testing.go:153: 2025-02-10T11:34:16.172Z [INFO] creating new snapshot: path=/tmp/raft083431353/snapshots/2-99-1739187256172.tmp testLoggerAdapter verbose: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:16.175Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: compacting logs: from=1 to=92 testing.go:149: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:16.175Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:16.175Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: snapshot complete up to: index=99 testing.go:149: server-030319e1-fd9e-1785-6008-7d4008060bfe: 2025-02-10T11:34:16.175Z [INFO] server-030319e1-fd9e-1785-6008-7d4008060bfe: snapshot complete up to: index=99 raft_test.go:1383: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.178Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to get log: index=2 error="log not found" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.178Z [ERROR] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: failed to get log: index=2 error="log not found" testing.go:153: 2025-02-10T11:34:16.178Z [INFO] creating new snapshot: path=/tmp/raft767681455/snapshots/2-102-1739187256178.tmp testLoggerAdapter verbose: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:16.181Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: copied to local snapshot: bytes=693 testing.go:149: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:16.181Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: copied to local snapshot: bytes=693 testLoggerAdapter verbose: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:16.181Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: Installed remote snapshot testing.go:149: server-a80fefcc-48e7-f68a-31a7-2757e3746482: 2025-02-10T11:34:16.181Z [INFO] server-a80fefcc-48e7-f68a-31a7-2757e3746482: Installed remote snapshot testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.182Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: pipelining replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.182Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: pipelining replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.187Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: aborting pipeline replication: peer="{Voter server-030319e1-fd9e-1785-6008-7d4008060bfe 030319e1-fd9e-1785-6008-7d4008060bfe}" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.187Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: aborting pipeline replication: peer="{Voter server-030319e1-fd9e-1785-6008-7d4008060bfe 030319e1-fd9e-1785-6008-7d4008060bfe}" testLoggerAdapter verbose: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.187Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: aborting pipeline replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" testing.go:149: server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: 2025-02-10T11:34:16.187Z [INFO] server-180b40c2-ef69-a3ce-ffbb-997856cc17cc: aborting pipeline replication: peer="{Voter server-a80fefcc-48e7-f68a-31a7-2757e3746482 a80fefcc-48e7-f68a-31a7-2757e3746482}" --- PASS: TestRaft_SendSnapshotAndLogsFollower (0.28s) === RUN TestRaft_ReJoinFollower testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.188Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50} {Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.188Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50} {Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.188Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering follower state: follower="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Follower]" leader= testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.188Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering follower state: follower="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Follower]" leader= testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.188Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50} {Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.188Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50} {Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.188Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering follower state: follower="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Follower]" leader= testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.188Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering follower state: follower="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Follower]" leader= testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.188Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50} {Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.188Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50} {Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.188Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:16.188Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.188Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering follower state: follower="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Follower]" leader= testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.188Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering follower state: follower="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Follower]" leader= testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.274Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.274Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.274Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering candidate state: node="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Candidate]" term=2 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.274Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering candidate state: node="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Candidate]" term=2 testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.274Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.274Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.274Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering candidate state: node="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Candidate]" term=2 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: election won: tally=2 testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.274Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering candidate state: node="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Candidate]" term=2 testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.275Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: duplicate requestVote for same term: term=2 testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.275Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: duplicate requestVote for same term: term=2 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: election won: tally=2 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering leader state: leader="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Leader]" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering leader state: leader="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Leader]" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.275Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: duplicate requestVote for same term: term=2 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.275Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: duplicate requestVote for same term: term=2 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.275Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering follower state: follower="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Follower]" leader=7554165b-63d8-d697-3e13-234c07743bdf testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.275Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering follower state: follower="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Follower]" leader=7554165b-63d8-d697-3e13-234c07743bdf testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: rejecting vote request since we have a leader: from=1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 leader=7554165b-63d8-d697-3e13-234c07743bdf testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.275Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: rejecting vote request since we have a leader: from=1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 leader=7554165b-63d8-d697-3e13-234c07743bdf testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.380Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.275226613 +0000 UTC m=+12.644130916 (1 nodes), 86.38433ms from start of poll, 86.932269ms from cluster start. Timeout at 2025-02-10 11:34:16.380323459 +0000 UTC m=+12.749227755, 105.096839ms after stability testing.go:149: cluster: 2025-02-10T11:34:16.380Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.275226613 +0000 UTC m=+12.644130916 (1 nodes), 86.38433ms from start of poll, 86.932269ms from cluster start. Timeout at 2025-02-10 11:34:16.380323459 +0000 UTC m=+12.749227755, 105.096839ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.385Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:16.385Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.490Z [INFO] stable state for Follower reached at 2025-02-10 11:34:16.38542618 +0000 UTC m=+12.754330483 (2 nodes), 609ns from start of poll, 197.131836ms from cluster start. Timeout at 2025-02-10 11:34:16.490498135 +0000 UTC m=+12.859402432, 105.071949ms after stability testing.go:149: cluster: 2025-02-10T11:34:16.490Z [INFO] stable state for Follower reached at 2025-02-10 11:34:16.38542618 +0000 UTC m=+12.754330483 (2 nodes), 609ns from start of poll, 197.131836ms from cluster start. Timeout at 2025-02-10 11:34:16.490498135 +0000 UTC m=+12.859402432, 105.071949ms after stability testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.490Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: updating configuration: command=RemoveServer server-id=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 server-addr= servers="[{Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.490Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: updating configuration: command=RemoveServer server-id=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 server-addr= servers="[{Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}]" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.490Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: removed peer, stopping replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 last-index=3 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.490Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: removed peer, stopping replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 last-index=3 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.490Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.490Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.545Z [WARN] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: heartbeat timeout reached, starting election: last-leader=7554165b-63d8-d697-3e13-234c07743bdf testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.545Z [WARN] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: heartbeat timeout reached, starting election: last-leader=7554165b-63d8-d697-3e13-234c07743bdf testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.545Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering candidate state: node="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Candidate]" term=3 testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.545Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering candidate state: node="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Candidate]" term=3 testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.545Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: rejecting vote request since we have a leader: from=d8405245-e81a-0c2e-0b48-5f081bfa1a50 leader=7554165b-63d8-d697-3e13-234c07743bdf testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.545Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: rejecting vote request since we have a leader: from=d8405245-e81a-0c2e-0b48-5f081bfa1a50 leader=7554165b-63d8-d697-3e13-234c07743bdf testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.545Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: rejecting vote request since we have a leader: from=d8405245-e81a-0c2e-0b48-5f081bfa1a50 leader=7554165b-63d8-d697-3e13-234c07743bdf testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.545Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: rejecting vote request since we have a leader: from=d8405245-e81a-0c2e-0b48-5f081bfa1a50 leader=7554165b-63d8-d697-3e13-234c07743bdf testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: updating configuration: command=AddStaging server-id=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 server-addr=d8405245-e81a-0c2e-0b48-5f081bfa1a50 servers="[{Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26} {Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50}]" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: updating configuration: command=AddStaging server-id=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 server-addr=d8405245-e81a-0c2e-0b48-5f081bfa1a50 servers="[{Suffrage:Voter ID:server-7554165b-63d8-d697-3e13-234c07743bdf Address:7554165b-63d8-d697-3e13-234c07743bdf} {Suffrage:Voter ID:server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 Address:1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26} {Suffrage:Voter ID:server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 Address:d8405245-e81a-0c2e-0b48-5f081bfa1a50}]" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [ERROR] server-7554165b-63d8-d697-3e13-234c07743bdf: peer has newer term, stopping replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [ERROR] server-7554165b-63d8-d697-3e13-234c07743bdf: peer has newer term, stopping replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering follower state: follower="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Follower]" leader= testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering follower state: follower="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Follower]" leader= testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.595Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:16.595Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.595Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.643Z [WARN] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: Election timeout reached, restarting election testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.643Z [WARN] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: Election timeout reached, restarting election testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.643Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering candidate state: node="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Candidate]" term=4 testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.643Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering candidate state: node="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Candidate]" term=4 testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.643Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: rejecting vote request since we have a leader: from=d8405245-e81a-0c2e-0b48-5f081bfa1a50 leader=7554165b-63d8-d697-3e13-234c07743bdf testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.643Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: rejecting vote request since our last index is greater: candidate=d8405245-e81a-0c2e-0b48-5f081bfa1a50 last-index=4 last-candidate-index=3 testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.643Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: rejecting vote request since we have a leader: from=d8405245-e81a-0c2e-0b48-5f081bfa1a50 leader=7554165b-63d8-d697-3e13-234c07743bdf testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.643Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: rejecting vote request since our last index is greater: candidate=d8405245-e81a-0c2e-0b48-5f081bfa1a50 last-index=4 last-candidate-index=3 testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.649Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: heartbeat timeout reached, starting election: last-leader=7554165b-63d8-d697-3e13-234c07743bdf testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.649Z [WARN] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: heartbeat timeout reached, starting election: last-leader=7554165b-63d8-d697-3e13-234c07743bdf testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.649Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering candidate state: node="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Candidate]" term=3 testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.649Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering candidate state: node="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Candidate]" term=3 testLoggerAdapter verbose: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.649Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering follower state: follower="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Follower]" leader= testing.go:149: server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: 2025-02-10T11:34:16.649Z [INFO] server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26: entering follower state: follower="Node at 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 [Follower]" leader= testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering candidate state: node="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Candidate]" term=5 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering candidate state: node="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Candidate]" term=5 testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.659Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering follower state: follower="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Follower]" leader= testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.659Z [INFO] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: entering follower state: follower="Node at d8405245-e81a-0c2e-0b48-5f081bfa1a50 [Follower]" leader= testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: election won: tally=2 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: election won: tally=2 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering leader state: leader="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Leader]" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: entering leader state: leader="Node at 7554165b-63d8-d697-3e13-234c07743bdf [Leader]" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: added peer, starting replication: peer=server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 testLoggerAdapter verbose: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.659Z [WARN] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: failed to get previous log: previous-index=4 last-index=3 error="log not found" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" testing.go:149: server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: 2025-02-10T11:34:16.659Z [WARN] server-d8405245-e81a-0c2e-0b48-5f081bfa1a50: failed to get previous log: previous-index=4 last-index=3 error="log not found" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: appendEntries rejected, sending older logs: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" next=4 testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [WARN] server-7554165b-63d8-d697-3e13-234c07743bdf: appendEntries rejected, sending older logs: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" next=4 testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.659Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: pipelining replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.764Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.659408581 +0000 UTC m=+13.028313000 (1 nodes), 63.468318ms from start of poll, 471.114353ms from cluster start. Timeout at 2025-02-10 11:34:16.76446748 +0000 UTC m=+13.133371833, 105.058833ms after stability testing.go:149: cluster: 2025-02-10T11:34:16.764Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.659408581 +0000 UTC m=+13.028313000 (1 nodes), 63.468318ms from start of poll, 471.114353ms from cluster start. Timeout at 2025-02-10 11:34:16.76446748 +0000 UTC m=+13.133371833, 105.058833ms after stability testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.764Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.764Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-d8405245-e81a-0c2e-0b48-5f081bfa1a50 d8405245-e81a-0c2e-0b48-5f081bfa1a50}" testLoggerAdapter verbose: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.764Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" testing.go:149: server-7554165b-63d8-d697-3e13-234c07743bdf: 2025-02-10T11:34:16.764Z [INFO] server-7554165b-63d8-d697-3e13-234c07743bdf: aborting pipeline replication: peer="{Voter server-1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26 1fdd7c5e-b0eb-47ab-6d87-3cfcffbaed26}" --- PASS: TestRaft_ReJoinFollower (0.58s) === RUN TestRaft_LeaderLeaseExpire testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:16.765Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 Address:d1441b8d-1ba0-5689-291b-e3dfd1fc3f53} {Suffrage:Voter ID:server-ecf585ab-2037-e4fe-34a0-12acb18a618e Address:ecf585ab-2037-e4fe-34a0-12acb18a618e}]" testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:16.765Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 Address:d1441b8d-1ba0-5689-291b-e3dfd1fc3f53} {Suffrage:Voter ID:server-ecf585ab-2037-e4fe-34a0-12acb18a618e Address:ecf585ab-2037-e4fe-34a0-12acb18a618e}]" testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:16.765Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering follower state: follower="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Follower]" leader= testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:16.765Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering follower state: follower="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Follower]" leader= testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.765Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 Address:d1441b8d-1ba0-5689-291b-e3dfd1fc3f53} {Suffrage:Voter ID:server-ecf585ab-2037-e4fe-34a0-12acb18a618e Address:ecf585ab-2037-e4fe-34a0-12acb18a618e}]" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.765Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 Address:d1441b8d-1ba0-5689-291b-e3dfd1fc3f53} {Suffrage:Voter ID:server-ecf585ab-2037-e4fe-34a0-12acb18a618e Address:ecf585ab-2037-e4fe-34a0-12acb18a618e}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.766Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:16.766Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.766Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering follower state: follower="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Follower]" leader= testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.766Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering follower state: follower="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Follower]" leader= testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=2 testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=2 testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: election won: tally=2 testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: election won: tally=2 testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering leader state: leader="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Leader]" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering leader state: leader="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Leader]" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: added peer, starting replication: peer=server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: added peer, starting replication: peer=server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: pipelining replication: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:16.826Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: pipelining replication: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.931Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.826303756 +0000 UTC m=+13.195208079 (1 nodes), 60.228856ms from start of poll, 60.579536ms from cluster start. Timeout at 2025-02-10 11:34:16.93140396 +0000 UTC m=+13.300308313, 105.100234ms after stability testing.go:149: cluster: 2025-02-10T11:34:16.931Z [INFO] stable state for Leader reached at 2025-02-10 11:34:16.826303756 +0000 UTC m=+13.195208079 (1 nodes), 60.228856ms from start of poll, 60.579536ms from cluster start. Timeout at 2025-02-10 11:34:16.93140396 +0000 UTC m=+13.300308313, 105.100234ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:16.936Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:16.936Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.041Z [INFO] stable state for Follower reached at 2025-02-10 11:34:16.936643466 +0000 UTC m=+13.305547766 (1 nodes), 406ns from start of poll, 170.919223ms from cluster start. Timeout at 2025-02-10 11:34:17.041718669 +0000 UTC m=+13.410622973, 105.075207ms after stability testing.go:149: cluster: 2025-02-10T11:34:17.041Z [INFO] stable state for Follower reached at 2025-02-10 11:34:16.936643466 +0000 UTC m=+13.305547766 (1 nodes), 406ns from start of poll, 170.919223ms from cluster start. Timeout at 2025-02-10 11:34:17.041718669 +0000 UTC m=+13.410622973, 105.075207ms after stability raft_test.go:1492: [INFO] Disconnecting Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Follower] testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.044Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to heartbeat to: peer=d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.044Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to heartbeat to: peer=d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.047Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to pipeline appendEntries: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="append pipeline closed" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.047Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to pipeline appendEntries: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="append pipeline closed" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.047Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: aborting pipeline replication: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.047Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: aborting pipeline replication: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.054Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to appendEntries to: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.054Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to appendEntries to: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.061Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to heartbeat to: peer=d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.061Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to heartbeat to: peer=d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.073Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to appendEntries to: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.073Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to appendEntries to: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.077Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to heartbeat to: peer=d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.077Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to heartbeat to: peer=d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.089Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to contact: server-id=server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 time=50.069392ms testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.089Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to contact: server-id=server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 time=50.069392ms testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.089Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to contact quorum of nodes, stepping down testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.089Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.089Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering follower state: follower="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Follower]" leader= testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.089Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering follower state: follower="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Follower]" leader= testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.089Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:17.089Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.091Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to appendEntries to: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.091Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to appendEntries to: peer="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.102Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: heartbeat timeout reached, starting election: last-leader=ecf585ab-2037-e4fe-34a0-12acb18a618e testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.102Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: heartbeat timeout reached, starting election: last-leader=ecf585ab-2037-e4fe-34a0-12acb18a618e testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.102Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=3 testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.102Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=3 testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.102Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.102Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.168Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.168Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.168Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=4 testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.168Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=4 testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.168Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.168Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.185Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.185Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.186Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=3 testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.186Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=3 testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.186Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.186Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.219Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.219Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.219Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=5 testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.219Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=5 testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.219Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.219Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.241Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: Election timeout reached, restarting election testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.241Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: Election timeout reached, restarting election testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.241Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=4 testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.241Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=4 testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.241Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.241Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.291Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.186134319 +0000 UTC m=+13.555038623 (0 nodes), 96.515538ms from start of poll, 420.41008ms from cluster start. Timeout at 2025-02-10 11:34:17.291234745 +0000 UTC m=+13.660139041, 105.100418ms after stability testing.go:149: cluster: 2025-02-10T11:34:17.291Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.186134319 +0000 UTC m=+13.555038623 (0 nodes), 96.515538ms from start of poll, 420.41008ms from cluster start. Timeout at 2025-02-10 11:34:17.291234745 +0000 UTC m=+13.660139041, 105.100418ms after stability testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.309Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.309Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.309Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=6 testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.309Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=6 testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.309Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.309Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.319Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: Election timeout reached, restarting election testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.319Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: Election timeout reached, restarting election testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.319Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=5 testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.319Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=5 testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.319Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.319Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.362Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.362Z [WARN] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: Election timeout reached, restarting election testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.362Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=7 testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.362Z [INFO] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: entering candidate state: node="Node at d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 [Candidate]" term=7 testLoggerAdapter verbose: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.362Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testing.go:149: server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: 2025-02-10T11:34:17.362Z [ERROR] server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53: failed to make requestVote RPC: target="{Voter server-ecf585ab-2037-e4fe-34a0-12acb18a618e ecf585ab-2037-e4fe-34a0-12acb18a618e}" error="failed to connect to peer: ecf585ab-2037-e4fe-34a0-12acb18a618e" testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.375Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: Election timeout reached, restarting election testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.375Z [WARN] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: Election timeout reached, restarting election testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.375Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=6 testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.375Z [INFO] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: entering candidate state: node="Node at ecf585ab-2037-e4fe-34a0-12acb18a618e [Candidate]" term=6 testLoggerAdapter verbose: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.375Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" testing.go:149: server-ecf585ab-2037-e4fe-34a0-12acb18a618e: 2025-02-10T11:34:17.375Z [ERROR] server-ecf585ab-2037-e4fe-34a0-12acb18a618e: failed to make requestVote RPC: target="{Voter server-d1441b8d-1ba0-5689-291b-e3dfd1fc3f53 d1441b8d-1ba0-5689-291b-e3dfd1fc3f53}" error="failed to connect to peer: d1441b8d-1ba0-5689-291b-e3dfd1fc3f53" --- PASS: TestRaft_LeaderLeaseExpire (0.63s) === RUN TestRaft_Barrier testLoggerAdapter verbose: server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: 2025-02-10T11:34:17.397Z [INFO] server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4ff9cfaa-097a-6628-a236-b6bde823ae5d Address:4ff9cfaa-097a-6628-a236-b6bde823ae5d} {Suffrage:Voter ID:server-b8eecf14-3b74-dcb4-d192-467613a45d7a Address:b8eecf14-3b74-dcb4-d192-467613a45d7a} {Suffrage:Voter ID:server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 Address:486d0fd4-852d-d6fb-50fb-1b52d4fb0036}]" testing.go:149: server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: 2025-02-10T11:34:17.397Z [INFO] server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4ff9cfaa-097a-6628-a236-b6bde823ae5d Address:4ff9cfaa-097a-6628-a236-b6bde823ae5d} {Suffrage:Voter ID:server-b8eecf14-3b74-dcb4-d192-467613a45d7a Address:b8eecf14-3b74-dcb4-d192-467613a45d7a} {Suffrage:Voter ID:server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 Address:486d0fd4-852d-d6fb-50fb-1b52d4fb0036}]" testLoggerAdapter verbose: server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: 2025-02-10T11:34:17.397Z [INFO] server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: entering follower state: follower="Node at 4ff9cfaa-097a-6628-a236-b6bde823ae5d [Follower]" leader= testing.go:149: server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: 2025-02-10T11:34:17.397Z [INFO] server-4ff9cfaa-097a-6628-a236-b6bde823ae5d: entering follower state: follower="Node at 4ff9cfaa-097a-6628-a236-b6bde823ae5d [Follower]" leader= testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.397Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4ff9cfaa-097a-6628-a236-b6bde823ae5d Address:4ff9cfaa-097a-6628-a236-b6bde823ae5d} {Suffrage:Voter ID:server-b8eecf14-3b74-dcb4-d192-467613a45d7a Address:b8eecf14-3b74-dcb4-d192-467613a45d7a} {Suffrage:Voter ID:server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 Address:486d0fd4-852d-d6fb-50fb-1b52d4fb0036}]" testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.397Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4ff9cfaa-097a-6628-a236-b6bde823ae5d Address:4ff9cfaa-097a-6628-a236-b6bde823ae5d} {Suffrage:Voter ID:server-b8eecf14-3b74-dcb4-d192-467613a45d7a Address:b8eecf14-3b74-dcb4-d192-467613a45d7a} {Suffrage:Voter ID:server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 Address:486d0fd4-852d-d6fb-50fb-1b52d4fb0036}]" testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.397Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: entering follower state: follower="Node at b8eecf14-3b74-dcb4-d192-467613a45d7a [Follower]" leader= testLoggerAdapter verbose: server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: 2025-02-10T11:34:17.397Z [INFO] server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4ff9cfaa-097a-6628-a236-b6bde823ae5d Address:4ff9cfaa-097a-6628-a236-b6bde823ae5d} {Suffrage:Voter ID:server-b8eecf14-3b74-dcb4-d192-467613a45d7a Address:b8eecf14-3b74-dcb4-d192-467613a45d7a} {Suffrage:Voter ID:server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 Address:486d0fd4-852d-d6fb-50fb-1b52d4fb0036}]" testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.397Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: entering follower state: follower="Node at b8eecf14-3b74-dcb4-d192-467613a45d7a [Follower]" leader= testing.go:149: server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: 2025-02-10T11:34:17.397Z [INFO] server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4ff9cfaa-097a-6628-a236-b6bde823ae5d Address:4ff9cfaa-097a-6628-a236-b6bde823ae5d} {Suffrage:Voter ID:server-b8eecf14-3b74-dcb4-d192-467613a45d7a Address:b8eecf14-3b74-dcb4-d192-467613a45d7a} {Suffrage:Voter ID:server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 Address:486d0fd4-852d-d6fb-50fb-1b52d4fb0036}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.397Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:17.397Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: 2025-02-10T11:34:17.397Z [INFO] server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: entering follower state: follower="Node at 486d0fd4-852d-d6fb-50fb-1b52d4fb0036 [Follower]" leader= testing.go:149: server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: 2025-02-10T11:34:17.397Z [INFO] server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036: entering follower state: follower="Node at 486d0fd4-852d-d6fb-50fb-1b52d4fb0036 [Follower]" leader= testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [WARN] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [WARN] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: entering candidate state: node="Node at b8eecf14-3b74-dcb4-d192-467613a45d7a [Candidate]" term=2 testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: entering candidate state: node="Node at b8eecf14-3b74-dcb4-d192-467613a45d7a [Candidate]" term=2 testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: election won: tally=2 testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: election won: tally=2 testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: entering leader state: leader="Node at b8eecf14-3b74-dcb4-d192-467613a45d7a [Leader]" testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: entering leader state: leader="Node at b8eecf14-3b74-dcb4-d192-467613a45d7a [Leader]" testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: added peer, starting replication: peer=server-4ff9cfaa-097a-6628-a236-b6bde823ae5d testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: added peer, starting replication: peer=server-4ff9cfaa-097a-6628-a236-b6bde823ae5d testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: added peer, starting replication: peer=server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: added peer, starting replication: peer=server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: pipelining replication: peer="{Voter server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 486d0fd4-852d-d6fb-50fb-1b52d4fb0036}" testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: pipelining replication: peer="{Voter server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 486d0fd4-852d-d6fb-50fb-1b52d4fb0036}" testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: pipelining replication: peer="{Voter server-4ff9cfaa-097a-6628-a236-b6bde823ae5d 4ff9cfaa-097a-6628-a236-b6bde823ae5d}" testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.452Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: pipelining replication: peer="{Voter server-4ff9cfaa-097a-6628-a236-b6bde823ae5d 4ff9cfaa-097a-6628-a236-b6bde823ae5d}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.557Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.452459672 +0000 UTC m=+13.821364052 (1 nodes), 54.846404ms from start of poll, 55.236084ms from cluster start. Timeout at 2025-02-10 11:34:17.557586813 +0000 UTC m=+13.926491223, 105.127171ms after stability testing.go:149: cluster: 2025-02-10T11:34:17.557Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.452459672 +0000 UTC m=+13.821364052 (1 nodes), 54.846404ms from start of poll, 55.236084ms from cluster start. Timeout at 2025-02-10 11:34:17.557586813 +0000 UTC m=+13.926491223, 105.127171ms after stability testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.570Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: aborting pipeline replication: peer="{Voter server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 486d0fd4-852d-d6fb-50fb-1b52d4fb0036}" testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.570Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: aborting pipeline replication: peer="{Voter server-486d0fd4-852d-d6fb-50fb-1b52d4fb0036 486d0fd4-852d-d6fb-50fb-1b52d4fb0036}" testLoggerAdapter verbose: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.570Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: aborting pipeline replication: peer="{Voter server-4ff9cfaa-097a-6628-a236-b6bde823ae5d 4ff9cfaa-097a-6628-a236-b6bde823ae5d}" testing.go:149: server-b8eecf14-3b74-dcb4-d192-467613a45d7a: 2025-02-10T11:34:17.570Z [INFO] server-b8eecf14-3b74-dcb4-d192-467613a45d7a: aborting pipeline replication: peer="{Voter server-4ff9cfaa-097a-6628-a236-b6bde823ae5d 4ff9cfaa-097a-6628-a236-b6bde823ae5d}" --- PASS: TestRaft_Barrier (0.17s) === RUN TestRaft_VerifyLeader testLoggerAdapter verbose: server-637376c3-5987-6205-0524-1df5e1074f91: 2025-02-10T11:34:17.571Z [INFO] server-637376c3-5987-6205-0524-1df5e1074f91: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-637376c3-5987-6205-0524-1df5e1074f91 Address:637376c3-5987-6205-0524-1df5e1074f91} {Suffrage:Voter ID:server-07c83c14-1bc4-c7cc-280d-08154b7a3beb Address:07c83c14-1bc4-c7cc-280d-08154b7a3beb} {Suffrage:Voter ID:server-449d221e-17ce-46a1-5002-49732756b151 Address:449d221e-17ce-46a1-5002-49732756b151}]" testing.go:149: server-637376c3-5987-6205-0524-1df5e1074f91: 2025-02-10T11:34:17.571Z [INFO] server-637376c3-5987-6205-0524-1df5e1074f91: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-637376c3-5987-6205-0524-1df5e1074f91 Address:637376c3-5987-6205-0524-1df5e1074f91} {Suffrage:Voter ID:server-07c83c14-1bc4-c7cc-280d-08154b7a3beb Address:07c83c14-1bc4-c7cc-280d-08154b7a3beb} {Suffrage:Voter ID:server-449d221e-17ce-46a1-5002-49732756b151 Address:449d221e-17ce-46a1-5002-49732756b151}]" testLoggerAdapter verbose: server-637376c3-5987-6205-0524-1df5e1074f91: 2025-02-10T11:34:17.571Z [INFO] server-637376c3-5987-6205-0524-1df5e1074f91: entering follower state: follower="Node at 637376c3-5987-6205-0524-1df5e1074f91 [Follower]" leader= testing.go:149: server-637376c3-5987-6205-0524-1df5e1074f91: 2025-02-10T11:34:17.571Z [INFO] server-637376c3-5987-6205-0524-1df5e1074f91: entering follower state: follower="Node at 637376c3-5987-6205-0524-1df5e1074f91 [Follower]" leader= testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.571Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-637376c3-5987-6205-0524-1df5e1074f91 Address:637376c3-5987-6205-0524-1df5e1074f91} {Suffrage:Voter ID:server-07c83c14-1bc4-c7cc-280d-08154b7a3beb Address:07c83c14-1bc4-c7cc-280d-08154b7a3beb} {Suffrage:Voter ID:server-449d221e-17ce-46a1-5002-49732756b151 Address:449d221e-17ce-46a1-5002-49732756b151}]" testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.571Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-637376c3-5987-6205-0524-1df5e1074f91 Address:637376c3-5987-6205-0524-1df5e1074f91} {Suffrage:Voter ID:server-07c83c14-1bc4-c7cc-280d-08154b7a3beb Address:07c83c14-1bc4-c7cc-280d-08154b7a3beb} {Suffrage:Voter ID:server-449d221e-17ce-46a1-5002-49732756b151 Address:449d221e-17ce-46a1-5002-49732756b151}]" testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.571Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: entering follower state: follower="Node at 07c83c14-1bc4-c7cc-280d-08154b7a3beb [Follower]" leader= testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.571Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: entering follower state: follower="Node at 07c83c14-1bc4-c7cc-280d-08154b7a3beb [Follower]" leader= testLoggerAdapter verbose: server-449d221e-17ce-46a1-5002-49732756b151: 2025-02-10T11:34:17.571Z [INFO] server-449d221e-17ce-46a1-5002-49732756b151: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-637376c3-5987-6205-0524-1df5e1074f91 Address:637376c3-5987-6205-0524-1df5e1074f91} {Suffrage:Voter ID:server-07c83c14-1bc4-c7cc-280d-08154b7a3beb Address:07c83c14-1bc4-c7cc-280d-08154b7a3beb} {Suffrage:Voter ID:server-449d221e-17ce-46a1-5002-49732756b151 Address:449d221e-17ce-46a1-5002-49732756b151}]" testing.go:149: server-449d221e-17ce-46a1-5002-49732756b151: 2025-02-10T11:34:17.571Z [INFO] server-449d221e-17ce-46a1-5002-49732756b151: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-637376c3-5987-6205-0524-1df5e1074f91 Address:637376c3-5987-6205-0524-1df5e1074f91} {Suffrage:Voter ID:server-07c83c14-1bc4-c7cc-280d-08154b7a3beb Address:07c83c14-1bc4-c7cc-280d-08154b7a3beb} {Suffrage:Voter ID:server-449d221e-17ce-46a1-5002-49732756b151 Address:449d221e-17ce-46a1-5002-49732756b151}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.571Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:17.571Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-449d221e-17ce-46a1-5002-49732756b151: 2025-02-10T11:34:17.571Z [INFO] server-449d221e-17ce-46a1-5002-49732756b151: entering follower state: follower="Node at 449d221e-17ce-46a1-5002-49732756b151 [Follower]" leader= testing.go:149: server-449d221e-17ce-46a1-5002-49732756b151: 2025-02-10T11:34:17.571Z [INFO] server-449d221e-17ce-46a1-5002-49732756b151: entering follower state: follower="Node at 449d221e-17ce-46a1-5002-49732756b151 [Follower]" leader= testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [WARN] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [WARN] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: entering candidate state: node="Node at 07c83c14-1bc4-c7cc-280d-08154b7a3beb [Candidate]" term=2 testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: entering candidate state: node="Node at 07c83c14-1bc4-c7cc-280d-08154b7a3beb [Candidate]" term=2 testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: election won: tally=2 testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: election won: tally=2 testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: entering leader state: leader="Node at 07c83c14-1bc4-c7cc-280d-08154b7a3beb [Leader]" testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: entering leader state: leader="Node at 07c83c14-1bc4-c7cc-280d-08154b7a3beb [Leader]" testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: added peer, starting replication: peer=server-637376c3-5987-6205-0524-1df5e1074f91 testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: added peer, starting replication: peer=server-637376c3-5987-6205-0524-1df5e1074f91 testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: added peer, starting replication: peer=server-449d221e-17ce-46a1-5002-49732756b151 testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: added peer, starting replication: peer=server-449d221e-17ce-46a1-5002-49732756b151 testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: pipelining replication: peer="{Voter server-449d221e-17ce-46a1-5002-49732756b151 449d221e-17ce-46a1-5002-49732756b151}" testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: pipelining replication: peer="{Voter server-449d221e-17ce-46a1-5002-49732756b151 449d221e-17ce-46a1-5002-49732756b151}" testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: pipelining replication: peer="{Voter server-637376c3-5987-6205-0524-1df5e1074f91 637376c3-5987-6205-0524-1df5e1074f91}" testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.623Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: pipelining replication: peer="{Voter server-637376c3-5987-6205-0524-1df5e1074f91 637376c3-5987-6205-0524-1df5e1074f91}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.728Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.623304925 +0000 UTC m=+13.992209225 (1 nodes), 51.907423ms from start of poll, 52.339024ms from cluster start. Timeout at 2025-02-10 11:34:17.728376301 +0000 UTC m=+14.097280597, 105.071372ms after stability testing.go:149: cluster: 2025-02-10T11:34:17.728Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.623304925 +0000 UTC m=+13.992209225 (1 nodes), 51.907423ms from start of poll, 52.339024ms from cluster start. Timeout at 2025-02-10 11:34:17.728376301 +0000 UTC m=+14.097280597, 105.071372ms after stability testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.728Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: aborting pipeline replication: peer="{Voter server-637376c3-5987-6205-0524-1df5e1074f91 637376c3-5987-6205-0524-1df5e1074f91}" testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.728Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: aborting pipeline replication: peer="{Voter server-637376c3-5987-6205-0524-1df5e1074f91 637376c3-5987-6205-0524-1df5e1074f91}" testLoggerAdapter verbose: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.728Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: aborting pipeline replication: peer="{Voter server-449d221e-17ce-46a1-5002-49732756b151 449d221e-17ce-46a1-5002-49732756b151}" testing.go:149: server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: 2025-02-10T11:34:17.728Z [INFO] server-07c83c14-1bc4-c7cc-280d-08154b7a3beb: aborting pipeline replication: peer="{Voter server-449d221e-17ce-46a1-5002-49732756b151 449d221e-17ce-46a1-5002-49732756b151}" --- PASS: TestRaft_VerifyLeader (0.16s) === RUN TestRaft_VerifyLeader_Single testLoggerAdapter verbose: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.729Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5183fb-121f-cc46-6595-e7bd8f536dc0 Address:fb5183fb-121f-cc46-6595-e7bd8f536dc0}]" testing.go:149: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.729Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5183fb-121f-cc46-6595-e7bd8f536dc0 Address:fb5183fb-121f-cc46-6595-e7bd8f536dc0}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.729Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:17.729Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.729Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: entering follower state: follower="Node at fb5183fb-121f-cc46-6595-e7bd8f536dc0 [Follower]" leader= testing.go:149: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.729Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: entering follower state: follower="Node at fb5183fb-121f-cc46-6595-e7bd8f536dc0 [Follower]" leader= testLoggerAdapter verbose: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [WARN] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [WARN] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: entering candidate state: node="Node at fb5183fb-121f-cc46-6595-e7bd8f536dc0 [Candidate]" term=2 testing.go:149: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: entering candidate state: node="Node at fb5183fb-121f-cc46-6595-e7bd8f536dc0 [Candidate]" term=2 testLoggerAdapter verbose: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: election won: tally=1 testing.go:149: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: election won: tally=1 testLoggerAdapter verbose: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: entering leader state: leader="Node at fb5183fb-121f-cc46-6595-e7bd8f536dc0 [Leader]" testing.go:149: server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: 2025-02-10T11:34:17.793Z [INFO] server-fb5183fb-121f-cc46-6595-e7bd8f536dc0: entering leader state: leader="Node at fb5183fb-121f-cc46-6595-e7bd8f536dc0 [Leader]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.898Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.793507339 +0000 UTC m=+14.162411638 (1 nodes), 64.19744ms from start of poll, 64.353711ms from cluster start. Timeout at 2025-02-10 11:34:17.898583979 +0000 UTC m=+14.267488276, 105.076638ms after stability testing.go:149: cluster: 2025-02-10T11:34:17.898Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.793507339 +0000 UTC m=+14.162411638 (1 nodes), 64.19744ms from start of poll, 64.353711ms from cluster start. Timeout at 2025-02-10 11:34:17.898583979 +0000 UTC m=+14.267488276, 105.076638ms after stability --- PASS: TestRaft_VerifyLeader_Single (0.17s) === RUN TestRaft_VerifyLeader_Fail testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.899Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5df9c90-71f4-faab-2144-f384ec8b5b64 Address:d5df9c90-71f4-faab-2144-f384ec8b5b64} {Suffrage:Voter ID:server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 Address:47ce0eb2-eeda-d98d-aac1-2bd025bddc52}]" testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.899Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5df9c90-71f4-faab-2144-f384ec8b5b64 Address:d5df9c90-71f4-faab-2144-f384ec8b5b64} {Suffrage:Voter ID:server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 Address:47ce0eb2-eeda-d98d-aac1-2bd025bddc52}]" testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.899Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering follower state: follower="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Follower]" leader= testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.899Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering follower state: follower="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Follower]" leader= testLoggerAdapter verbose: server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: 2025-02-10T11:34:17.899Z [INFO] server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5df9c90-71f4-faab-2144-f384ec8b5b64 Address:d5df9c90-71f4-faab-2144-f384ec8b5b64} {Suffrage:Voter ID:server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 Address:47ce0eb2-eeda-d98d-aac1-2bd025bddc52}]" testing.go:149: server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: 2025-02-10T11:34:17.899Z [INFO] server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5df9c90-71f4-faab-2144-f384ec8b5b64 Address:d5df9c90-71f4-faab-2144-f384ec8b5b64} {Suffrage:Voter ID:server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 Address:47ce0eb2-eeda-d98d-aac1-2bd025bddc52}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:17.899Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:17.899Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: 2025-02-10T11:34:17.899Z [INFO] server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: entering follower state: follower="Node at 47ce0eb2-eeda-d98d-aac1-2bd025bddc52 [Follower]" leader= testing.go:149: server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: 2025-02-10T11:34:17.899Z [INFO] server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52: entering follower state: follower="Node at 47ce0eb2-eeda-d98d-aac1-2bd025bddc52 [Follower]" leader= testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [WARN] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [WARN] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering candidate state: node="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Candidate]" term=2 testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering candidate state: node="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Candidate]" term=2 testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: election won: tally=2 testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: election won: tally=2 testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering leader state: leader="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Leader]" testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering leader state: leader="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Leader]" testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: added peer, starting replication: peer=server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: added peer, starting replication: peer=server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: pipelining replication: peer="{Voter server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 47ce0eb2-eeda-d98d-aac1-2bd025bddc52}" testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:17.965Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: pipelining replication: peer="{Voter server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 47ce0eb2-eeda-d98d-aac1-2bd025bddc52}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.070Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.96558081 +0000 UTC m=+14.334485112 (1 nodes), 66.10447ms from start of poll, 66.38718ms from cluster start. Timeout at 2025-02-10 11:34:18.070655788 +0000 UTC m=+14.439560106, 105.074994ms after stability testing.go:149: cluster: 2025-02-10T11:34:18.070Z [INFO] stable state for Leader reached at 2025-02-10 11:34:17.96558081 +0000 UTC m=+14.334485112 (1 nodes), 66.10447ms from start of poll, 66.38718ms from cluster start. Timeout at 2025-02-10 11:34:18.070655788 +0000 UTC m=+14.439560106, 105.074994ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.070Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:18.070Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.175Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.070765975 +0000 UTC m=+14.439670279 (1 nodes), 554ns from start of poll, 171.572347ms from cluster start. Timeout at 2025-02-10 11:34:18.175868455 +0000 UTC m=+14.544772804, 105.102525ms after stability testing.go:149: cluster: 2025-02-10T11:34:18.175Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.070765975 +0000 UTC m=+14.439670279 (1 nodes), 554ns from start of poll, 171.572347ms from cluster start. Timeout at 2025-02-10 11:34:18.175868455 +0000 UTC m=+14.544772804, 105.102525ms after stability testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:18.176Z [WARN] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: new leader elected, stepping down testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:18.176Z [WARN] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: new leader elected, stepping down testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:18.176Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering follower state: follower="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Follower]" leader= testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:18.176Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: entering follower state: follower="Node at d5df9c90-71f4-faab-2144-f384ec8b5b64 [Follower]" leader= testLoggerAdapter verbose: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:18.176Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: aborting pipeline replication: peer="{Voter server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 47ce0eb2-eeda-d98d-aac1-2bd025bddc52}" testing.go:149: server-d5df9c90-71f4-faab-2144-f384ec8b5b64: 2025-02-10T11:34:18.176Z [INFO] server-d5df9c90-71f4-faab-2144-f384ec8b5b64: aborting pipeline replication: peer="{Voter server-47ce0eb2-eeda-d98d-aac1-2bd025bddc52 47ce0eb2-eeda-d98d-aac1-2bd025bddc52}" --- PASS: TestRaft_VerifyLeader_Fail (0.28s) === RUN TestRaft_VerifyLeader_PartialConnect testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.176Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f Address:bac5c5bb-1eac-a02d-69eb-ea4fc942051f} {Suffrage:Voter ID:server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd Address:03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd} {Suffrage:Voter ID:server-12ed9356-7797-de21-5e72-dd20b1a0314b Address:12ed9356-7797-de21-5e72-dd20b1a0314b}]" testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.176Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f Address:bac5c5bb-1eac-a02d-69eb-ea4fc942051f} {Suffrage:Voter ID:server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd Address:03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd} {Suffrage:Voter ID:server-12ed9356-7797-de21-5e72-dd20b1a0314b Address:12ed9356-7797-de21-5e72-dd20b1a0314b}]" testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.177Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: entering follower state: follower="Node at bac5c5bb-1eac-a02d-69eb-ea4fc942051f [Follower]" leader= testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.177Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: entering follower state: follower="Node at bac5c5bb-1eac-a02d-69eb-ea4fc942051f [Follower]" leader= testLoggerAdapter verbose: server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: 2025-02-10T11:34:18.177Z [INFO] server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f Address:bac5c5bb-1eac-a02d-69eb-ea4fc942051f} {Suffrage:Voter ID:server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd Address:03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd} {Suffrage:Voter ID:server-12ed9356-7797-de21-5e72-dd20b1a0314b Address:12ed9356-7797-de21-5e72-dd20b1a0314b}]" testing.go:149: server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: 2025-02-10T11:34:18.177Z [INFO] server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f Address:bac5c5bb-1eac-a02d-69eb-ea4fc942051f} {Suffrage:Voter ID:server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd Address:03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd} {Suffrage:Voter ID:server-12ed9356-7797-de21-5e72-dd20b1a0314b Address:12ed9356-7797-de21-5e72-dd20b1a0314b}]" testLoggerAdapter verbose: server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: 2025-02-10T11:34:18.177Z [INFO] server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: entering follower state: follower="Node at 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd [Follower]" leader= testing.go:149: server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: 2025-02-10T11:34:18.177Z [INFO] server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd: entering follower state: follower="Node at 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd [Follower]" leader= testLoggerAdapter verbose: server-12ed9356-7797-de21-5e72-dd20b1a0314b: 2025-02-10T11:34:18.177Z [INFO] server-12ed9356-7797-de21-5e72-dd20b1a0314b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f Address:bac5c5bb-1eac-a02d-69eb-ea4fc942051f} {Suffrage:Voter ID:server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd Address:03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd} {Suffrage:Voter ID:server-12ed9356-7797-de21-5e72-dd20b1a0314b Address:12ed9356-7797-de21-5e72-dd20b1a0314b}]" testing.go:149: server-12ed9356-7797-de21-5e72-dd20b1a0314b: 2025-02-10T11:34:18.177Z [INFO] server-12ed9356-7797-de21-5e72-dd20b1a0314b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f Address:bac5c5bb-1eac-a02d-69eb-ea4fc942051f} {Suffrage:Voter ID:server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd Address:03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd} {Suffrage:Voter ID:server-12ed9356-7797-de21-5e72-dd20b1a0314b Address:12ed9356-7797-de21-5e72-dd20b1a0314b}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.177Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:18.177Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-12ed9356-7797-de21-5e72-dd20b1a0314b: 2025-02-10T11:34:18.177Z [INFO] server-12ed9356-7797-de21-5e72-dd20b1a0314b: entering follower state: follower="Node at 12ed9356-7797-de21-5e72-dd20b1a0314b [Follower]" leader= testing.go:149: server-12ed9356-7797-de21-5e72-dd20b1a0314b: 2025-02-10T11:34:18.177Z [INFO] server-12ed9356-7797-de21-5e72-dd20b1a0314b: entering follower state: follower="Node at 12ed9356-7797-de21-5e72-dd20b1a0314b [Follower]" leader= testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [WARN] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [WARN] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: entering candidate state: node="Node at bac5c5bb-1eac-a02d-69eb-ea4fc942051f [Candidate]" term=2 testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: entering candidate state: node="Node at bac5c5bb-1eac-a02d-69eb-ea4fc942051f [Candidate]" term=2 testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: election won: tally=2 testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: election won: tally=2 testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: entering leader state: leader="Node at bac5c5bb-1eac-a02d-69eb-ea4fc942051f [Leader]" testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: entering leader state: leader="Node at bac5c5bb-1eac-a02d-69eb-ea4fc942051f [Leader]" testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: added peer, starting replication: peer=server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: added peer, starting replication: peer=server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: added peer, starting replication: peer=server-12ed9356-7797-de21-5e72-dd20b1a0314b testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: added peer, starting replication: peer=server-12ed9356-7797-de21-5e72-dd20b1a0314b testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: pipelining replication: peer="{Voter server-12ed9356-7797-de21-5e72-dd20b1a0314b 12ed9356-7797-de21-5e72-dd20b1a0314b}" testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: pipelining replication: peer="{Voter server-12ed9356-7797-de21-5e72-dd20b1a0314b 12ed9356-7797-de21-5e72-dd20b1a0314b}" testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: pipelining replication: peer="{Voter server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd}" testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.230Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: pipelining replication: peer="{Voter server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.335Z [INFO] stable state for Leader reached at 2025-02-10 11:34:18.2306041 +0000 UTC m=+14.599508400 (1 nodes), 53.351796ms from start of poll, 53.721178ms from cluster start. Timeout at 2025-02-10 11:34:18.335707291 +0000 UTC m=+14.704611587, 105.103187ms after stability testing.go:149: cluster: 2025-02-10T11:34:18.335Z [INFO] stable state for Leader reached at 2025-02-10 11:34:18.2306041 +0000 UTC m=+14.599508400 (1 nodes), 53.351796ms from start of poll, 53.721178ms from cluster start. Timeout at 2025-02-10 11:34:18.335707291 +0000 UTC m=+14.704611587, 105.103187ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.340Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:18.340Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.446Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.340941489 +0000 UTC m=+14.709845855 (2 nodes), 1.53µs from start of poll, 164.058633ms from cluster start. Timeout at 2025-02-10 11:34:18.446005357 +0000 UTC m=+14.814909710, 105.063855ms after stability testing.go:149: cluster: 2025-02-10T11:34:18.446Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.340941489 +0000 UTC m=+14.709845855 (2 nodes), 1.53µs from start of poll, 164.058633ms from cluster start. Timeout at 2025-02-10 11:34:18.446005357 +0000 UTC m=+14.814909710, 105.063855ms after stability raft_test.go:1647: [INFO] Disconnecting Node at 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd [Follower] testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.446Z [ERROR] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: failed to heartbeat to: peer=03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd error="failed to connect to peer: 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd" testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.446Z [ERROR] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: failed to heartbeat to: peer=03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd error="failed to connect to peer: 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd" testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.446Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: aborting pipeline replication: peer="{Voter server-12ed9356-7797-de21-5e72-dd20b1a0314b 12ed9356-7797-de21-5e72-dd20b1a0314b}" testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.446Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: aborting pipeline replication: peer="{Voter server-12ed9356-7797-de21-5e72-dd20b1a0314b 12ed9356-7797-de21-5e72-dd20b1a0314b}" testLoggerAdapter verbose: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.446Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: aborting pipeline replication: peer="{Voter server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd}" testing.go:149: server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: 2025-02-10T11:34:18.446Z [INFO] server-bac5c5bb-1eac-a02d-69eb-ea4fc942051f: aborting pipeline replication: peer="{Voter server-03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd 03a0d760-2d05-e76c-b1b7-d8b35d1fa7bd}" --- PASS: TestRaft_VerifyLeader_PartialConnect (0.27s) === RUN TestRaft_StartAsLeader testLoggerAdapter verbose: server-b88fa5e6-0613-426c-d773-9351a28558dd: 2025-02-10T11:34:18.446Z [INFO] server-b88fa5e6-0613-426c-d773-9351a28558dd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b88fa5e6-0613-426c-d773-9351a28558dd Address:b88fa5e6-0613-426c-d773-9351a28558dd}]" testing.go:149: server-b88fa5e6-0613-426c-d773-9351a28558dd: 2025-02-10T11:34:18.446Z [INFO] server-b88fa5e6-0613-426c-d773-9351a28558dd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b88fa5e6-0613-426c-d773-9351a28558dd Address:b88fa5e6-0613-426c-d773-9351a28558dd}]" testLoggerAdapter verbose: server-b88fa5e6-0613-426c-d773-9351a28558dd: 2025-02-10T11:34:18.446Z [INFO] server-b88fa5e6-0613-426c-d773-9351a28558dd: entering leader state: leader="Node at b88fa5e6-0613-426c-d773-9351a28558dd [Leader]" testing.go:149: server-b88fa5e6-0613-426c-d773-9351a28558dd: 2025-02-10T11:34:18.446Z [INFO] server-b88fa5e6-0613-426c-d773-9351a28558dd: entering leader state: leader="Node at b88fa5e6-0613-426c-d773-9351a28558dd [Leader]" --- PASS: TestRaft_StartAsLeader (0.00s) === RUN TestRaft_NotifyCh testLoggerAdapter verbose: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.447Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-98ed08f6-7bd1-9066-db4d-500934aeb0f4 Address:98ed08f6-7bd1-9066-db4d-500934aeb0f4}]" testing.go:149: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.447Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-98ed08f6-7bd1-9066-db4d-500934aeb0f4 Address:98ed08f6-7bd1-9066-db4d-500934aeb0f4}]" testLoggerAdapter verbose: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.447Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: entering follower state: follower="Node at 98ed08f6-7bd1-9066-db4d-500934aeb0f4 [Follower]" leader= testing.go:149: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.447Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: entering follower state: follower="Node at 98ed08f6-7bd1-9066-db4d-500934aeb0f4 [Follower]" leader= testLoggerAdapter verbose: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [WARN] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [WARN] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: entering candidate state: node="Node at 98ed08f6-7bd1-9066-db4d-500934aeb0f4 [Candidate]" term=2 testing.go:149: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: entering candidate state: node="Node at 98ed08f6-7bd1-9066-db4d-500934aeb0f4 [Candidate]" term=2 testLoggerAdapter verbose: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: election won: tally=1 testing.go:149: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: election won: tally=1 testLoggerAdapter verbose: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: entering leader state: leader="Node at 98ed08f6-7bd1-9066-db4d-500934aeb0f4 [Leader]" testing.go:149: server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: 2025-02-10T11:34:18.504Z [INFO] server-98ed08f6-7bd1-9066-db4d-500934aeb0f4: entering leader state: leader="Node at 98ed08f6-7bd1-9066-db4d-500934aeb0f4 [Leader]" --- PASS: TestRaft_NotifyCh (0.06s) === RUN TestRaft_Voting testLoggerAdapter verbose: server-5538d946-b962-50c5-417b-f59a15c4777e: 2025-02-10T11:34:18.505Z [INFO] server-5538d946-b962-50c5-417b-f59a15c4777e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5538d946-b962-50c5-417b-f59a15c4777e Address:5538d946-b962-50c5-417b-f59a15c4777e} {Suffrage:Voter ID:server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60 Address:73b68a81-53cb-35bf-6a1b-4f62aad6ca60} {Suffrage:Voter ID:server-09601f2e-2db5-c268-1618-90a2f3c92d1e Address:09601f2e-2db5-c268-1618-90a2f3c92d1e}]" testing.go:149: server-5538d946-b962-50c5-417b-f59a15c4777e: 2025-02-10T11:34:18.505Z [INFO] server-5538d946-b962-50c5-417b-f59a15c4777e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5538d946-b962-50c5-417b-f59a15c4777e Address:5538d946-b962-50c5-417b-f59a15c4777e} {Suffrage:Voter ID:server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60 Address:73b68a81-53cb-35bf-6a1b-4f62aad6ca60} {Suffrage:Voter ID:server-09601f2e-2db5-c268-1618-90a2f3c92d1e Address:09601f2e-2db5-c268-1618-90a2f3c92d1e}]" testLoggerAdapter verbose: server-5538d946-b962-50c5-417b-f59a15c4777e: 2025-02-10T11:34:18.505Z [INFO] server-5538d946-b962-50c5-417b-f59a15c4777e: entering follower state: follower="Node at 5538d946-b962-50c5-417b-f59a15c4777e [Follower]" leader= testing.go:149: server-5538d946-b962-50c5-417b-f59a15c4777e: 2025-02-10T11:34:18.505Z [INFO] server-5538d946-b962-50c5-417b-f59a15c4777e: entering follower state: follower="Node at 5538d946-b962-50c5-417b-f59a15c4777e [Follower]" leader= testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.505Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5538d946-b962-50c5-417b-f59a15c4777e Address:5538d946-b962-50c5-417b-f59a15c4777e} {Suffrage:Voter ID:server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60 Address:73b68a81-53cb-35bf-6a1b-4f62aad6ca60} {Suffrage:Voter ID:server-09601f2e-2db5-c268-1618-90a2f3c92d1e Address:09601f2e-2db5-c268-1618-90a2f3c92d1e}]" testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.505Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5538d946-b962-50c5-417b-f59a15c4777e Address:5538d946-b962-50c5-417b-f59a15c4777e} {Suffrage:Voter ID:server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60 Address:73b68a81-53cb-35bf-6a1b-4f62aad6ca60} {Suffrage:Voter ID:server-09601f2e-2db5-c268-1618-90a2f3c92d1e Address:09601f2e-2db5-c268-1618-90a2f3c92d1e}]" testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.505Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: entering follower state: follower="Node at 73b68a81-53cb-35bf-6a1b-4f62aad6ca60 [Follower]" leader= testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.505Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: entering follower state: follower="Node at 73b68a81-53cb-35bf-6a1b-4f62aad6ca60 [Follower]" leader= testLoggerAdapter verbose: server-09601f2e-2db5-c268-1618-90a2f3c92d1e: 2025-02-10T11:34:18.505Z [INFO] server-09601f2e-2db5-c268-1618-90a2f3c92d1e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5538d946-b962-50c5-417b-f59a15c4777e Address:5538d946-b962-50c5-417b-f59a15c4777e} {Suffrage:Voter ID:server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60 Address:73b68a81-53cb-35bf-6a1b-4f62aad6ca60} {Suffrage:Voter ID:server-09601f2e-2db5-c268-1618-90a2f3c92d1e Address:09601f2e-2db5-c268-1618-90a2f3c92d1e}]" testing.go:149: server-09601f2e-2db5-c268-1618-90a2f3c92d1e: 2025-02-10T11:34:18.505Z [INFO] server-09601f2e-2db5-c268-1618-90a2f3c92d1e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5538d946-b962-50c5-417b-f59a15c4777e Address:5538d946-b962-50c5-417b-f59a15c4777e} {Suffrage:Voter ID:server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60 Address:73b68a81-53cb-35bf-6a1b-4f62aad6ca60} {Suffrage:Voter ID:server-09601f2e-2db5-c268-1618-90a2f3c92d1e Address:09601f2e-2db5-c268-1618-90a2f3c92d1e}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.505Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:18.505Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-09601f2e-2db5-c268-1618-90a2f3c92d1e: 2025-02-10T11:34:18.505Z [INFO] server-09601f2e-2db5-c268-1618-90a2f3c92d1e: entering follower state: follower="Node at 09601f2e-2db5-c268-1618-90a2f3c92d1e [Follower]" leader= testing.go:149: server-09601f2e-2db5-c268-1618-90a2f3c92d1e: 2025-02-10T11:34:18.505Z [INFO] server-09601f2e-2db5-c268-1618-90a2f3c92d1e: entering follower state: follower="Node at 09601f2e-2db5-c268-1618-90a2f3c92d1e [Follower]" leader= testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [WARN] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [WARN] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: entering candidate state: node="Node at 73b68a81-53cb-35bf-6a1b-4f62aad6ca60 [Candidate]" term=2 testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: entering candidate state: node="Node at 73b68a81-53cb-35bf-6a1b-4f62aad6ca60 [Candidate]" term=2 testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: election won: tally=2 testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: election won: tally=2 testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: entering leader state: leader="Node at 73b68a81-53cb-35bf-6a1b-4f62aad6ca60 [Leader]" testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: entering leader state: leader="Node at 73b68a81-53cb-35bf-6a1b-4f62aad6ca60 [Leader]" testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: added peer, starting replication: peer=server-5538d946-b962-50c5-417b-f59a15c4777e testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: added peer, starting replication: peer=server-5538d946-b962-50c5-417b-f59a15c4777e testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: added peer, starting replication: peer=server-09601f2e-2db5-c268-1618-90a2f3c92d1e testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: added peer, starting replication: peer=server-09601f2e-2db5-c268-1618-90a2f3c92d1e testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: pipelining replication: peer="{Voter server-5538d946-b962-50c5-417b-f59a15c4777e 5538d946-b962-50c5-417b-f59a15c4777e}" testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: pipelining replication: peer="{Voter server-5538d946-b962-50c5-417b-f59a15c4777e 5538d946-b962-50c5-417b-f59a15c4777e}" testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: pipelining replication: peer="{Voter server-09601f2e-2db5-c268-1618-90a2f3c92d1e 09601f2e-2db5-c268-1618-90a2f3c92d1e}" testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.570Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: pipelining replication: peer="{Voter server-09601f2e-2db5-c268-1618-90a2f3c92d1e 09601f2e-2db5-c268-1618-90a2f3c92d1e}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.675Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.570658868 +0000 UTC m=+14.939563168 (2 nodes), 64.742183ms from start of poll, 65.137762ms from cluster start. Timeout at 2025-02-10 11:34:18.675752776 +0000 UTC m=+15.044657215, 105.094047ms after stability testing.go:149: cluster: 2025-02-10T11:34:18.675Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.570658868 +0000 UTC m=+14.939563168 (2 nodes), 64.742183ms from start of poll, 65.137762ms from cluster start. Timeout at 2025-02-10 11:34:18.675752776 +0000 UTC m=+15.044657215, 105.094047ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.675Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:18.675Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.780Z [INFO] stable state for Leader reached at 2025-02-10 11:34:18.675893497 +0000 UTC m=+15.044797864 (1 nodes), 919ns from start of poll, 170.372458ms from cluster start. Timeout at 2025-02-10 11:34:18.780971934 +0000 UTC m=+15.149876225, 105.078361ms after stability testing.go:149: cluster: 2025-02-10T11:34:18.780Z [INFO] stable state for Leader reached at 2025-02-10 11:34:18.675893497 +0000 UTC m=+15.044797864 (1 nodes), 919ns from start of poll, 170.372458ms from cluster start. Timeout at 2025-02-10 11:34:18.780971934 +0000 UTC m=+15.149876225, 105.078361ms after stability testLoggerAdapter verbose: server-09601f2e-2db5-c268-1618-90a2f3c92d1e: 2025-02-10T11:34:18.781Z [WARN] server-09601f2e-2db5-c268-1618-90a2f3c92d1e: rejecting vote request since we have a leader: from=5538d946-b962-50c5-417b-f59a15c4777e leader=73b68a81-53cb-35bf-6a1b-4f62aad6ca60 testing.go:149: server-09601f2e-2db5-c268-1618-90a2f3c92d1e: 2025-02-10T11:34:18.781Z [WARN] server-09601f2e-2db5-c268-1618-90a2f3c92d1e: rejecting vote request since we have a leader: from=5538d946-b962-50c5-417b-f59a15c4777e leader=73b68a81-53cb-35bf-6a1b-4f62aad6ca60 testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.781Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: aborting pipeline replication: peer="{Voter server-09601f2e-2db5-c268-1618-90a2f3c92d1e 09601f2e-2db5-c268-1618-90a2f3c92d1e}" testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.781Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: aborting pipeline replication: peer="{Voter server-09601f2e-2db5-c268-1618-90a2f3c92d1e 09601f2e-2db5-c268-1618-90a2f3c92d1e}" testLoggerAdapter verbose: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.781Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: aborting pipeline replication: peer="{Voter server-5538d946-b962-50c5-417b-f59a15c4777e 5538d946-b962-50c5-417b-f59a15c4777e}" testing.go:149: server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: 2025-02-10T11:34:18.781Z [INFO] server-73b68a81-53cb-35bf-6a1b-4f62aad6ca60: aborting pipeline replication: peer="{Voter server-5538d946-b962-50c5-417b-f59a15c4777e 5538d946-b962-50c5-417b-f59a15c4777e}" --- PASS: TestRaft_Voting (0.28s) === RUN TestRaft_ProtocolVersion_RejectRPC testLoggerAdapter verbose: server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: 2025-02-10T11:34:18.782Z [INFO] server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad Address:e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad} {Suffrage:Voter ID:server-fab3e159-c600-8d9b-2dae-4217b3a13060 Address:fab3e159-c600-8d9b-2dae-4217b3a13060} {Suffrage:Voter ID:server-651b1828-d980-261c-fd3f-9fe7a58fc860 Address:651b1828-d980-261c-fd3f-9fe7a58fc860}]" testing.go:149: server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: 2025-02-10T11:34:18.782Z [INFO] server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad Address:e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad} {Suffrage:Voter ID:server-fab3e159-c600-8d9b-2dae-4217b3a13060 Address:fab3e159-c600-8d9b-2dae-4217b3a13060} {Suffrage:Voter ID:server-651b1828-d980-261c-fd3f-9fe7a58fc860 Address:651b1828-d980-261c-fd3f-9fe7a58fc860}]" testLoggerAdapter verbose: server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: 2025-02-10T11:34:18.782Z [INFO] server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: entering follower state: follower="Node at e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad [Follower]" leader= testing.go:149: server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: 2025-02-10T11:34:18.782Z [INFO] server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad: entering follower state: follower="Node at e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad [Follower]" leader= testLoggerAdapter verbose: server-fab3e159-c600-8d9b-2dae-4217b3a13060: 2025-02-10T11:34:18.782Z [INFO] server-fab3e159-c600-8d9b-2dae-4217b3a13060: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad Address:e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad} {Suffrage:Voter ID:server-fab3e159-c600-8d9b-2dae-4217b3a13060 Address:fab3e159-c600-8d9b-2dae-4217b3a13060} {Suffrage:Voter ID:server-651b1828-d980-261c-fd3f-9fe7a58fc860 Address:651b1828-d980-261c-fd3f-9fe7a58fc860}]" testing.go:149: server-fab3e159-c600-8d9b-2dae-4217b3a13060: 2025-02-10T11:34:18.782Z [INFO] server-fab3e159-c600-8d9b-2dae-4217b3a13060: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad Address:e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad} {Suffrage:Voter ID:server-fab3e159-c600-8d9b-2dae-4217b3a13060 Address:fab3e159-c600-8d9b-2dae-4217b3a13060} {Suffrage:Voter ID:server-651b1828-d980-261c-fd3f-9fe7a58fc860 Address:651b1828-d980-261c-fd3f-9fe7a58fc860}]" testLoggerAdapter verbose: server-fab3e159-c600-8d9b-2dae-4217b3a13060: 2025-02-10T11:34:18.782Z [INFO] server-fab3e159-c600-8d9b-2dae-4217b3a13060: entering follower state: follower="Node at fab3e159-c600-8d9b-2dae-4217b3a13060 [Follower]" leader= testing.go:149: server-fab3e159-c600-8d9b-2dae-4217b3a13060: 2025-02-10T11:34:18.782Z [INFO] server-fab3e159-c600-8d9b-2dae-4217b3a13060: entering follower state: follower="Node at fab3e159-c600-8d9b-2dae-4217b3a13060 [Follower]" leader= testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.782Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad Address:e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad} {Suffrage:Voter ID:server-fab3e159-c600-8d9b-2dae-4217b3a13060 Address:fab3e159-c600-8d9b-2dae-4217b3a13060} {Suffrage:Voter ID:server-651b1828-d980-261c-fd3f-9fe7a58fc860 Address:651b1828-d980-261c-fd3f-9fe7a58fc860}]" testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.782Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad Address:e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad} {Suffrage:Voter ID:server-fab3e159-c600-8d9b-2dae-4217b3a13060 Address:fab3e159-c600-8d9b-2dae-4217b3a13060} {Suffrage:Voter ID:server-651b1828-d980-261c-fd3f-9fe7a58fc860 Address:651b1828-d980-261c-fd3f-9fe7a58fc860}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.782Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:18.782Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.782Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: entering follower state: follower="Node at 651b1828-d980-261c-fd3f-9fe7a58fc860 [Follower]" leader= testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.782Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: entering follower state: follower="Node at 651b1828-d980-261c-fd3f-9fe7a58fc860 [Follower]" leader= testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [WARN] server-651b1828-d980-261c-fd3f-9fe7a58fc860: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [WARN] server-651b1828-d980-261c-fd3f-9fe7a58fc860: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: entering candidate state: node="Node at 651b1828-d980-261c-fd3f-9fe7a58fc860 [Candidate]" term=2 testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: entering candidate state: node="Node at 651b1828-d980-261c-fd3f-9fe7a58fc860 [Candidate]" term=2 testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: election won: tally=2 testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: election won: tally=2 testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: entering leader state: leader="Node at 651b1828-d980-261c-fd3f-9fe7a58fc860 [Leader]" testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: entering leader state: leader="Node at 651b1828-d980-261c-fd3f-9fe7a58fc860 [Leader]" testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: added peer, starting replication: peer=server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: added peer, starting replication: peer=server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: added peer, starting replication: peer=server-fab3e159-c600-8d9b-2dae-4217b3a13060 testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: added peer, starting replication: peer=server-fab3e159-c600-8d9b-2dae-4217b3a13060 testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: pipelining replication: peer="{Voter server-fab3e159-c600-8d9b-2dae-4217b3a13060 fab3e159-c600-8d9b-2dae-4217b3a13060}" testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: pipelining replication: peer="{Voter server-fab3e159-c600-8d9b-2dae-4217b3a13060 fab3e159-c600-8d9b-2dae-4217b3a13060}" testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: pipelining replication: peer="{Voter server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad}" testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:18.862Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: pipelining replication: peer="{Voter server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.967Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.862506272 +0000 UTC m=+15.231410573 (2 nodes), 80.062498ms from start of poll, 80.460056ms from cluster start. Timeout at 2025-02-10 11:34:18.967629312 +0000 UTC m=+15.336533668, 105.123095ms after stability testing.go:149: cluster: 2025-02-10T11:34:18.967Z [INFO] stable state for Follower reached at 2025-02-10 11:34:18.862506272 +0000 UTC m=+15.231410573 (2 nodes), 80.062498ms from start of poll, 80.460056ms from cluster start. Timeout at 2025-02-10 11:34:18.967629312 +0000 UTC m=+15.336533668, 105.123095ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:18.967Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:18.967Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.072Z [INFO] stable state for Leader reached at 2025-02-10 11:34:18.967715384 +0000 UTC m=+15.336619689 (1 nodes), 480ns from start of poll, 185.669172ms from cluster start. Timeout at 2025-02-10 11:34:19.072818767 +0000 UTC m=+15.441723116, 105.103427ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.072Z [INFO] stable state for Leader reached at 2025-02-10 11:34:18.967715384 +0000 UTC m=+15.336619689 (1 nodes), 480ns from start of poll, 185.669172ms from cluster start. Timeout at 2025-02-10 11:34:19.072818767 +0000 UTC m=+15.441723116, 105.103427ms after stability testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:19.072Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: aborting pipeline replication: peer="{Voter server-fab3e159-c600-8d9b-2dae-4217b3a13060 fab3e159-c600-8d9b-2dae-4217b3a13060}" testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:19.072Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: aborting pipeline replication: peer="{Voter server-fab3e159-c600-8d9b-2dae-4217b3a13060 fab3e159-c600-8d9b-2dae-4217b3a13060}" testLoggerAdapter verbose: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:19.073Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: aborting pipeline replication: peer="{Voter server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad}" testing.go:149: server-651b1828-d980-261c-fd3f-9fe7a58fc860: 2025-02-10T11:34:19.073Z [INFO] server-651b1828-d980-261c-fd3f-9fe7a58fc860: aborting pipeline replication: peer="{Voter server-e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad e0367ca5-f0bf-cf68-dd2b-aca7e4a8b8ad}" --- PASS: TestRaft_ProtocolVersion_RejectRPC (0.29s) === RUN TestRaft_ProtocolVersion_Upgrade_1_2 testLoggerAdapter verbose: 820096a2-2c4d-d93e-d653-e1acec3ecd72: 2025-02-10T11:34:19.073Z [INFO] 820096a2-2c4d-d93e-d653-e1acec3ecd72: initial configuration: index=1 servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e}]" testing.go:149: 820096a2-2c4d-d93e-d653-e1acec3ecd72: 2025-02-10T11:34:19.073Z [INFO] 820096a2-2c4d-d93e-d653-e1acec3ecd72: initial configuration: index=1 servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e}]" testLoggerAdapter verbose: 820096a2-2c4d-d93e-d653-e1acec3ecd72: 2025-02-10T11:34:19.074Z [INFO] 820096a2-2c4d-d93e-d653-e1acec3ecd72: entering follower state: follower="Node at 820096a2-2c4d-d93e-d653-e1acec3ecd72 [Follower]" leader= testing.go:149: 820096a2-2c4d-d93e-d653-e1acec3ecd72: 2025-02-10T11:34:19.074Z [INFO] 820096a2-2c4d-d93e-d653-e1acec3ecd72: entering follower state: follower="Node at 820096a2-2c4d-d93e-d653-e1acec3ecd72 [Follower]" leader= testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.074Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: initial configuration: index=1 servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e}]" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.074Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: initial configuration: index=1 servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e}]" testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.074Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: entering follower state: follower="Node at 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e [Follower]" leader= testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.074Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: entering follower state: follower="Node at 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e [Follower]" leader= testLoggerAdapter verbose: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.074Z [INFO] 57860ed7-e494-1908-3873-847abb3203bb: initial configuration: index=0 servers=[] testing.go:149: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.074Z [INFO] 57860ed7-e494-1908-3873-847abb3203bb: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.074Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:19.074Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.074Z [INFO] 57860ed7-e494-1908-3873-847abb3203bb: entering follower state: follower="Node at 57860ed7-e494-1908-3873-847abb3203bb [Follower]" leader= testing.go:149: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.074Z [INFO] 57860ed7-e494-1908-3873-847abb3203bb: entering follower state: follower="Node at 57860ed7-e494-1908-3873-847abb3203bb [Follower]" leader= testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [WARN] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: heartbeat timeout reached, starting election: last-leader= testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [WARN] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: entering candidate state: node="Node at 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e [Candidate]" term=2 testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: entering candidate state: node="Node at 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e [Candidate]" term=2 testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: election won: tally=2 testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: election won: tally=2 testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: entering leader state: leader="Node at 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e [Leader]" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: entering leader state: leader="Node at 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e [Leader]" testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: added peer, starting replication: peer=820096a2-2c4d-d93e-d653-e1acec3ecd72 testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: added peer, starting replication: peer=820096a2-2c4d-d93e-d653-e1acec3ecd72 testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: pipelining replication: peer="{Voter 820096a2-2c4d-d93e-d653-e1acec3ecd72 820096a2-2c4d-d93e-d653-e1acec3ecd72}" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.159Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: pipelining replication: peer="{Voter 820096a2-2c4d-d93e-d653-e1acec3ecd72 820096a2-2c4d-d93e-d653-e1acec3ecd72}" testLoggerAdapter verbose: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.169Z [WARN] 57860ed7-e494-1908-3873-847abb3203bb: no known peers, aborting election testing.go:149: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.169Z [WARN] 57860ed7-e494-1908-3873-847abb3203bb: no known peers, aborting election testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.264Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.159122873 +0000 UTC m=+15.528027175 (1 nodes), 84.800972ms from start of poll, 85.273288ms from cluster start. Timeout at 2025-02-10 11:34:19.264220798 +0000 UTC m=+15.633125155, 105.09798ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.264Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.159122873 +0000 UTC m=+15.528027175 (1 nodes), 84.800972ms from start of poll, 85.273288ms from cluster start. Timeout at 2025-02-10 11:34:19.264220798 +0000 UTC m=+15.633125155, 105.09798ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.264Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:19.264Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.369Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.264376859 +0000 UTC m=+15.633281222 (1 nodes), 912ns from start of poll, 190.527335ms from cluster start. Timeout at 2025-02-10 11:34:19.369451587 +0000 UTC m=+15.738355885, 105.074663ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.369Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.264376859 +0000 UTC m=+15.633281222 (1 nodes), 912ns from start of poll, 190.527335ms from cluster start. Timeout at 2025-02-10 11:34:19.369451587 +0000 UTC m=+15.738355885, 105.074663ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.369Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:19.369Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.474Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.369525422 +0000 UTC m=+15.738429721 (1 nodes), 511ns from start of poll, 295.675834ms from cluster start. Timeout at 2025-02-10 11:34:19.474598367 +0000 UTC m=+15.843502661, 105.07294ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.474Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.369525422 +0000 UTC m=+15.738429721 (1 nodes), 511ns from start of poll, 295.675834ms from cluster start. Timeout at 2025-02-10 11:34:19.474598367 +0000 UTC m=+15.843502661, 105.07294ms after stability testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: updating configuration: command=AddStaging server-id=57860ed7-e494-1908-3873-847abb3203bb server-addr=57860ed7-e494-1908-3873-847abb3203bb servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e} {Suffrage:Voter ID:57860ed7-e494-1908-3873-847abb3203bb Address:57860ed7-e494-1908-3873-847abb3203bb}]" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: updating configuration: command=AddStaging server-id=57860ed7-e494-1908-3873-847abb3203bb server-addr=57860ed7-e494-1908-3873-847abb3203bb servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e} {Suffrage:Voter ID:57860ed7-e494-1908-3873-847abb3203bb Address:57860ed7-e494-1908-3873-847abb3203bb}]" testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: added peer, starting replication: peer=57860ed7-e494-1908-3873-847abb3203bb testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: added peer, starting replication: peer=57860ed7-e494-1908-3873-847abb3203bb testLoggerAdapter verbose: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.474Z [WARN] 57860ed7-e494-1908-3873-847abb3203bb: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: 57860ed7-e494-1908-3873-847abb3203bb: 2025-02-10T11:34:19.474Z [WARN] 57860ed7-e494-1908-3873-847abb3203bb: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [WARN] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: appendEntries rejected, sending older logs: peer="{Voter 57860ed7-e494-1908-3873-847abb3203bb 57860ed7-e494-1908-3873-847abb3203bb}" next=1 testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [WARN] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: appendEntries rejected, sending older logs: peer="{Voter 57860ed7-e494-1908-3873-847abb3203bb 57860ed7-e494-1908-3873-847abb3203bb}" next=1 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.474Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:19.474Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: pipelining replication: peer="{Voter 57860ed7-e494-1908-3873-847abb3203bb 57860ed7-e494-1908-3873-847abb3203bb}" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.474Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: pipelining replication: peer="{Voter 57860ed7-e494-1908-3873-847abb3203bb 57860ed7-e494-1908-3873-847abb3203bb}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.580Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.474974456 +0000 UTC m=+15.843878764 (1 nodes), 728ns from start of poll, 401.124877ms from cluster start. Timeout at 2025-02-10 11:34:19.580067964 +0000 UTC m=+15.948972317, 105.093553ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.580Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.474974456 +0000 UTC m=+15.843878764 (1 nodes), 728ns from start of poll, 401.124877ms from cluster start. Timeout at 2025-02-10 11:34:19.580067964 +0000 UTC m=+15.948972317, 105.093553ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.580Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:19.580Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.685Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.580208241 +0000 UTC m=+15.949112611 (1 nodes), 832ns from start of poll, 506.358724ms from cluster start. Timeout at 2025-02-10 11:34:19.685276684 +0000 UTC m=+16.054180981, 105.06837ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.685Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.580208241 +0000 UTC m=+15.949112611 (1 nodes), 832ns from start of poll, 506.358724ms from cluster start. Timeout at 2025-02-10 11:34:19.685276684 +0000 UTC m=+16.054180981, 105.06837ms after stability testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: updating configuration: command=RemoveServer server-id=57860ed7-e494-1908-3873-847abb3203bb server-addr= servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e}]" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: updating configuration: command=RemoveServer server-id=57860ed7-e494-1908-3873-847abb3203bb server-addr= servers="[{Suffrage:Voter ID:820096a2-2c4d-d93e-d653-e1acec3ecd72 Address:820096a2-2c4d-d93e-d653-e1acec3ecd72} {Suffrage:Voter ID:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e Address:879bd73a-7e9c-c4b9-83bf-69016e3a1b6e}]" testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: removed peer, stopping replication: peer=57860ed7-e494-1908-3873-847abb3203bb last-index=4 testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: removed peer, stopping replication: peer=57860ed7-e494-1908-3873-847abb3203bb last-index=4 testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: aborting pipeline replication: peer="{Voter 57860ed7-e494-1908-3873-847abb3203bb 57860ed7-e494-1908-3873-847abb3203bb}" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: aborting pipeline replication: peer="{Voter 57860ed7-e494-1908-3873-847abb3203bb 57860ed7-e494-1908-3873-847abb3203bb}" testLoggerAdapter verbose: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: aborting pipeline replication: peer="{Voter 820096a2-2c4d-d93e-d653-e1acec3ecd72 820096a2-2c4d-d93e-d653-e1acec3ecd72}" testing.go:149: 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: 2025-02-10T11:34:19.685Z [INFO] 879bd73a-7e9c-c4b9-83bf-69016e3a1b6e: aborting pipeline replication: peer="{Voter 820096a2-2c4d-d93e-d653-e1acec3ecd72 820096a2-2c4d-d93e-d653-e1acec3ecd72}" --- PASS: TestRaft_ProtocolVersion_Upgrade_1_2 (0.61s) === RUN TestRaft_ProtocolVersion_Upgrade_2_3 testLoggerAdapter verbose: 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: 2025-02-10T11:34:19.687Z [INFO] 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: initial configuration: index=1 servers="[{Suffrage:Voter ID:2ff23827-10c6-84b8-e7a4-acd51c6d28fc Address:2ff23827-10c6-84b8-e7a4-acd51c6d28fc} {Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a}]" testing.go:149: 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: 2025-02-10T11:34:19.687Z [INFO] 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: initial configuration: index=1 servers="[{Suffrage:Voter ID:2ff23827-10c6-84b8-e7a4-acd51c6d28fc Address:2ff23827-10c6-84b8-e7a4-acd51c6d28fc} {Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a}]" testLoggerAdapter verbose: 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: 2025-02-10T11:34:19.687Z [INFO] 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: entering follower state: follower="Node at 2ff23827-10c6-84b8-e7a4-acd51c6d28fc [Follower]" leader= testing.go:149: 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: 2025-02-10T11:34:19.687Z [INFO] 2ff23827-10c6-84b8-e7a4-acd51c6d28fc: entering follower state: follower="Node at 2ff23827-10c6-84b8-e7a4-acd51c6d28fc [Follower]" leader= testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.687Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: initial configuration: index=1 servers="[{Suffrage:Voter ID:2ff23827-10c6-84b8-e7a4-acd51c6d28fc Address:2ff23827-10c6-84b8-e7a4-acd51c6d28fc} {Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a}]" testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.687Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: initial configuration: index=1 servers="[{Suffrage:Voter ID:2ff23827-10c6-84b8-e7a4-acd51c6d28fc Address:2ff23827-10c6-84b8-e7a4-acd51c6d28fc} {Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.687Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.687Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: entering follower state: follower="Node at e21603ea-911a-e69c-4c1b-44b4ff19324a [Follower]" leader= testing.go:149: cluster: 2025-02-10T11:34:19.687Z [INFO] starting stability test: raft-state=Follower testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.687Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: entering follower state: follower="Node at e21603ea-911a-e69c-4c1b-44b4ff19324a [Follower]" leader= testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [WARN] e21603ea-911a-e69c-4c1b-44b4ff19324a: heartbeat timeout reached, starting election: last-leader= testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [WARN] e21603ea-911a-e69c-4c1b-44b4ff19324a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: entering candidate state: node="Node at e21603ea-911a-e69c-4c1b-44b4ff19324a [Candidate]" term=2 testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: entering candidate state: node="Node at e21603ea-911a-e69c-4c1b-44b4ff19324a [Candidate]" term=2 testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: election won: tally=2 testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: election won: tally=2 testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: entering leader state: leader="Node at e21603ea-911a-e69c-4c1b-44b4ff19324a [Leader]" testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: entering leader state: leader="Node at e21603ea-911a-e69c-4c1b-44b4ff19324a [Leader]" testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: added peer, starting replication: peer=2ff23827-10c6-84b8-e7a4-acd51c6d28fc testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: added peer, starting replication: peer=2ff23827-10c6-84b8-e7a4-acd51c6d28fc testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: pipelining replication: peer="{Voter 2ff23827-10c6-84b8-e7a4-acd51c6d28fc 2ff23827-10c6-84b8-e7a4-acd51c6d28fc}" testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.739Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: pipelining replication: peer="{Voter 2ff23827-10c6-84b8-e7a4-acd51c6d28fc 2ff23827-10c6-84b8-e7a4-acd51c6d28fc}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.845Z [INFO] stable state for Follower reached at 2025-02-10 11:34:19.739848152 +0000 UTC m=+16.108752493 (1 nodes), 52.340641ms from start of poll, 52.731566ms from cluster start. Timeout at 2025-02-10 11:34:19.844953176 +0000 UTC m=+16.213857532, 105.105039ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.845Z [INFO] stable state for Follower reached at 2025-02-10 11:34:19.739848152 +0000 UTC m=+16.108752493 (1 nodes), 52.340641ms from start of poll, 52.731566ms from cluster start. Timeout at 2025-02-10 11:34:19.844953176 +0000 UTC m=+16.213857532, 105.105039ms after stability testLoggerAdapter verbose: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.845Z [INFO] server-dd42822a-5656-f993-ffac-3e11b416e675: initial configuration: index=0 servers=[] testing.go:149: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.845Z [INFO] server-dd42822a-5656-f993-ffac-3e11b416e675: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.845Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:19.845Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.845Z [INFO] server-dd42822a-5656-f993-ffac-3e11b416e675: entering follower state: follower="Node at dd42822a-5656-f993-ffac-3e11b416e675 [Follower]" leader= testing.go:149: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.845Z [INFO] server-dd42822a-5656-f993-ffac-3e11b416e675: entering follower state: follower="Node at dd42822a-5656-f993-ffac-3e11b416e675 [Follower]" leader= testLoggerAdapter verbose: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.927Z [WARN] server-dd42822a-5656-f993-ffac-3e11b416e675: no known peers, aborting election testing.go:149: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.927Z [WARN] server-dd42822a-5656-f993-ffac-3e11b416e675: no known peers, aborting election testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.950Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.845441527 +0000 UTC m=+16.214345828 (1 nodes), 482ns from start of poll, 158.324901ms from cluster start. Timeout at 2025-02-10 11:34:19.950514667 +0000 UTC m=+16.319418963, 105.073135ms after stability testing.go:149: cluster: 2025-02-10T11:34:19.950Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.845441527 +0000 UTC m=+16.214345828 (1 nodes), 482ns from start of poll, 158.324901ms from cluster start. Timeout at 2025-02-10 11:34:19.950514667 +0000 UTC m=+16.319418963, 105.073135ms after stability testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: updating configuration: command=AddStaging server-id=server-dd42822a-5656-f993-ffac-3e11b416e675 server-addr=dd42822a-5656-f993-ffac-3e11b416e675 servers="[{Suffrage:Voter ID:2ff23827-10c6-84b8-e7a4-acd51c6d28fc Address:2ff23827-10c6-84b8-e7a4-acd51c6d28fc} {Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a} {Suffrage:Voter ID:server-dd42822a-5656-f993-ffac-3e11b416e675 Address:dd42822a-5656-f993-ffac-3e11b416e675}]" testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: updating configuration: command=AddStaging server-id=server-dd42822a-5656-f993-ffac-3e11b416e675 server-addr=dd42822a-5656-f993-ffac-3e11b416e675 servers="[{Suffrage:Voter ID:2ff23827-10c6-84b8-e7a4-acd51c6d28fc Address:2ff23827-10c6-84b8-e7a4-acd51c6d28fc} {Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a} {Suffrage:Voter ID:server-dd42822a-5656-f993-ffac-3e11b416e675 Address:dd42822a-5656-f993-ffac-3e11b416e675}]" testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: added peer, starting replication: peer=server-dd42822a-5656-f993-ffac-3e11b416e675 testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: added peer, starting replication: peer=server-dd42822a-5656-f993-ffac-3e11b416e675 testLoggerAdapter verbose: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.950Z [WARN] server-dd42822a-5656-f993-ffac-3e11b416e675: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-dd42822a-5656-f993-ffac-3e11b416e675: 2025-02-10T11:34:19.950Z [WARN] server-dd42822a-5656-f993-ffac-3e11b416e675: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [WARN] e21603ea-911a-e69c-4c1b-44b4ff19324a: appendEntries rejected, sending older logs: peer="{Voter server-dd42822a-5656-f993-ffac-3e11b416e675 dd42822a-5656-f993-ffac-3e11b416e675}" next=1 testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [WARN] e21603ea-911a-e69c-4c1b-44b4ff19324a: appendEntries rejected, sending older logs: peer="{Voter server-dd42822a-5656-f993-ffac-3e11b416e675 dd42822a-5656-f993-ffac-3e11b416e675}" next=1 testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: pipelining replication: peer="{Voter server-dd42822a-5656-f993-ffac-3e11b416e675 dd42822a-5656-f993-ffac-3e11b416e675}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:19.950Z [INFO] starting stability test: raft-state=Leader testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:19.950Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: pipelining replication: peer="{Voter server-dd42822a-5656-f993-ffac-3e11b416e675 dd42822a-5656-f993-ffac-3e11b416e675}" testing.go:149: cluster: 2025-02-10T11:34:19.950Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.056Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.950942655 +0000 UTC m=+16.319846956 (1 nodes), 661ns from start of poll, 263.826029ms from cluster start. Timeout at 2025-02-10 11:34:20.05601515 +0000 UTC m=+16.424919446, 105.07249ms after stability testing.go:149: cluster: 2025-02-10T11:34:20.056Z [INFO] stable state for Leader reached at 2025-02-10 11:34:19.950942655 +0000 UTC m=+16.319846956 (1 nodes), 661ns from start of poll, 263.826029ms from cluster start. Timeout at 2025-02-10 11:34:20.05601515 +0000 UTC m=+16.424919446, 105.07249ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.056Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:20.056Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.161Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.056095734 +0000 UTC m=+16.425000032 (1 nodes), 406ns from start of poll, 368.979105ms from cluster start. Timeout at 2025-02-10 11:34:20.161211244 +0000 UTC m=+16.530115565, 105.115533ms after stability testing.go:149: cluster: 2025-02-10T11:34:20.161Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.056095734 +0000 UTC m=+16.425000032 (1 nodes), 406ns from start of poll, 368.979105ms from cluster start. Timeout at 2025-02-10 11:34:20.161211244 +0000 UTC m=+16.530115565, 105.115533ms after stability testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: updating configuration: command=RemoveServer server-id=2ff23827-10c6-84b8-e7a4-acd51c6d28fc server-addr= servers="[{Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a} {Suffrage:Voter ID:server-dd42822a-5656-f993-ffac-3e11b416e675 Address:dd42822a-5656-f993-ffac-3e11b416e675}]" testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: updating configuration: command=RemoveServer server-id=2ff23827-10c6-84b8-e7a4-acd51c6d28fc server-addr= servers="[{Suffrage:Voter ID:e21603ea-911a-e69c-4c1b-44b4ff19324a Address:e21603ea-911a-e69c-4c1b-44b4ff19324a} {Suffrage:Voter ID:server-dd42822a-5656-f993-ffac-3e11b416e675 Address:dd42822a-5656-f993-ffac-3e11b416e675}]" testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: removed peer, stopping replication: peer=2ff23827-10c6-84b8-e7a4-acd51c6d28fc last-index=4 testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: removed peer, stopping replication: peer=2ff23827-10c6-84b8-e7a4-acd51c6d28fc last-index=4 testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: aborting pipeline replication: peer="{Voter 2ff23827-10c6-84b8-e7a4-acd51c6d28fc 2ff23827-10c6-84b8-e7a4-acd51c6d28fc}" testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: aborting pipeline replication: peer="{Voter 2ff23827-10c6-84b8-e7a4-acd51c6d28fc 2ff23827-10c6-84b8-e7a4-acd51c6d28fc}" testLoggerAdapter verbose: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: aborting pipeline replication: peer="{Voter server-dd42822a-5656-f993-ffac-3e11b416e675 dd42822a-5656-f993-ffac-3e11b416e675}" testing.go:149: e21603ea-911a-e69c-4c1b-44b4ff19324a: 2025-02-10T11:34:20.161Z [INFO] e21603ea-911a-e69c-4c1b-44b4ff19324a: aborting pipeline replication: peer="{Voter server-dd42822a-5656-f993-ffac-3e11b416e675 dd42822a-5656-f993-ffac-3e11b416e675}" --- PASS: TestRaft_ProtocolVersion_Upgrade_2_3 (0.48s) === RUN TestRaft_LeadershipTransferInProgress --- PASS: TestRaft_LeadershipTransferInProgress (0.00s) === RUN TestRaft_LeadershipTransferPickServer --- PASS: TestRaft_LeadershipTransferPickServer (0.00s) === RUN TestRaft_LeadershipTransfer testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.162Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 Address:a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44} {Suffrage:Voter ID:server-7572b4a9-ed33-44cd-89b5-751227669e8b Address:7572b4a9-ed33-44cd-89b5-751227669e8b} {Suffrage:Voter ID:server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca Address:e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}]" testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.162Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 Address:a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44} {Suffrage:Voter ID:server-7572b4a9-ed33-44cd-89b5-751227669e8b Address:7572b4a9-ed33-44cd-89b5-751227669e8b} {Suffrage:Voter ID:server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca Address:e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}]" testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.162Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering follower state: follower="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Follower]" leader= testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.162Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering follower state: follower="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Follower]" leader= testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.163Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 Address:a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44} {Suffrage:Voter ID:server-7572b4a9-ed33-44cd-89b5-751227669e8b Address:7572b4a9-ed33-44cd-89b5-751227669e8b} {Suffrage:Voter ID:server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca Address:e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}]" testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.163Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 Address:a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44} {Suffrage:Voter ID:server-7572b4a9-ed33-44cd-89b5-751227669e8b Address:7572b4a9-ed33-44cd-89b5-751227669e8b} {Suffrage:Voter ID:server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca Address:e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}]" testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.163Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: entering follower state: follower="Node at 7572b4a9-ed33-44cd-89b5-751227669e8b [Follower]" leader= testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.163Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: entering follower state: follower="Node at 7572b4a9-ed33-44cd-89b5-751227669e8b [Follower]" leader= testLoggerAdapter verbose: server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: 2025-02-10T11:34:20.163Z [INFO] server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 Address:a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44} {Suffrage:Voter ID:server-7572b4a9-ed33-44cd-89b5-751227669e8b Address:7572b4a9-ed33-44cd-89b5-751227669e8b} {Suffrage:Voter ID:server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca Address:e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}]" testing.go:149: server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: 2025-02-10T11:34:20.163Z [INFO] server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 Address:a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44} {Suffrage:Voter ID:server-7572b4a9-ed33-44cd-89b5-751227669e8b Address:7572b4a9-ed33-44cd-89b5-751227669e8b} {Suffrage:Voter ID:server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca Address:e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.163Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: 2025-02-10T11:34:20.163Z [INFO] server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: entering follower state: follower="Node at e32b7316-c8b4-fd9c-7930-d8a3dbb037ca [Follower]" leader= testing.go:149: cluster: 2025-02-10T11:34:20.163Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: 2025-02-10T11:34:20.163Z [INFO] server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca: entering follower state: follower="Node at e32b7316-c8b4-fd9c-7930-d8a3dbb037ca [Follower]" leader= testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [WARN] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [WARN] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering candidate state: node="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Candidate]" term=2 testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering candidate state: node="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Candidate]" term=2 testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: election won: tally=2 testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: election won: tally=2 testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering leader state: leader="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Leader]" testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering leader state: leader="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Leader]" testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: added peer, starting replication: peer=server-7572b4a9-ed33-44cd-89b5-751227669e8b testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: added peer, starting replication: peer=server-7572b4a9-ed33-44cd-89b5-751227669e8b testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: added peer, starting replication: peer=server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: added peer, starting replication: peer=server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: pipelining replication: peer="{Voter server-7572b4a9-ed33-44cd-89b5-751227669e8b 7572b4a9-ed33-44cd-89b5-751227669e8b}" testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: pipelining replication: peer="{Voter server-7572b4a9-ed33-44cd-89b5-751227669e8b 7572b4a9-ed33-44cd-89b5-751227669e8b}" testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: pipelining replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.226Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: pipelining replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.331Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.226413377 +0000 UTC m=+16.595317684 (1 nodes), 63.104608ms from start of poll, 63.662272ms from cluster start. Timeout at 2025-02-10 11:34:20.331487019 +0000 UTC m=+16.700391315, 105.073631ms after stability testing.go:149: cluster: 2025-02-10T11:34:20.331Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.226413377 +0000 UTC m=+16.595317684 (1 nodes), 63.104608ms from start of poll, 63.662272ms from cluster start. Timeout at 2025-02-10 11:34:20.331487019 +0000 UTC m=+16.700391315, 105.073631ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.331Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:20.331Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.436Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.33155468 +0000 UTC m=+16.700458980 (1 nodes), 471ns from start of poll, 168.803568ms from cluster start. Timeout at 2025-02-10 11:34:20.436603224 +0000 UTC m=+16.805507518, 105.048538ms after stability testing.go:149: cluster: 2025-02-10T11:34:20.436Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.33155468 +0000 UTC m=+16.700458980 (1 nodes), 471ns from start of poll, 168.803568ms from cluster start. Timeout at 2025-02-10 11:34:20.436603224 +0000 UTC m=+16.805507518, 105.048538ms after stability testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: entering candidate state: node="Node at 7572b4a9-ed33-44cd-89b5-751227669e8b [Candidate]" term=3 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.436Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: entering candidate state: node="Node at 7572b4a9-ed33-44cd-89b5-751227669e8b [Candidate]" term=3 testing.go:149: cluster: 2025-02-10T11:34:20.436Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.436Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering follower state: follower="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Follower]" leader= testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: election won: tally=2 testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.436Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: entering follower state: follower="Node at a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 [Follower]" leader= testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.436Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: aborting pipeline replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: election won: tally=2 testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: entering leader state: leader="Node at 7572b4a9-ed33-44cd-89b5-751227669e8b [Leader]" testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.436Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: aborting pipeline replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testLoggerAdapter verbose: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.436Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: aborting pipeline replication: peer="{Voter server-7572b4a9-ed33-44cd-89b5-751227669e8b 7572b4a9-ed33-44cd-89b5-751227669e8b}" testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: entering leader state: leader="Node at 7572b4a9-ed33-44cd-89b5-751227669e8b [Leader]" testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: added peer, starting replication: peer=server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: added peer, starting replication: peer=server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: added peer, starting replication: peer=server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.436Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: added peer, starting replication: peer=server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.437Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: pipelining replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testing.go:149: server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: 2025-02-10T11:34:20.436Z [INFO] server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44: aborting pipeline replication: peer="{Voter server-7572b4a9-ed33-44cd-89b5-751227669e8b 7572b4a9-ed33-44cd-89b5-751227669e8b}" testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.437Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: pipelining replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.437Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: pipelining replication: peer="{Voter server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44}" testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.437Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: pipelining replication: peer="{Voter server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.542Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.436910549 +0000 UTC m=+16.805814867 (1 nodes), 73.749µs from start of poll, 274.159455ms from cluster start. Timeout at 2025-02-10 11:34:20.541982878 +0000 UTC m=+16.910887173, 105.072306ms after stability testing.go:149: cluster: 2025-02-10T11:34:20.542Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.436910549 +0000 UTC m=+16.805814867 (1 nodes), 73.749µs from start of poll, 274.159455ms from cluster start. Timeout at 2025-02-10 11:34:20.541982878 +0000 UTC m=+16.910887173, 105.072306ms after stability testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.542Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: aborting pipeline replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.542Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: aborting pipeline replication: peer="{Voter server-e32b7316-c8b4-fd9c-7930-d8a3dbb037ca e32b7316-c8b4-fd9c-7930-d8a3dbb037ca}" testLoggerAdapter verbose: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.542Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: aborting pipeline replication: peer="{Voter server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44}" testing.go:149: server-7572b4a9-ed33-44cd-89b5-751227669e8b: 2025-02-10T11:34:20.542Z [INFO] server-7572b4a9-ed33-44cd-89b5-751227669e8b: aborting pipeline replication: peer="{Voter server-a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44 a5df6a28-fb4a-d9d3-d1f3-619ff0b2dc44}" --- PASS: TestRaft_LeadershipTransfer (0.38s) === RUN TestRaft_LeadershipTransferWithOneNode testLoggerAdapter verbose: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.542Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 Address:1ef26271-7495-7c2b-3ae7-f8b5ed3012a2}]" testing.go:149: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.542Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 Address:1ef26271-7495-7c2b-3ae7-f8b5ed3012a2}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.543Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:20.543Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.543Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: entering follower state: follower="Node at 1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 [Follower]" leader= testing.go:149: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.543Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: entering follower state: follower="Node at 1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 [Follower]" leader= testLoggerAdapter verbose: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [WARN] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [WARN] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: entering candidate state: node="Node at 1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 [Candidate]" term=2 testing.go:149: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: entering candidate state: node="Node at 1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 [Candidate]" term=2 testLoggerAdapter verbose: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: election won: tally=1 testing.go:149: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: election won: tally=1 testLoggerAdapter verbose: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: entering leader state: leader="Node at 1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 [Leader]" testing.go:149: server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: 2025-02-10T11:34:20.642Z [INFO] server-1ef26271-7495-7c2b-3ae7-f8b5ed3012a2: entering leader state: leader="Node at 1ef26271-7495-7c2b-3ae7-f8b5ed3012a2 [Leader]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.747Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.642236096 +0000 UTC m=+17.011140396 (1 nodes), 99.21222ms from start of poll, 99.372209ms from cluster start. Timeout at 2025-02-10 11:34:20.747313529 +0000 UTC m=+17.116217835, 105.077439ms after stability testing.go:149: cluster: 2025-02-10T11:34:20.747Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.642236096 +0000 UTC m=+17.011140396 (1 nodes), 99.21222ms from start of poll, 99.372209ms from cluster start. Timeout at 2025-02-10 11:34:20.747313529 +0000 UTC m=+17.116217835, 105.077439ms after stability --- PASS: TestRaft_LeadershipTransferWithOneNode (0.20s) === RUN TestRaft_LeadershipTransferWithSevenNodes testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.748Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.748Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.748Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering follower state: follower="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Follower]" leader= testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.748Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering follower state: follower="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Follower]" leader= testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:20.748Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:20.748Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:20.748Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: entering follower state: follower="Node at 25a854f9-f140-ed8d-2d3a-161f77156e27 [Follower]" leader= testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:20.748Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: entering follower state: follower="Node at 25a854f9-f140-ed8d-2d3a-161f77156e27 [Follower]" leader= testLoggerAdapter verbose: server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: 2025-02-10T11:34:20.748Z [INFO] server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testing.go:149: server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: 2025-02-10T11:34:20.748Z [INFO] server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testLoggerAdapter verbose: server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: 2025-02-10T11:34:20.749Z [INFO] server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: entering follower state: follower="Node at dc67c5a0-ce21-9e8b-2780-aa037bb68e8b [Follower]" leader= testing.go:149: server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: 2025-02-10T11:34:20.749Z [INFO] server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b: entering follower state: follower="Node at dc67c5a0-ce21-9e8b-2780-aa037bb68e8b [Follower]" leader= testLoggerAdapter verbose: server-12b32eb4-d484-8f08-39d8-de6dca092171: 2025-02-10T11:34:20.749Z [INFO] server-12b32eb4-d484-8f08-39d8-de6dca092171: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testing.go:149: server-12b32eb4-d484-8f08-39d8-de6dca092171: 2025-02-10T11:34:20.749Z [INFO] server-12b32eb4-d484-8f08-39d8-de6dca092171: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testLoggerAdapter verbose: server-12b32eb4-d484-8f08-39d8-de6dca092171: 2025-02-10T11:34:20.749Z [INFO] server-12b32eb4-d484-8f08-39d8-de6dca092171: entering follower state: follower="Node at 12b32eb4-d484-8f08-39d8-de6dca092171 [Follower]" leader= testing.go:149: server-12b32eb4-d484-8f08-39d8-de6dca092171: 2025-02-10T11:34:20.749Z [INFO] server-12b32eb4-d484-8f08-39d8-de6dca092171: entering follower state: follower="Node at 12b32eb4-d484-8f08-39d8-de6dca092171 [Follower]" leader= testLoggerAdapter verbose: server-625613d0-9b59-45be-eb90-2d86fe02a2ef: 2025-02-10T11:34:20.749Z [INFO] server-625613d0-9b59-45be-eb90-2d86fe02a2ef: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testing.go:149: server-625613d0-9b59-45be-eb90-2d86fe02a2ef: 2025-02-10T11:34:20.749Z [INFO] server-625613d0-9b59-45be-eb90-2d86fe02a2ef: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testLoggerAdapter verbose: server-625613d0-9b59-45be-eb90-2d86fe02a2ef: 2025-02-10T11:34:20.749Z [INFO] server-625613d0-9b59-45be-eb90-2d86fe02a2ef: entering follower state: follower="Node at 625613d0-9b59-45be-eb90-2d86fe02a2ef [Follower]" leader= testing.go:149: server-625613d0-9b59-45be-eb90-2d86fe02a2ef: 2025-02-10T11:34:20.749Z [INFO] server-625613d0-9b59-45be-eb90-2d86fe02a2ef: entering follower state: follower="Node at 625613d0-9b59-45be-eb90-2d86fe02a2ef [Follower]" leader= testLoggerAdapter verbose: server-440806d4-301c-3044-e2a4-a9fd8537f71e: 2025-02-10T11:34:20.749Z [INFO] server-440806d4-301c-3044-e2a4-a9fd8537f71e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testing.go:149: server-440806d4-301c-3044-e2a4-a9fd8537f71e: 2025-02-10T11:34:20.749Z [INFO] server-440806d4-301c-3044-e2a4-a9fd8537f71e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testLoggerAdapter verbose: server-440806d4-301c-3044-e2a4-a9fd8537f71e: 2025-02-10T11:34:20.749Z [INFO] server-440806d4-301c-3044-e2a4-a9fd8537f71e: entering follower state: follower="Node at 440806d4-301c-3044-e2a4-a9fd8537f71e [Follower]" leader= testing.go:149: server-440806d4-301c-3044-e2a4-a9fd8537f71e: 2025-02-10T11:34:20.749Z [INFO] server-440806d4-301c-3044-e2a4-a9fd8537f71e: entering follower state: follower="Node at 440806d4-301c-3044-e2a4-a9fd8537f71e [Follower]" leader= testLoggerAdapter verbose: server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: 2025-02-10T11:34:20.749Z [INFO] server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testing.go:149: server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: 2025-02-10T11:34:20.749Z [INFO] server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b11175b4-c9eb-f41e-51a0-11096ad0cfea Address:b11175b4-c9eb-f41e-51a0-11096ad0cfea} {Suffrage:Voter ID:server-25a854f9-f140-ed8d-2d3a-161f77156e27 Address:25a854f9-f140-ed8d-2d3a-161f77156e27} {Suffrage:Voter ID:server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b Address:dc67c5a0-ce21-9e8b-2780-aa037bb68e8b} {Suffrage:Voter ID:server-12b32eb4-d484-8f08-39d8-de6dca092171 Address:12b32eb4-d484-8f08-39d8-de6dca092171} {Suffrage:Voter ID:server-625613d0-9b59-45be-eb90-2d86fe02a2ef Address:625613d0-9b59-45be-eb90-2d86fe02a2ef} {Suffrage:Voter ID:server-440806d4-301c-3044-e2a4-a9fd8537f71e Address:440806d4-301c-3044-e2a4-a9fd8537f71e} {Suffrage:Voter ID:server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 Address:f4651c05-e7f2-0754-efb2-4b34bbcd6345}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.749Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:20.749Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: 2025-02-10T11:34:20.749Z [INFO] server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: entering follower state: follower="Node at f4651c05-e7f2-0754-efb2-4b34bbcd6345 [Follower]" leader= testing.go:149: server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: 2025-02-10T11:34:20.749Z [INFO] server-f4651c05-e7f2-0754-efb2-4b34bbcd6345: entering follower state: follower="Node at f4651c05-e7f2-0754-efb2-4b34bbcd6345 [Follower]" leader= testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [WARN] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [WARN] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering candidate state: node="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Candidate]" term=2 testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering candidate state: node="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Candidate]" term=2 testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: election won: tally=4 testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: election won: tally=4 testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering leader state: leader="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Leader]" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering leader state: leader="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Leader]" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-25a854f9-f140-ed8d-2d3a-161f77156e27 testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-25a854f9-f140-ed8d-2d3a-161f77156e27 testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-12b32eb4-d484-8f08-39d8-de6dca092171 testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-12b32eb4-d484-8f08-39d8-de6dca092171 testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-625613d0-9b59-45be-eb90-2d86fe02a2ef testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-625613d0-9b59-45be-eb90-2d86fe02a2ef testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-25a854f9-f140-ed8d-2d3a-161f77156e27 25a854f9-f140-ed8d-2d3a-161f77156e27}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-25a854f9-f140-ed8d-2d3a-161f77156e27 25a854f9-f140-ed8d-2d3a-161f77156e27}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-440806d4-301c-3044-e2a4-a9fd8537f71e testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-440806d4-301c-3044-e2a4-a9fd8537f71e testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.811Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.811Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: added peer, starting replication: peer=server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.811Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.811Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.810Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.811Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:20.811Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: pipelining replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.915Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.810827432 +0000 UTC m=+17.179731733 (1 nodes), 61.050514ms from start of poll, 62.320167ms from cluster start. Timeout at 2025-02-10 11:34:20.915926001 +0000 UTC m=+17.284830297, 105.098564ms after stability testing.go:149: cluster: 2025-02-10T11:34:20.915Z [INFO] stable state for Leader reached at 2025-02-10 11:34:20.810827432 +0000 UTC m=+17.179731733 (1 nodes), 61.050514ms from start of poll, 62.320167ms from cluster start. Timeout at 2025-02-10 11:34:20.915926001 +0000 UTC m=+17.284830297, 105.098564ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:20.915Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:20.915Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.021Z [INFO] stable state for Follower reached at 2025-02-10 11:34:20.91598905 +0000 UTC m=+17.284893351 (6 nodes), 1.13µs from start of poll, 167.481785ms from cluster start. Timeout at 2025-02-10 11:34:21.021050781 +0000 UTC m=+17.389955113, 105.061762ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.021Z [INFO] stable state for Follower reached at 2025-02-10 11:34:20.91598905 +0000 UTC m=+17.284893351 (6 nodes), 1.13µs from start of poll, 167.481785ms from cluster start. Timeout at 2025-02-10 11:34:21.021050781 +0000 UTC m=+17.389955113, 105.061762ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.021Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:21.021Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.126Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.021108931 +0000 UTC m=+17.390013232 (1 nodes), 504ns from start of poll, 272.601666ms from cluster start. Timeout at 2025-02-10 11:34:21.126184058 +0000 UTC m=+17.495088354, 105.075122ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.126Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.021108931 +0000 UTC m=+17.390013232 (1 nodes), 504ns from start of poll, 272.601666ms from cluster start. Timeout at 2025-02-10 11:34:21.126184058 +0000 UTC m=+17.495088354, 105.075122ms after stability testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: entering candidate state: node="Node at 25a854f9-f140-ed8d-2d3a-161f77156e27 [Candidate]" term=3 testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: entering candidate state: node="Node at 25a854f9-f140-ed8d-2d3a-161f77156e27 [Candidate]" term=3 testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering follower state: follower="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Follower]" leader= testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: election won: tally=4 testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: entering follower state: follower="Node at b11175b4-c9eb-f41e-51a0-11096ad0cfea [Follower]" leader= testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: election won: tally=4 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.126Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: entering leader state: leader="Node at 25a854f9-f140-ed8d-2d3a-161f77156e27 [Leader]" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testing.go:149: cluster: 2025-02-10T11:34:21.126Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: entering leader state: leader="Node at 25a854f9-f140-ed8d-2d3a-161f77156e27 [Leader]" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-b11175b4-c9eb-f41e-51a0-11096ad0cfea testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-b11175b4-c9eb-f41e-51a0-11096ad0cfea testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-12b32eb4-d484-8f08-39d8-de6dca092171 testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-12b32eb4-d484-8f08-39d8-de6dca092171 testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-625613d0-9b59-45be-eb90-2d86fe02a2ef testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-625613d0-9b59-45be-eb90-2d86fe02a2ef testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-440806d4-301c-3044-e2a4-a9fd8537f71e testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-440806d4-301c-3044-e2a4-a9fd8537f71e testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: added peer, starting replication: peer=server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-b11175b4-c9eb-f41e-51a0-11096ad0cfea b11175b4-c9eb-f41e-51a0-11096ad0cfea}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-b11175b4-c9eb-f41e-51a0-11096ad0cfea b11175b4-c9eb-f41e-51a0-11096ad0cfea}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-25a854f9-f140-ed8d-2d3a-161f77156e27 25a854f9-f140-ed8d-2d3a-161f77156e27}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-25a854f9-f140-ed8d-2d3a-161f77156e27 25a854f9-f140-ed8d-2d3a-161f77156e27}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testLoggerAdapter verbose: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" testing.go:149: server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: 2025-02-10T11:34:21.126Z [INFO] server-b11175b4-c9eb-f41e-51a0-11096ad0cfea: aborting pipeline replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.126Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: pipelining replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.231Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.126416149 +0000 UTC m=+17.495320448 (1 nodes), 26.702µs from start of poll, 377.908882ms from cluster start. Timeout at 2025-02-10 11:34:21.231453139 +0000 UTC m=+17.600357435, 105.036987ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.231Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.126416149 +0000 UTC m=+17.495320448 (1 nodes), 26.702µs from start of poll, 377.908882ms from cluster start. Timeout at 2025-02-10 11:34:21.231453139 +0000 UTC m=+17.600357435, 105.036987ms after stability testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-12b32eb4-d484-8f08-39d8-de6dca092171 12b32eb4-d484-8f08-39d8-de6dca092171}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-440806d4-301c-3044-e2a4-a9fd8537f71e 440806d4-301c-3044-e2a4-a9fd8537f71e}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-625613d0-9b59-45be-eb90-2d86fe02a2ef 625613d0-9b59-45be-eb90-2d86fe02a2ef}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-b11175b4-c9eb-f41e-51a0-11096ad0cfea b11175b4-c9eb-f41e-51a0-11096ad0cfea}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-b11175b4-c9eb-f41e-51a0-11096ad0cfea b11175b4-c9eb-f41e-51a0-11096ad0cfea}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-f4651c05-e7f2-0754-efb2-4b34bbcd6345 f4651c05-e7f2-0754-efb2-4b34bbcd6345}" testLoggerAdapter verbose: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" testing.go:149: server-25a854f9-f140-ed8d-2d3a-161f77156e27: 2025-02-10T11:34:21.231Z [INFO] server-25a854f9-f140-ed8d-2d3a-161f77156e27: aborting pipeline replication: peer="{Voter server-dc67c5a0-ce21-9e8b-2780-aa037bb68e8b dc67c5a0-ce21-9e8b-2780-aa037bb68e8b}" --- PASS: TestRaft_LeadershipTransferWithSevenNodes (0.49s) === RUN TestRaft_LeadershipTransferToInvalidID testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.234Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 Address:75a215e4-4b2e-9bbf-83b7-221cc9fa82c6} {Suffrage:Voter ID:server-075183af-98dd-7927-699f-53ba4d3eeb7a Address:075183af-98dd-7927-699f-53ba4d3eeb7a} {Suffrage:Voter ID:server-73ce0480-bae7-fa62-3215-7c9d150b1cdf Address:73ce0480-bae7-fa62-3215-7c9d150b1cdf}]" testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.234Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 Address:75a215e4-4b2e-9bbf-83b7-221cc9fa82c6} {Suffrage:Voter ID:server-075183af-98dd-7927-699f-53ba4d3eeb7a Address:075183af-98dd-7927-699f-53ba4d3eeb7a} {Suffrage:Voter ID:server-73ce0480-bae7-fa62-3215-7c9d150b1cdf Address:73ce0480-bae7-fa62-3215-7c9d150b1cdf}]" testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.234Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: entering follower state: follower="Node at 75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 [Follower]" leader= testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.234Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: entering follower state: follower="Node at 75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 [Follower]" leader= testLoggerAdapter verbose: server-075183af-98dd-7927-699f-53ba4d3eeb7a: 2025-02-10T11:34:21.234Z [INFO] server-075183af-98dd-7927-699f-53ba4d3eeb7a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 Address:75a215e4-4b2e-9bbf-83b7-221cc9fa82c6} {Suffrage:Voter ID:server-075183af-98dd-7927-699f-53ba4d3eeb7a Address:075183af-98dd-7927-699f-53ba4d3eeb7a} {Suffrage:Voter ID:server-73ce0480-bae7-fa62-3215-7c9d150b1cdf Address:73ce0480-bae7-fa62-3215-7c9d150b1cdf}]" testing.go:149: server-075183af-98dd-7927-699f-53ba4d3eeb7a: 2025-02-10T11:34:21.234Z [INFO] server-075183af-98dd-7927-699f-53ba4d3eeb7a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 Address:75a215e4-4b2e-9bbf-83b7-221cc9fa82c6} {Suffrage:Voter ID:server-075183af-98dd-7927-699f-53ba4d3eeb7a Address:075183af-98dd-7927-699f-53ba4d3eeb7a} {Suffrage:Voter ID:server-73ce0480-bae7-fa62-3215-7c9d150b1cdf Address:73ce0480-bae7-fa62-3215-7c9d150b1cdf}]" testLoggerAdapter verbose: server-075183af-98dd-7927-699f-53ba4d3eeb7a: 2025-02-10T11:34:21.234Z [INFO] server-075183af-98dd-7927-699f-53ba4d3eeb7a: entering follower state: follower="Node at 075183af-98dd-7927-699f-53ba4d3eeb7a [Follower]" leader= testing.go:149: server-075183af-98dd-7927-699f-53ba4d3eeb7a: 2025-02-10T11:34:21.234Z [INFO] server-075183af-98dd-7927-699f-53ba4d3eeb7a: entering follower state: follower="Node at 075183af-98dd-7927-699f-53ba4d3eeb7a [Follower]" leader= testLoggerAdapter verbose: server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: 2025-02-10T11:34:21.234Z [INFO] server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 Address:75a215e4-4b2e-9bbf-83b7-221cc9fa82c6} {Suffrage:Voter ID:server-075183af-98dd-7927-699f-53ba4d3eeb7a Address:075183af-98dd-7927-699f-53ba4d3eeb7a} {Suffrage:Voter ID:server-73ce0480-bae7-fa62-3215-7c9d150b1cdf Address:73ce0480-bae7-fa62-3215-7c9d150b1cdf}]" testing.go:149: server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: 2025-02-10T11:34:21.234Z [INFO] server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 Address:75a215e4-4b2e-9bbf-83b7-221cc9fa82c6} {Suffrage:Voter ID:server-075183af-98dd-7927-699f-53ba4d3eeb7a Address:075183af-98dd-7927-699f-53ba4d3eeb7a} {Suffrage:Voter ID:server-73ce0480-bae7-fa62-3215-7c9d150b1cdf Address:73ce0480-bae7-fa62-3215-7c9d150b1cdf}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.235Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:21.235Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: 2025-02-10T11:34:21.235Z [INFO] server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: entering follower state: follower="Node at 73ce0480-bae7-fa62-3215-7c9d150b1cdf [Follower]" leader= testing.go:149: server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: 2025-02-10T11:34:21.235Z [INFO] server-73ce0480-bae7-fa62-3215-7c9d150b1cdf: entering follower state: follower="Node at 73ce0480-bae7-fa62-3215-7c9d150b1cdf [Follower]" leader= testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.290Z [WARN] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.290Z [WARN] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.290Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: entering candidate state: node="Node at 75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 [Candidate]" term=2 testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.290Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: entering candidate state: node="Node at 75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 [Candidate]" term=2 testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: election won: tally=2 testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: election won: tally=2 testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: entering leader state: leader="Node at 75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 [Leader]" testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: entering leader state: leader="Node at 75a215e4-4b2e-9bbf-83b7-221cc9fa82c6 [Leader]" testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: added peer, starting replication: peer=server-075183af-98dd-7927-699f-53ba4d3eeb7a testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: added peer, starting replication: peer=server-075183af-98dd-7927-699f-53ba4d3eeb7a testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: added peer, starting replication: peer=server-73ce0480-bae7-fa62-3215-7c9d150b1cdf testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: added peer, starting replication: peer=server-73ce0480-bae7-fa62-3215-7c9d150b1cdf testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: pipelining replication: peer="{Voter server-73ce0480-bae7-fa62-3215-7c9d150b1cdf 73ce0480-bae7-fa62-3215-7c9d150b1cdf}" testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: pipelining replication: peer="{Voter server-73ce0480-bae7-fa62-3215-7c9d150b1cdf 73ce0480-bae7-fa62-3215-7c9d150b1cdf}" testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: pipelining replication: peer="{Voter server-075183af-98dd-7927-699f-53ba4d3eeb7a 075183af-98dd-7927-699f-53ba4d3eeb7a}" testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.291Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: pipelining replication: peer="{Voter server-075183af-98dd-7927-699f-53ba4d3eeb7a 075183af-98dd-7927-699f-53ba4d3eeb7a}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.396Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.291156812 +0000 UTC m=+17.660061113 (1 nodes), 56.105269ms from start of poll, 56.482589ms from cluster start. Timeout at 2025-02-10 11:34:21.396275433 +0000 UTC m=+17.765179782, 105.118669ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.396Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.291156812 +0000 UTC m=+17.660061113 (1 nodes), 56.105269ms from start of poll, 56.482589ms from cluster start. Timeout at 2025-02-10 11:34:21.396275433 +0000 UTC m=+17.765179782, 105.118669ms after stability testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.396Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: aborting pipeline replication: peer="{Voter server-73ce0480-bae7-fa62-3215-7c9d150b1cdf 73ce0480-bae7-fa62-3215-7c9d150b1cdf}" testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.396Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: aborting pipeline replication: peer="{Voter server-73ce0480-bae7-fa62-3215-7c9d150b1cdf 73ce0480-bae7-fa62-3215-7c9d150b1cdf}" testLoggerAdapter verbose: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.396Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: aborting pipeline replication: peer="{Voter server-075183af-98dd-7927-699f-53ba4d3eeb7a 075183af-98dd-7927-699f-53ba4d3eeb7a}" testing.go:149: server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: 2025-02-10T11:34:21.396Z [INFO] server-75a215e4-4b2e-9bbf-83b7-221cc9fa82c6: aborting pipeline replication: peer="{Voter server-075183af-98dd-7927-699f-53ba4d3eeb7a 075183af-98dd-7927-699f-53ba4d3eeb7a}" --- PASS: TestRaft_LeadershipTransferToInvalidID (0.16s) === RUN TestRaft_LeadershipTransferToInvalidAddress testLoggerAdapter verbose: server-8115d111-090f-b0b3-7491-04e0e0bd226a: 2025-02-10T11:34:21.398Z [INFO] server-8115d111-090f-b0b3-7491-04e0e0bd226a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8115d111-090f-b0b3-7491-04e0e0bd226a Address:8115d111-090f-b0b3-7491-04e0e0bd226a} {Suffrage:Voter ID:server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25 Address:29c44097-7ef6-4ac1-a9b4-2aa1521faa25} {Suffrage:Voter ID:server-c7f35895-4b95-1222-0a53-f702213edfa1 Address:c7f35895-4b95-1222-0a53-f702213edfa1}]" testing.go:149: server-8115d111-090f-b0b3-7491-04e0e0bd226a: 2025-02-10T11:34:21.398Z [INFO] server-8115d111-090f-b0b3-7491-04e0e0bd226a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8115d111-090f-b0b3-7491-04e0e0bd226a Address:8115d111-090f-b0b3-7491-04e0e0bd226a} {Suffrage:Voter ID:server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25 Address:29c44097-7ef6-4ac1-a9b4-2aa1521faa25} {Suffrage:Voter ID:server-c7f35895-4b95-1222-0a53-f702213edfa1 Address:c7f35895-4b95-1222-0a53-f702213edfa1}]" testLoggerAdapter verbose: server-8115d111-090f-b0b3-7491-04e0e0bd226a: 2025-02-10T11:34:21.398Z [INFO] server-8115d111-090f-b0b3-7491-04e0e0bd226a: entering follower state: follower="Node at 8115d111-090f-b0b3-7491-04e0e0bd226a [Follower]" leader= testing.go:149: server-8115d111-090f-b0b3-7491-04e0e0bd226a: 2025-02-10T11:34:21.398Z [INFO] server-8115d111-090f-b0b3-7491-04e0e0bd226a: entering follower state: follower="Node at 8115d111-090f-b0b3-7491-04e0e0bd226a [Follower]" leader= testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.398Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8115d111-090f-b0b3-7491-04e0e0bd226a Address:8115d111-090f-b0b3-7491-04e0e0bd226a} {Suffrage:Voter ID:server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25 Address:29c44097-7ef6-4ac1-a9b4-2aa1521faa25} {Suffrage:Voter ID:server-c7f35895-4b95-1222-0a53-f702213edfa1 Address:c7f35895-4b95-1222-0a53-f702213edfa1}]" testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.398Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8115d111-090f-b0b3-7491-04e0e0bd226a Address:8115d111-090f-b0b3-7491-04e0e0bd226a} {Suffrage:Voter ID:server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25 Address:29c44097-7ef6-4ac1-a9b4-2aa1521faa25} {Suffrage:Voter ID:server-c7f35895-4b95-1222-0a53-f702213edfa1 Address:c7f35895-4b95-1222-0a53-f702213edfa1}]" testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.398Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: entering follower state: follower="Node at 29c44097-7ef6-4ac1-a9b4-2aa1521faa25 [Follower]" leader= testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.398Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: entering follower state: follower="Node at 29c44097-7ef6-4ac1-a9b4-2aa1521faa25 [Follower]" leader= testLoggerAdapter verbose: server-c7f35895-4b95-1222-0a53-f702213edfa1: 2025-02-10T11:34:21.398Z [INFO] server-c7f35895-4b95-1222-0a53-f702213edfa1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8115d111-090f-b0b3-7491-04e0e0bd226a Address:8115d111-090f-b0b3-7491-04e0e0bd226a} {Suffrage:Voter ID:server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25 Address:29c44097-7ef6-4ac1-a9b4-2aa1521faa25} {Suffrage:Voter ID:server-c7f35895-4b95-1222-0a53-f702213edfa1 Address:c7f35895-4b95-1222-0a53-f702213edfa1}]" testing.go:149: server-c7f35895-4b95-1222-0a53-f702213edfa1: 2025-02-10T11:34:21.398Z [INFO] server-c7f35895-4b95-1222-0a53-f702213edfa1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8115d111-090f-b0b3-7491-04e0e0bd226a Address:8115d111-090f-b0b3-7491-04e0e0bd226a} {Suffrage:Voter ID:server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25 Address:29c44097-7ef6-4ac1-a9b4-2aa1521faa25} {Suffrage:Voter ID:server-c7f35895-4b95-1222-0a53-f702213edfa1 Address:c7f35895-4b95-1222-0a53-f702213edfa1}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.398Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-c7f35895-4b95-1222-0a53-f702213edfa1: 2025-02-10T11:34:21.398Z [INFO] server-c7f35895-4b95-1222-0a53-f702213edfa1: entering follower state: follower="Node at c7f35895-4b95-1222-0a53-f702213edfa1 [Follower]" leader= testing.go:149: cluster: 2025-02-10T11:34:21.398Z [INFO] starting stability test: raft-state=Follower testing.go:149: server-c7f35895-4b95-1222-0a53-f702213edfa1: 2025-02-10T11:34:21.398Z [INFO] server-c7f35895-4b95-1222-0a53-f702213edfa1: entering follower state: follower="Node at c7f35895-4b95-1222-0a53-f702213edfa1 [Follower]" leader= testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [WARN] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [WARN] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: entering candidate state: node="Node at 29c44097-7ef6-4ac1-a9b4-2aa1521faa25 [Candidate]" term=2 testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: entering candidate state: node="Node at 29c44097-7ef6-4ac1-a9b4-2aa1521faa25 [Candidate]" term=2 testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: election won: tally=2 testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: election won: tally=2 testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: entering leader state: leader="Node at 29c44097-7ef6-4ac1-a9b4-2aa1521faa25 [Leader]" testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: entering leader state: leader="Node at 29c44097-7ef6-4ac1-a9b4-2aa1521faa25 [Leader]" testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: added peer, starting replication: peer=server-8115d111-090f-b0b3-7491-04e0e0bd226a testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: added peer, starting replication: peer=server-8115d111-090f-b0b3-7491-04e0e0bd226a testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: added peer, starting replication: peer=server-c7f35895-4b95-1222-0a53-f702213edfa1 testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: added peer, starting replication: peer=server-c7f35895-4b95-1222-0a53-f702213edfa1 testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: pipelining replication: peer="{Voter server-8115d111-090f-b0b3-7491-04e0e0bd226a 8115d111-090f-b0b3-7491-04e0e0bd226a}" testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: pipelining replication: peer="{Voter server-8115d111-090f-b0b3-7491-04e0e0bd226a 8115d111-090f-b0b3-7491-04e0e0bd226a}" testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: pipelining replication: peer="{Voter server-c7f35895-4b95-1222-0a53-f702213edfa1 c7f35895-4b95-1222-0a53-f702213edfa1}" testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.456Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: pipelining replication: peer="{Voter server-c7f35895-4b95-1222-0a53-f702213edfa1 c7f35895-4b95-1222-0a53-f702213edfa1}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.561Z [INFO] stable state for Follower reached at 2025-02-10 11:34:21.456877254 +0000 UTC m=+17.825781553 (2 nodes), 58.36095ms from start of poll, 58.741377ms from cluster start. Timeout at 2025-02-10 11:34:21.561950114 +0000 UTC m=+17.930854413, 105.07286ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.561Z [INFO] stable state for Follower reached at 2025-02-10 11:34:21.456877254 +0000 UTC m=+17.825781553 (2 nodes), 58.36095ms from start of poll, 58.741377ms from cluster start. Timeout at 2025-02-10 11:34:21.561950114 +0000 UTC m=+17.930854413, 105.07286ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.562Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:21.562Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.667Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.562037597 +0000 UTC m=+17.930941897 (1 nodes), 445ns from start of poll, 163.901721ms from cluster start. Timeout at 2025-02-10 11:34:21.667137931 +0000 UTC m=+18.036042284, 105.100387ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.667Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.562037597 +0000 UTC m=+17.930941897 (1 nodes), 445ns from start of poll, 163.901721ms from cluster start. Timeout at 2025-02-10 11:34:21.667137931 +0000 UTC m=+18.036042284, 105.100387ms after stability testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.667Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: aborting pipeline replication: peer="{Voter server-c7f35895-4b95-1222-0a53-f702213edfa1 c7f35895-4b95-1222-0a53-f702213edfa1}" testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.667Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: aborting pipeline replication: peer="{Voter server-c7f35895-4b95-1222-0a53-f702213edfa1 c7f35895-4b95-1222-0a53-f702213edfa1}" testLoggerAdapter verbose: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.667Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: aborting pipeline replication: peer="{Voter server-8115d111-090f-b0b3-7491-04e0e0bd226a 8115d111-090f-b0b3-7491-04e0e0bd226a}" testing.go:149: server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: 2025-02-10T11:34:21.667Z [INFO] server-29c44097-7ef6-4ac1-a9b4-2aa1521faa25: aborting pipeline replication: peer="{Voter server-8115d111-090f-b0b3-7491-04e0e0bd226a 8115d111-090f-b0b3-7491-04e0e0bd226a}" --- PASS: TestRaft_LeadershipTransferToInvalidAddress (0.27s) === RUN TestRaft_LeadershipTransferToBehindServer testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.668Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff256bae-ee32-9bed-349f-6e0a606ba47b Address:ff256bae-ee32-9bed-349f-6e0a606ba47b} {Suffrage:Voter ID:server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 Address:c098f7e6-002b-ac67-5e71-e0df37edf1c0} {Suffrage:Voter ID:server-0ae438cc-47a6-00c4-a233-e8547944a17f Address:0ae438cc-47a6-00c4-a233-e8547944a17f}]" testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.668Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff256bae-ee32-9bed-349f-6e0a606ba47b Address:ff256bae-ee32-9bed-349f-6e0a606ba47b} {Suffrage:Voter ID:server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 Address:c098f7e6-002b-ac67-5e71-e0df37edf1c0} {Suffrage:Voter ID:server-0ae438cc-47a6-00c4-a233-e8547944a17f Address:0ae438cc-47a6-00c4-a233-e8547944a17f}]" testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.668Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: entering follower state: follower="Node at ff256bae-ee32-9bed-349f-6e0a606ba47b [Follower]" leader= testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.668Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: entering follower state: follower="Node at ff256bae-ee32-9bed-349f-6e0a606ba47b [Follower]" leader= testLoggerAdapter verbose: server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: 2025-02-10T11:34:21.668Z [INFO] server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff256bae-ee32-9bed-349f-6e0a606ba47b Address:ff256bae-ee32-9bed-349f-6e0a606ba47b} {Suffrage:Voter ID:server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 Address:c098f7e6-002b-ac67-5e71-e0df37edf1c0} {Suffrage:Voter ID:server-0ae438cc-47a6-00c4-a233-e8547944a17f Address:0ae438cc-47a6-00c4-a233-e8547944a17f}]" testing.go:149: server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: 2025-02-10T11:34:21.668Z [INFO] server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff256bae-ee32-9bed-349f-6e0a606ba47b Address:ff256bae-ee32-9bed-349f-6e0a606ba47b} {Suffrage:Voter ID:server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 Address:c098f7e6-002b-ac67-5e71-e0df37edf1c0} {Suffrage:Voter ID:server-0ae438cc-47a6-00c4-a233-e8547944a17f Address:0ae438cc-47a6-00c4-a233-e8547944a17f}]" testLoggerAdapter verbose: server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: 2025-02-10T11:34:21.668Z [INFO] server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: entering follower state: follower="Node at c098f7e6-002b-ac67-5e71-e0df37edf1c0 [Follower]" leader= testing.go:149: server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: 2025-02-10T11:34:21.668Z [INFO] server-c098f7e6-002b-ac67-5e71-e0df37edf1c0: entering follower state: follower="Node at c098f7e6-002b-ac67-5e71-e0df37edf1c0 [Follower]" leader= testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.668Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff256bae-ee32-9bed-349f-6e0a606ba47b Address:ff256bae-ee32-9bed-349f-6e0a606ba47b} {Suffrage:Voter ID:server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 Address:c098f7e6-002b-ac67-5e71-e0df37edf1c0} {Suffrage:Voter ID:server-0ae438cc-47a6-00c4-a233-e8547944a17f Address:0ae438cc-47a6-00c4-a233-e8547944a17f}]" testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.668Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff256bae-ee32-9bed-349f-6e0a606ba47b Address:ff256bae-ee32-9bed-349f-6e0a606ba47b} {Suffrage:Voter ID:server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 Address:c098f7e6-002b-ac67-5e71-e0df37edf1c0} {Suffrage:Voter ID:server-0ae438cc-47a6-00c4-a233-e8547944a17f Address:0ae438cc-47a6-00c4-a233-e8547944a17f}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.668Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:21.668Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.668Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering follower state: follower="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Follower]" leader= testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.668Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering follower state: follower="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Follower]" leader= testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [WARN] server-0ae438cc-47a6-00c4-a233-e8547944a17f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [WARN] server-0ae438cc-47a6-00c4-a233-e8547944a17f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering candidate state: node="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Candidate]" term=2 testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering candidate state: node="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Candidate]" term=2 testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: election won: tally=2 testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: election won: tally=2 testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering leader state: leader="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Leader]" testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering leader state: leader="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Leader]" testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: added peer, starting replication: peer=server-ff256bae-ee32-9bed-349f-6e0a606ba47b testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: added peer, starting replication: peer=server-ff256bae-ee32-9bed-349f-6e0a606ba47b testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: added peer, starting replication: peer=server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: added peer, starting replication: peer=server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: pipelining replication: peer="{Voter server-ff256bae-ee32-9bed-349f-6e0a606ba47b ff256bae-ee32-9bed-349f-6e0a606ba47b}" testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: pipelining replication: peer="{Voter server-ff256bae-ee32-9bed-349f-6e0a606ba47b ff256bae-ee32-9bed-349f-6e0a606ba47b}" testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: pipelining replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.728Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: pipelining replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.833Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.728608896 +0000 UTC m=+18.097513197 (1 nodes), 59.848473ms from start of poll, 60.193435ms from cluster start. Timeout at 2025-02-10 11:34:21.833682647 +0000 UTC m=+18.202586946, 105.073749ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.833Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.728608896 +0000 UTC m=+18.097513197 (1 nodes), 59.848473ms from start of poll, 60.193435ms from cluster start. Timeout at 2025-02-10 11:34:21.833682647 +0000 UTC m=+18.202586946, 105.073749ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.833Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:21.833Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.938Z [INFO] stable state for Follower reached at 2025-02-10 11:34:21.833744607 +0000 UTC m=+18.202648907 (2 nodes), 638ns from start of poll, 165.329145ms from cluster start. Timeout at 2025-02-10 11:34:21.938846097 +0000 UTC m=+18.307750450, 105.101543ms after stability testing.go:149: cluster: 2025-02-10T11:34:21.938Z [INFO] stable state for Follower reached at 2025-02-10 11:34:21.833744607 +0000 UTC m=+18.202648907 (2 nodes), 638ns from start of poll, 165.329145ms from cluster start. Timeout at 2025-02-10 11:34:21.938846097 +0000 UTC m=+18.307750450, 105.101543ms after stability testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: entering candidate state: node="Node at ff256bae-ee32-9bed-349f-6e0a606ba47b [Candidate]" term=3 testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: entering candidate state: node="Node at ff256bae-ee32-9bed-349f-6e0a606ba47b [Candidate]" term=3 testLoggerAdapter verbose: cluster: 2025-02-10T11:34:21.950Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.950Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering follower state: follower="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Follower]" leader= testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: election won: tally=2 testing.go:149: cluster: 2025-02-10T11:34:21.950Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.950Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: entering follower state: follower="Node at 0ae438cc-47a6-00c4-a233-e8547944a17f [Follower]" leader= testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.950Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: aborting pipeline replication: peer="{Voter server-ff256bae-ee32-9bed-349f-6e0a606ba47b ff256bae-ee32-9bed-349f-6e0a606ba47b}" testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: election won: tally=2 testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: entering leader state: leader="Node at ff256bae-ee32-9bed-349f-6e0a606ba47b [Leader]" testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.950Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: aborting pipeline replication: peer="{Voter server-ff256bae-ee32-9bed-349f-6e0a606ba47b ff256bae-ee32-9bed-349f-6e0a606ba47b}" testLoggerAdapter verbose: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.950Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: aborting pipeline replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: entering leader state: leader="Node at ff256bae-ee32-9bed-349f-6e0a606ba47b [Leader]" testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: added peer, starting replication: peer=server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: added peer, starting replication: peer=server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: added peer, starting replication: peer=server-0ae438cc-47a6-00c4-a233-e8547944a17f testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.950Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: added peer, starting replication: peer=server-0ae438cc-47a6-00c4-a233-e8547944a17f testing.go:149: server-0ae438cc-47a6-00c4-a233-e8547944a17f: 2025-02-10T11:34:21.950Z [INFO] server-0ae438cc-47a6-00c4-a233-e8547944a17f: aborting pipeline replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.951Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: pipelining replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.951Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: pipelining replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.951Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: pipelining replication: peer="{Voter server-0ae438cc-47a6-00c4-a233-e8547944a17f 0ae438cc-47a6-00c4-a233-e8547944a17f}" testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:21.951Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: pipelining replication: peer="{Voter server-0ae438cc-47a6-00c4-a233-e8547944a17f 0ae438cc-47a6-00c4-a233-e8547944a17f}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.055Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.950897352 +0000 UTC m=+18.319801652 (1 nodes), 78.172µs from start of poll, 282.48189ms from cluster start. Timeout at 2025-02-10 11:34:22.05596965 +0000 UTC m=+18.424873944, 105.072292ms after stability testing.go:149: cluster: 2025-02-10T11:34:22.055Z [INFO] stable state for Leader reached at 2025-02-10 11:34:21.950897352 +0000 UTC m=+18.319801652 (1 nodes), 78.172µs from start of poll, 282.48189ms from cluster start. Timeout at 2025-02-10 11:34:22.05596965 +0000 UTC m=+18.424873944, 105.072292ms after stability testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:22.056Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: aborting pipeline replication: peer="{Voter server-0ae438cc-47a6-00c4-a233-e8547944a17f 0ae438cc-47a6-00c4-a233-e8547944a17f}" testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:22.056Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: aborting pipeline replication: peer="{Voter server-0ae438cc-47a6-00c4-a233-e8547944a17f 0ae438cc-47a6-00c4-a233-e8547944a17f}" testLoggerAdapter verbose: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:22.056Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: aborting pipeline replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" testing.go:149: server-ff256bae-ee32-9bed-349f-6e0a606ba47b: 2025-02-10T11:34:22.056Z [INFO] server-ff256bae-ee32-9bed-349f-6e0a606ba47b: aborting pipeline replication: peer="{Voter server-c098f7e6-002b-ac67-5e71-e0df37edf1c0 c098f7e6-002b-ac67-5e71-e0df37edf1c0}" --- PASS: TestRaft_LeadershipTransferToBehindServer (0.39s) === RUN TestRaft_LeadershipTransferToItself testLoggerAdapter verbose: server-42198df6-ee40-1b10-5914-b94129e72bc0: 2025-02-10T11:34:22.057Z [INFO] server-42198df6-ee40-1b10-5914-b94129e72bc0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42198df6-ee40-1b10-5914-b94129e72bc0 Address:42198df6-ee40-1b10-5914-b94129e72bc0} {Suffrage:Voter ID:server-f286f4b5-df17-1e6c-cd26-acd57f83e12c Address:f286f4b5-df17-1e6c-cd26-acd57f83e12c} {Suffrage:Voter ID:server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a Address:0d4b481b-b09b-77da-b4ca-452aa42ccf2a}]" testing.go:149: server-42198df6-ee40-1b10-5914-b94129e72bc0: 2025-02-10T11:34:22.057Z [INFO] server-42198df6-ee40-1b10-5914-b94129e72bc0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42198df6-ee40-1b10-5914-b94129e72bc0 Address:42198df6-ee40-1b10-5914-b94129e72bc0} {Suffrage:Voter ID:server-f286f4b5-df17-1e6c-cd26-acd57f83e12c Address:f286f4b5-df17-1e6c-cd26-acd57f83e12c} {Suffrage:Voter ID:server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a Address:0d4b481b-b09b-77da-b4ca-452aa42ccf2a}]" testLoggerAdapter verbose: server-42198df6-ee40-1b10-5914-b94129e72bc0: 2025-02-10T11:34:22.057Z [INFO] server-42198df6-ee40-1b10-5914-b94129e72bc0: entering follower state: follower="Node at 42198df6-ee40-1b10-5914-b94129e72bc0 [Follower]" leader= testing.go:149: server-42198df6-ee40-1b10-5914-b94129e72bc0: 2025-02-10T11:34:22.057Z [INFO] server-42198df6-ee40-1b10-5914-b94129e72bc0: entering follower state: follower="Node at 42198df6-ee40-1b10-5914-b94129e72bc0 [Follower]" leader= testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.057Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42198df6-ee40-1b10-5914-b94129e72bc0 Address:42198df6-ee40-1b10-5914-b94129e72bc0} {Suffrage:Voter ID:server-f286f4b5-df17-1e6c-cd26-acd57f83e12c Address:f286f4b5-df17-1e6c-cd26-acd57f83e12c} {Suffrage:Voter ID:server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a Address:0d4b481b-b09b-77da-b4ca-452aa42ccf2a}]" testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.057Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42198df6-ee40-1b10-5914-b94129e72bc0 Address:42198df6-ee40-1b10-5914-b94129e72bc0} {Suffrage:Voter ID:server-f286f4b5-df17-1e6c-cd26-acd57f83e12c Address:f286f4b5-df17-1e6c-cd26-acd57f83e12c} {Suffrage:Voter ID:server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a Address:0d4b481b-b09b-77da-b4ca-452aa42ccf2a}]" testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.057Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: entering follower state: follower="Node at f286f4b5-df17-1e6c-cd26-acd57f83e12c [Follower]" leader= testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.057Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: entering follower state: follower="Node at f286f4b5-df17-1e6c-cd26-acd57f83e12c [Follower]" leader= testLoggerAdapter verbose: server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: 2025-02-10T11:34:22.057Z [INFO] server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42198df6-ee40-1b10-5914-b94129e72bc0 Address:42198df6-ee40-1b10-5914-b94129e72bc0} {Suffrage:Voter ID:server-f286f4b5-df17-1e6c-cd26-acd57f83e12c Address:f286f4b5-df17-1e6c-cd26-acd57f83e12c} {Suffrage:Voter ID:server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a Address:0d4b481b-b09b-77da-b4ca-452aa42ccf2a}]" testing.go:149: server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: 2025-02-10T11:34:22.057Z [INFO] server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42198df6-ee40-1b10-5914-b94129e72bc0 Address:42198df6-ee40-1b10-5914-b94129e72bc0} {Suffrage:Voter ID:server-f286f4b5-df17-1e6c-cd26-acd57f83e12c Address:f286f4b5-df17-1e6c-cd26-acd57f83e12c} {Suffrage:Voter ID:server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a Address:0d4b481b-b09b-77da-b4ca-452aa42ccf2a}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.057Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: 2025-02-10T11:34:22.057Z [INFO] server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: entering follower state: follower="Node at 0d4b481b-b09b-77da-b4ca-452aa42ccf2a [Follower]" leader= testing.go:149: cluster: 2025-02-10T11:34:22.057Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: 2025-02-10T11:34:22.057Z [INFO] server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a: entering follower state: follower="Node at 0d4b481b-b09b-77da-b4ca-452aa42ccf2a [Follower]" leader= testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.109Z [WARN] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.109Z [WARN] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.109Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: entering candidate state: node="Node at f286f4b5-df17-1e6c-cd26-acd57f83e12c [Candidate]" term=2 testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.109Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: entering candidate state: node="Node at f286f4b5-df17-1e6c-cd26-acd57f83e12c [Candidate]" term=2 testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: election won: tally=2 testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: election won: tally=2 testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: entering leader state: leader="Node at f286f4b5-df17-1e6c-cd26-acd57f83e12c [Leader]" testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: entering leader state: leader="Node at f286f4b5-df17-1e6c-cd26-acd57f83e12c [Leader]" testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: added peer, starting replication: peer=server-42198df6-ee40-1b10-5914-b94129e72bc0 testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: added peer, starting replication: peer=server-42198df6-ee40-1b10-5914-b94129e72bc0 testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: added peer, starting replication: peer=server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: added peer, starting replication: peer=server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: pipelining replication: peer="{Voter server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a 0d4b481b-b09b-77da-b4ca-452aa42ccf2a}" testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: pipelining replication: peer="{Voter server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a 0d4b481b-b09b-77da-b4ca-452aa42ccf2a}" testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: pipelining replication: peer="{Voter server-42198df6-ee40-1b10-5914-b94129e72bc0 42198df6-ee40-1b10-5914-b94129e72bc0}" testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.110Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: pipelining replication: peer="{Voter server-42198df6-ee40-1b10-5914-b94129e72bc0 42198df6-ee40-1b10-5914-b94129e72bc0}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.215Z [INFO] stable state for Leader reached at 2025-02-10 11:34:22.110100435 +0000 UTC m=+18.479004736 (1 nodes), 52.771579ms from start of poll, 53.146643ms from cluster start. Timeout at 2025-02-10 11:34:22.215172189 +0000 UTC m=+18.584076483, 105.071747ms after stability testing.go:149: cluster: 2025-02-10T11:34:22.215Z [INFO] stable state for Leader reached at 2025-02-10 11:34:22.110100435 +0000 UTC m=+18.479004736 (1 nodes), 52.771579ms from start of poll, 53.146643ms from cluster start. Timeout at 2025-02-10 11:34:22.215172189 +0000 UTC m=+18.584076483, 105.071747ms after stability testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.215Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: cannot transfer leadership to itself testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.215Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: cannot transfer leadership to itself testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.215Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: aborting pipeline replication: peer="{Voter server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a 0d4b481b-b09b-77da-b4ca-452aa42ccf2a}" testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.215Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: aborting pipeline replication: peer="{Voter server-0d4b481b-b09b-77da-b4ca-452aa42ccf2a 0d4b481b-b09b-77da-b4ca-452aa42ccf2a}" testLoggerAdapter verbose: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.215Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: aborting pipeline replication: peer="{Voter server-42198df6-ee40-1b10-5914-b94129e72bc0 42198df6-ee40-1b10-5914-b94129e72bc0}" testing.go:149: server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: 2025-02-10T11:34:22.215Z [INFO] server-f286f4b5-df17-1e6c-cd26-acd57f83e12c: aborting pipeline replication: peer="{Voter server-42198df6-ee40-1b10-5914-b94129e72bc0 42198df6-ee40-1b10-5914-b94129e72bc0}" --- PASS: TestRaft_LeadershipTransferToItself (0.16s) === RUN TestRaft_LeadershipTransferLeaderRejectsClientRequests raft_test.go:2064: Disabled unreliable test --- SKIP: TestRaft_LeadershipTransferLeaderRejectsClientRequests (0.00s) === RUN TestRaft_LeadershipTransferLeaderReplicationTimeout testLoggerAdapter verbose: server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: 2025-02-10T11:34:22.216Z [INFO] server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e Address:3a12c92e-fc2d-1f28-050f-adf44c1a0f2e} {Suffrage:Voter ID:server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a Address:3a70638d-9f2a-c382-2ad7-b0e872d06a3a} {Suffrage:Voter ID:server-9050a661-80a4-d229-72a8-da77335bbc36 Address:9050a661-80a4-d229-72a8-da77335bbc36}]" testing.go:149: server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: 2025-02-10T11:34:22.216Z [INFO] server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e Address:3a12c92e-fc2d-1f28-050f-adf44c1a0f2e} {Suffrage:Voter ID:server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a Address:3a70638d-9f2a-c382-2ad7-b0e872d06a3a} {Suffrage:Voter ID:server-9050a661-80a4-d229-72a8-da77335bbc36 Address:9050a661-80a4-d229-72a8-da77335bbc36}]" testLoggerAdapter verbose: server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: 2025-02-10T11:34:22.216Z [INFO] server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: entering follower state: follower="Node at 3a12c92e-fc2d-1f28-050f-adf44c1a0f2e [Follower]" leader= testing.go:149: server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: 2025-02-10T11:34:22.216Z [INFO] server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e: entering follower state: follower="Node at 3a12c92e-fc2d-1f28-050f-adf44c1a0f2e [Follower]" leader= testLoggerAdapter verbose: server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: 2025-02-10T11:34:22.216Z [INFO] server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e Address:3a12c92e-fc2d-1f28-050f-adf44c1a0f2e} {Suffrage:Voter ID:server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a Address:3a70638d-9f2a-c382-2ad7-b0e872d06a3a} {Suffrage:Voter ID:server-9050a661-80a4-d229-72a8-da77335bbc36 Address:9050a661-80a4-d229-72a8-da77335bbc36}]" testing.go:149: server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: 2025-02-10T11:34:22.216Z [INFO] server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e Address:3a12c92e-fc2d-1f28-050f-adf44c1a0f2e} {Suffrage:Voter ID:server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a Address:3a70638d-9f2a-c382-2ad7-b0e872d06a3a} {Suffrage:Voter ID:server-9050a661-80a4-d229-72a8-da77335bbc36 Address:9050a661-80a4-d229-72a8-da77335bbc36}]" testLoggerAdapter verbose: server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: 2025-02-10T11:34:22.216Z [INFO] server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: entering follower state: follower="Node at 3a70638d-9f2a-c382-2ad7-b0e872d06a3a [Follower]" leader= testing.go:149: server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: 2025-02-10T11:34:22.216Z [INFO] server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a: entering follower state: follower="Node at 3a70638d-9f2a-c382-2ad7-b0e872d06a3a [Follower]" leader= testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.216Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e Address:3a12c92e-fc2d-1f28-050f-adf44c1a0f2e} {Suffrage:Voter ID:server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a Address:3a70638d-9f2a-c382-2ad7-b0e872d06a3a} {Suffrage:Voter ID:server-9050a661-80a4-d229-72a8-da77335bbc36 Address:9050a661-80a4-d229-72a8-da77335bbc36}]" testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.216Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e Address:3a12c92e-fc2d-1f28-050f-adf44c1a0f2e} {Suffrage:Voter ID:server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a Address:3a70638d-9f2a-c382-2ad7-b0e872d06a3a} {Suffrage:Voter ID:server-9050a661-80a4-d229-72a8-da77335bbc36 Address:9050a661-80a4-d229-72a8-da77335bbc36}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.216Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:22.216Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.216Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: entering follower state: follower="Node at 9050a661-80a4-d229-72a8-da77335bbc36 [Follower]" leader= testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.216Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: entering follower state: follower="Node at 9050a661-80a4-d229-72a8-da77335bbc36 [Follower]" leader= testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [WARN] server-9050a661-80a4-d229-72a8-da77335bbc36: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [WARN] server-9050a661-80a4-d229-72a8-da77335bbc36: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: entering candidate state: node="Node at 9050a661-80a4-d229-72a8-da77335bbc36 [Candidate]" term=2 testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: entering candidate state: node="Node at 9050a661-80a4-d229-72a8-da77335bbc36 [Candidate]" term=2 testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: election won: tally=2 testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: election won: tally=2 testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: entering leader state: leader="Node at 9050a661-80a4-d229-72a8-da77335bbc36 [Leader]" testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: entering leader state: leader="Node at 9050a661-80a4-d229-72a8-da77335bbc36 [Leader]" testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: added peer, starting replication: peer=server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: added peer, starting replication: peer=server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: added peer, starting replication: peer=server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: added peer, starting replication: peer=server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: pipelining replication: peer="{Voter server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e 3a12c92e-fc2d-1f28-050f-adf44c1a0f2e}" testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: pipelining replication: peer="{Voter server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e 3a12c92e-fc2d-1f28-050f-adf44c1a0f2e}" testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: pipelining replication: peer="{Voter server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a 3a70638d-9f2a-c382-2ad7-b0e872d06a3a}" testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.289Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: pipelining replication: peer="{Voter server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a 3a70638d-9f2a-c382-2ad7-b0e872d06a3a}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.394Z [INFO] stable state for Leader reached at 2025-02-10 11:34:22.289543263 +0000 UTC m=+18.658447574 (1 nodes), 72.929579ms from start of poll, 73.324185ms from cluster start. Timeout at 2025-02-10 11:34:22.394615366 +0000 UTC m=+18.763519662, 105.072088ms after stability testing.go:149: cluster: 2025-02-10T11:34:22.394Z [INFO] stable state for Leader reached at 2025-02-10 11:34:22.289543263 +0000 UTC m=+18.658447574 (1 nodes), 72.929579ms from start of poll, 73.324185ms from cluster start. Timeout at 2025-02-10 11:34:22.394615366 +0000 UTC m=+18.763519662, 105.072088ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.394Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:22.394Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.499Z [INFO] stable state for Follower reached at 2025-02-10 11:34:22.394693105 +0000 UTC m=+18.763597404 (2 nodes), 630ns from start of poll, 178.474015ms from cluster start. Timeout at 2025-02-10 11:34:22.49976504 +0000 UTC m=+18.868669336, 105.071932ms after stability testing.go:149: cluster: 2025-02-10T11:34:22.499Z [INFO] stable state for Follower reached at 2025-02-10 11:34:22.394693105 +0000 UTC m=+18.763597404 (2 nodes), 630ns from start of poll, 178.474015ms from cluster start. Timeout at 2025-02-10 11:34:22.49976504 +0000 UTC m=+18.868669336, 105.071932ms after stability testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.621Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: aborting pipeline replication: peer="{Voter server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a 3a70638d-9f2a-c382-2ad7-b0e872d06a3a}" testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.621Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: aborting pipeline replication: peer="{Voter server-3a70638d-9f2a-c382-2ad7-b0e872d06a3a 3a70638d-9f2a-c382-2ad7-b0e872d06a3a}" testLoggerAdapter verbose: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.621Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: aborting pipeline replication: peer="{Voter server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e 3a12c92e-fc2d-1f28-050f-adf44c1a0f2e}" testing.go:149: server-9050a661-80a4-d229-72a8-da77335bbc36: 2025-02-10T11:34:22.621Z [INFO] server-9050a661-80a4-d229-72a8-da77335bbc36: aborting pipeline replication: peer="{Voter server-3a12c92e-fc2d-1f28-050f-adf44c1a0f2e 3a12c92e-fc2d-1f28-050f-adf44c1a0f2e}" --- PASS: TestRaft_LeadershipTransferLeaderReplicationTimeout (0.41s) === RUN TestRaft_LeadershipTransferStopRightAway --- PASS: TestRaft_LeadershipTransferStopRightAway (0.00s) === RUN TestRaft_GetConfigurationNoBootstrap testLoggerAdapter verbose: server-5d4aa415-ddd0-1158-abc6-271831494b0a: 2025-02-10T11:34:22.622Z [INFO] server-5d4aa415-ddd0-1158-abc6-271831494b0a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5d4aa415-ddd0-1158-abc6-271831494b0a Address:5d4aa415-ddd0-1158-abc6-271831494b0a} {Suffrage:Voter ID:server-ccd79914-4e22-1b53-2c72-90211931b3e5 Address:ccd79914-4e22-1b53-2c72-90211931b3e5}]" testing.go:149: server-5d4aa415-ddd0-1158-abc6-271831494b0a: 2025-02-10T11:34:22.622Z [INFO] server-5d4aa415-ddd0-1158-abc6-271831494b0a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5d4aa415-ddd0-1158-abc6-271831494b0a Address:5d4aa415-ddd0-1158-abc6-271831494b0a} {Suffrage:Voter ID:server-ccd79914-4e22-1b53-2c72-90211931b3e5 Address:ccd79914-4e22-1b53-2c72-90211931b3e5}]" testLoggerAdapter verbose: server-5d4aa415-ddd0-1158-abc6-271831494b0a: 2025-02-10T11:34:22.622Z [INFO] server-5d4aa415-ddd0-1158-abc6-271831494b0a: entering follower state: follower="Node at 5d4aa415-ddd0-1158-abc6-271831494b0a [Follower]" leader= testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.622Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5d4aa415-ddd0-1158-abc6-271831494b0a Address:5d4aa415-ddd0-1158-abc6-271831494b0a} {Suffrage:Voter ID:server-ccd79914-4e22-1b53-2c72-90211931b3e5 Address:ccd79914-4e22-1b53-2c72-90211931b3e5}]" testing.go:149: server-5d4aa415-ddd0-1158-abc6-271831494b0a: 2025-02-10T11:34:22.622Z [INFO] server-5d4aa415-ddd0-1158-abc6-271831494b0a: entering follower state: follower="Node at 5d4aa415-ddd0-1158-abc6-271831494b0a [Follower]" leader= testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.622Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5d4aa415-ddd0-1158-abc6-271831494b0a Address:5d4aa415-ddd0-1158-abc6-271831494b0a} {Suffrage:Voter ID:server-ccd79914-4e22-1b53-2c72-90211931b3e5 Address:ccd79914-4e22-1b53-2c72-90211931b3e5}]" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.622Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2025-02-10T11:34:22.622Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.622Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: entering follower state: follower="Node at ccd79914-4e22-1b53-2c72-90211931b3e5 [Follower]" leader= testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.622Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: entering follower state: follower="Node at ccd79914-4e22-1b53-2c72-90211931b3e5 [Follower]" leader= testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [WARN] server-ccd79914-4e22-1b53-2c72-90211931b3e5: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [WARN] server-ccd79914-4e22-1b53-2c72-90211931b3e5: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: entering candidate state: node="Node at ccd79914-4e22-1b53-2c72-90211931b3e5 [Candidate]" term=2 testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: entering candidate state: node="Node at ccd79914-4e22-1b53-2c72-90211931b3e5 [Candidate]" term=2 testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: election won: tally=2 testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: election won: tally=2 testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: entering leader state: leader="Node at ccd79914-4e22-1b53-2c72-90211931b3e5 [Leader]" testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: entering leader state: leader="Node at ccd79914-4e22-1b53-2c72-90211931b3e5 [Leader]" testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: added peer, starting replication: peer=server-5d4aa415-ddd0-1158-abc6-271831494b0a testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: added peer, starting replication: peer=server-5d4aa415-ddd0-1158-abc6-271831494b0a testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: pipelining replication: peer="{Voter server-5d4aa415-ddd0-1158-abc6-271831494b0a 5d4aa415-ddd0-1158-abc6-271831494b0a}" testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.705Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: pipelining replication: peer="{Voter server-5d4aa415-ddd0-1158-abc6-271831494b0a 5d4aa415-ddd0-1158-abc6-271831494b0a}" testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.810Z [INFO] stable state for Follower reached at 2025-02-10 11:34:22.705849563 +0000 UTC m=+19.074753862 (1 nodes), 83.618531ms from start of poll, 83.895021ms from cluster start. Timeout at 2025-02-10 11:34:22.810947068 +0000 UTC m=+19.179851365, 105.097503ms after stability testing.go:149: cluster: 2025-02-10T11:34:22.810Z [INFO] stable state for Follower reached at 2025-02-10 11:34:22.705849563 +0000 UTC m=+19.074753862 (1 nodes), 83.618531ms from start of poll, 83.895021ms from cluster start. Timeout at 2025-02-10 11:34:22.810947068 +0000 UTC m=+19.179851365, 105.097503ms after stability testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.811Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2025-02-10T11:34:22.811Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2025-02-10T11:34:22.916Z [INFO] stable state for Leader reached at 2025-02-10 11:34:22.811028212 +0000 UTC m=+19.179932513 (1 nodes), 758ns from start of poll, 189.073672ms from cluster start. Timeout at 2025-02-10 11:34:22.916046832 +0000 UTC m=+19.284951142, 105.018629ms after stability testing.go:149: cluster: 2025-02-10T11:34:22.916Z [INFO] stable state for Leader reached at 2025-02-10 11:34:22.811028212 +0000 UTC m=+19.179932513 (1 nodes), 758ns from start of poll, 189.073672ms from cluster start. Timeout at 2025-02-10 11:34:22.916046832 +0000 UTC m=+19.284951142, 105.018629ms after stability testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.926Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5d4aa415-ddd0-1158-abc6-271831494b0a Address:5d4aa415-ddd0-1158-abc6-271831494b0a} {Suffrage:Voter ID:server-ccd79914-4e22-1b53-2c72-90211931b3e5 Address:ccd79914-4e22-1b53-2c72-90211931b3e5}]" testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.926Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5d4aa415-ddd0-1158-abc6-271831494b0a Address:5d4aa415-ddd0-1158-abc6-271831494b0a} {Suffrage:Voter ID:server-ccd79914-4e22-1b53-2c72-90211931b3e5 Address:ccd79914-4e22-1b53-2c72-90211931b3e5}]" testLoggerAdapter verbose: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.926Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: aborting pipeline replication: peer="{Voter server-5d4aa415-ddd0-1158-abc6-271831494b0a 5d4aa415-ddd0-1158-abc6-271831494b0a}" testing.go:149: server-ccd79914-4e22-1b53-2c72-90211931b3e5: 2025-02-10T11:34:22.926Z [INFO] server-ccd79914-4e22-1b53-2c72-90211931b3e5: aborting pipeline replication: peer="{Voter server-5d4aa415-ddd0-1158-abc6-271831494b0a 5d4aa415-ddd0-1158-abc6-271831494b0a}" --- PASS: TestRaft_GetConfigurationNoBootstrap (0.31s) === RUN TestTCPTransport_BadAddr --- PASS: TestTCPTransport_BadAddr (0.00s) === RUN TestTCPTransport_WithAdvertise --- PASS: TestTCPTransport_WithAdvertise (0.00s) === RUN TestTransport_StartStop --- PASS: TestTransport_StartStop (0.00s) === RUN TestTransport_AppendEntries --- PASS: TestTransport_AppendEntries (0.00s) === RUN TestTransport_AppendEntriesPipeline --- PASS: TestTransport_AppendEntriesPipeline (0.00s) === RUN TestTransport_RequestVote --- PASS: TestTransport_RequestVote (0.00s) === RUN TestTransport_InstallSnapshot --- PASS: TestTransport_InstallSnapshot (0.00s) === RUN TestTransport_EncodeDecode --- PASS: TestTransport_EncodeDecode (0.00s) === RUN TestRandomTimeout --- PASS: TestRandomTimeout (0.00s) === RUN TestNewSeed --- PASS: TestNewSeed (0.00s) === RUN TestRandomTimeout_NoTime --- PASS: TestRandomTimeout_NoTime (0.00s) === RUN TestMin --- PASS: TestMin (0.00s) === RUN TestMax --- PASS: TestMax (0.00s) === RUN TestGenerateUUID --- PASS: TestGenerateUUID (0.00s) === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok github.com/hashicorp/raft 19.306s ? github.com/hashicorp/raft/bench [no test files] create-stamp debian/debhelper-build-stamp fakeroot debian/rules binary dh binary --buildsystem=golang --with=golang dh_testroot -O--buildsystem=golang dh_prep -O--buildsystem=golang dh_auto_install -O--buildsystem=golang dh_installdocs -O--buildsystem=golang dh_installchangelogs -O--buildsystem=golang dh_installinit -O--buildsystem=golang dh_installsystemduser -O--buildsystem=golang dh_perl -O--buildsystem=golang dh_link -O--buildsystem=golang dh_strip_nondeterminism -O--buildsystem=golang dh_compress -O--buildsystem=golang dh_fixperms -O--buildsystem=golang dh_missing -O--buildsystem=golang dh_installdeb -O--buildsystem=golang dh_golang -O--buildsystem=golang dh_gencontrol -O--buildsystem=golang dpkg-gencontrol: warning: Depends field of package golang-github-hashicorp-raft-dev: substitution variable ${shlibs:Depends} used, but is not defined dh_md5sums -O--buildsystem=golang dh_builddeb -O--buildsystem=golang dpkg-deb: building package 'golang-github-hashicorp-raft-dev' in '../golang-github-hashicorp-raft-dev_1.1.2-1_all.deb'. dpkg-genbuildinfo --build=binary dpkg-genchanges --build=binary >../golang-github-hashicorp-raft_1.1.2-1_i386.changes dpkg-genchanges: info: binary-only upload (no source code included) dpkg-source --after-build . dpkg-buildpackage: info: binary-only upload (no source included) dpkg-genchanges: info: including full source code in upload I: copying local configuration I: user script /srv/workspace/pbuilder/63060/tmp/hooks/B01_cleanup starting I: user script /srv/workspace/pbuilder/63060/tmp/hooks/B01_cleanup finished I: unmounting dev/ptmx filesystem I: unmounting dev/pts filesystem I: unmounting dev/shm filesystem I: unmounting proc filesystem I: unmounting sys filesystem I: cleaning the build env I: removing directory /srv/workspace/pbuilder/63060 and its subdirectories I: Current time: Tue Feb 11 01:34:26 +14 2025 I: pbuilder-time-stamp: 1739187266