I: pbuilder: network access will be disabled during build I: Current time: Wed Aug 25 23:28:56 +14 2021 I: pbuilder-time-stamp: 1629883736 I: Building the build Environment I: extracting base tarball [/var/cache/pbuilder/bullseye-reproducible-base.tgz] I: copying local configuration I: mounting /proc filesystem I: mounting /sys filesystem I: creating /{dev,run}/shm I: mounting /dev/pts filesystem I: redirecting /dev/ptmx to /dev/pts/ptmx I: policy-rc.d already exists I: Copying source file I: copying [golang-github-hashicorp-raft_1.1.2-1.dsc] I: copying [./golang-github-hashicorp-raft_1.1.2.orig.tar.xz] I: copying [./golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz] I: Extracting source gpgv: unknown type of key resource 'trustedkeys.kbx' gpgv: keyblock resource '/tmp/dpkg-verify-sig.JjUvhcv8/trustedkeys.kbx': General error gpgv: Signature made Sun Jan 19 21:24:28 2020 +14 gpgv: using RSA key 50BC7CF939D20C272A6B065652B6BBD953968D1B gpgv: Can't check signature: No public key dpkg-source: warning: failed to verify signature on ./golang-github-hashicorp-raft_1.1.2-1.dsc dpkg-source: info: extracting golang-github-hashicorp-raft in golang-github-hashicorp-raft-1.1.2 dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2.orig.tar.xz dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz dpkg-source: info: using patch list from debian/patches/series dpkg-source: info: applying t-skip--TestNetworkTransport_AppendEntriesPipeline_CloseStreams.patch dpkg-source: info: applying t-skip--TestRaft_AutoSnapshot.patch dpkg-source: info: applying t-skip--TestRaft_LeadershipTransferLeaderRejectsClientRequests.patch dpkg-source: info: applying t-skip--TestRaft_RecoverCluster.patch dpkg-source: info: applying t-skip--TestRaft_SnapshotRestore_PeerChange.patch I: using fakeroot in build. I: Installing the build-deps I: user script /srv/workspace/pbuilder/7057/tmp/hooks/D01_modify_environment starting debug: Running on virt32b. I: Changing host+domainname to test build reproducibility I: Adding a custom variable just for the fun of it... I: Changing /bin/sh to bash Removing 'diversion of /bin/sh to /bin/sh.distrib by dash' Adding 'diversion of /bin/sh to /bin/sh.distrib by bash' Removing 'diversion of /usr/share/man/man1/sh.1.gz to /usr/share/man/man1/sh.distrib.1.gz by dash' Adding 'diversion of /usr/share/man/man1/sh.1.gz to /usr/share/man/man1/sh.distrib.1.gz by bash' I: Setting pbuilder2's login shell to /bin/bash I: Setting pbuilder2's GECOS to second user,second room,second work-phone,second home-phone,second other I: user script /srv/workspace/pbuilder/7057/tmp/hooks/D01_modify_environment finished I: user script /srv/workspace/pbuilder/7057/tmp/hooks/D02_print_environment starting I: set BASH=/bin/sh BASHOPTS=checkwinsize:cmdhist:complete_fullquote:extquote:force_fignore:globasciiranges:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath BASH_ALIASES=() BASH_ARGC=() BASH_ARGV=() BASH_CMDS=() BASH_LINENO=([0]="12" [1]="0") BASH_SOURCE=([0]="/tmp/hooks/D02_print_environment" [1]="/tmp/hooks/D02_print_environment") BASH_VERSINFO=([0]="5" [1]="1" [2]="4" [3]="1" [4]="release" [5]="arm-unknown-linux-gnueabihf") BASH_VERSION='5.1.4(1)-release' BUILDDIR=/build BUILDUSERGECOS='second user,second room,second work-phone,second home-phone,second other' BUILDUSERNAME=pbuilder2 BUILD_ARCH=armhf DEBIAN_FRONTEND=noninteractive DEB_BUILD_OPTIONS='buildinfo=+all reproducible=+all,-fixfilepath parallel=4' DIRSTACK=() DISTRIBUTION= EUID=0 FUNCNAME=([0]="Echo" [1]="main") GROUPS=() HOME=/root HOSTNAME=i-capture-the-hostname HOSTTYPE=arm HOST_ARCH=armhf IFS=' ' INVOCATION_ID=3c69134a93044a789ab16a0186c8d167 LANG=C LANGUAGE=it_CH:it LC_ALL=C MACHTYPE=arm-unknown-linux-gnueabihf MAIL=/var/mail/root OPTERR=1 OPTIND=1 OSTYPE=linux-gnueabihf PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path PBCURRENTCOMMANDLINEOPERATION=build PBUILDER_OPERATION=build PBUILDER_PKGDATADIR=/usr/share/pbuilder PBUILDER_PKGLIBDIR=/usr/lib/pbuilder PBUILDER_SYSCONFDIR=/etc PIPESTATUS=([0]="0") POSIXLY_CORRECT=y PPID=7057 PS4='+ ' PWD=/ SHELL=/bin/bash SHELLOPTS=braceexpand:errexit:hashall:interactive-comments:posix SHLVL=3 SUDO_COMMAND='/usr/bin/timeout -k 24.1h 24h /usr/bin/ionice -c 3 /usr/bin/nice -n 11 /usr/bin/unshare --uts -- /usr/sbin/pbuilder --build --configfile /srv/reproducible-results/rbuild-debian/tmp.wjmIrRuTSj/pbuilderrc_HV2Y --hookdir /etc/pbuilder/rebuild-hooks --debbuildopts -b --basetgz /var/cache/pbuilder/bullseye-reproducible-base.tgz --buildresult /srv/reproducible-results/rbuild-debian/tmp.wjmIrRuTSj/b2 --logfile b2/build.log golang-github-hashicorp-raft_1.1.2-1.dsc' SUDO_GID=112 SUDO_UID=106 SUDO_USER=jenkins TERM=unknown TZ=/usr/share/zoneinfo/Etc/GMT-14 UID=0 USER=root _='I: set' http_proxy=http://10.0.0.15:8000/ I: uname -a Linux i-capture-the-hostname 5.10.0-8-armmp-lpae #1 SMP Debian 5.10.46-4 (2021-08-03) armv7l GNU/Linux I: ls -l /bin total 3580 -rwxr-xr-x 1 root root 816764 Aug 5 10:25 bash -rwxr-xr-x 3 root root 26052 Jul 21 2020 bunzip2 -rwxr-xr-x 3 root root 26052 Jul 21 2020 bzcat lrwxrwxrwx 1 root root 6 Jul 21 2020 bzcmp -> bzdiff -rwxr-xr-x 1 root root 2225 Jul 21 2020 bzdiff lrwxrwxrwx 1 root root 6 Jul 21 2020 bzegrep -> bzgrep -rwxr-xr-x 1 root root 4877 Sep 5 2019 bzexe lrwxrwxrwx 1 root root 6 Jul 21 2020 bzfgrep -> bzgrep -rwxr-xr-x 1 root root 3775 Jul 21 2020 bzgrep -rwxr-xr-x 3 root root 26052 Jul 21 2020 bzip2 -rwxr-xr-x 1 root root 9636 Jul 21 2020 bzip2recover lrwxrwxrwx 1 root root 6 Jul 21 2020 bzless -> bzmore -rwxr-xr-x 1 root root 1297 Jul 21 2020 bzmore -rwxr-xr-x 1 root root 26668 Sep 23 2020 cat -rwxr-xr-x 1 root root 43104 Sep 23 2020 chgrp -rwxr-xr-x 1 root root 38984 Sep 23 2020 chmod -rwxr-xr-x 1 root root 43112 Sep 23 2020 chown -rwxr-xr-x 1 root root 92616 Sep 23 2020 cp -rwxr-xr-x 1 root root 75524 Dec 11 2020 dash -rwxr-xr-x 1 root root 75880 Sep 23 2020 date -rwxr-xr-x 1 root root 55436 Sep 23 2020 dd -rwxr-xr-x 1 root root 59912 Sep 23 2020 df -rwxr-xr-x 1 root root 96764 Sep 23 2020 dir -rwxr-xr-x 1 root root 55012 Jul 29 09:09 dmesg lrwxrwxrwx 1 root root 8 Nov 8 2019 dnsdomainname -> hostname lrwxrwxrwx 1 root root 8 Nov 8 2019 domainname -> hostname -rwxr-xr-x 1 root root 22508 Sep 23 2020 echo -rwxr-xr-x 1 root root 28 Nov 10 2020 egrep -rwxr-xr-x 1 root root 22496 Sep 23 2020 false -rwxr-xr-x 1 root root 28 Nov 10 2020 fgrep -rwxr-xr-x 1 root root 47492 Jul 29 09:09 findmnt -rwsr-xr-x 1 root root 26076 Feb 27 06:12 fusermount -rwxr-xr-x 1 root root 124508 Nov 10 2020 grep -rwxr-xr-x 2 root root 2346 Mar 3 13:30 gunzip -rwxr-xr-x 1 root root 6376 Mar 3 13:30 gzexe -rwxr-xr-x 1 root root 64212 Mar 3 13:30 gzip -rwxr-xr-x 1 root root 13784 Nov 8 2019 hostname -rwxr-xr-x 1 root root 43180 Sep 23 2020 ln -rwxr-xr-x 1 root root 35068 Feb 8 2020 login -rwxr-xr-x 1 root root 96764 Sep 23 2020 ls -rwxr-xr-x 1 root root 99940 Jul 29 09:09 lsblk -rwxr-xr-x 1 root root 51408 Sep 23 2020 mkdir -rwxr-xr-x 1 root root 43184 Sep 23 2020 mknod -rwxr-xr-x 1 root root 30780 Sep 23 2020 mktemp -rwxr-xr-x 1 root root 34408 Jul 29 09:09 more -rwsr-xr-x 1 root root 34400 Jul 29 09:09 mount -rwxr-xr-x 1 root root 9824 Jul 29 09:09 mountpoint -rwxr-xr-x 1 root root 88524 Sep 23 2020 mv lrwxrwxrwx 1 root root 8 Nov 8 2019 nisdomainname -> hostname lrwxrwxrwx 1 root root 14 Apr 19 05:38 pidof -> /sbin/killall5 -rwxr-xr-x 1 root root 26652 Sep 23 2020 pwd lrwxrwxrwx 1 root root 4 Aug 5 10:25 rbash -> bash -rwxr-xr-x 1 root root 30740 Sep 23 2020 readlink -rwxr-xr-x 1 root root 43104 Sep 23 2020 rm -rwxr-xr-x 1 root root 30732 Sep 23 2020 rmdir -rwxr-xr-x 1 root root 14144 Sep 28 2020 run-parts -rwxr-xr-x 1 root root 76012 Dec 23 2018 sed lrwxrwxrwx 1 root root 4 Aug 25 23:29 sh -> bash lrwxrwxrwx 1 root root 4 Aug 21 23:25 sh.distrib -> dash -rwxr-xr-x 1 root root 22532 Sep 23 2020 sleep -rwxr-xr-x 1 root root 55360 Sep 23 2020 stty -rwsr-xr-x 1 root root 46704 Jul 29 09:09 su -rwxr-xr-x 1 root root 22532 Sep 23 2020 sync -rwxr-xr-x 1 root root 340872 Feb 17 2021 tar -rwxr-xr-x 1 root root 9808 Sep 28 2020 tempfile -rwxr-xr-x 1 root root 67696 Sep 23 2020 touch -rwxr-xr-x 1 root root 22496 Sep 23 2020 true -rwxr-xr-x 1 root root 9636 Feb 27 06:12 ulockmgr_server -rwsr-xr-x 1 root root 22108 Jul 29 09:09 umount -rwxr-xr-x 1 root root 22520 Sep 23 2020 uname -rwxr-xr-x 2 root root 2346 Mar 3 13:30 uncompress -rwxr-xr-x 1 root root 96764 Sep 23 2020 vdir -rwxr-xr-x 1 root root 38512 Jul 29 09:09 wdctl lrwxrwxrwx 1 root root 8 Nov 8 2019 ypdomainname -> hostname -rwxr-xr-x 1 root root 1984 Mar 3 13:30 zcat -rwxr-xr-x 1 root root 1678 Mar 3 13:30 zcmp -rwxr-xr-x 1 root root 5880 Mar 3 13:30 zdiff -rwxr-xr-x 1 root root 29 Mar 3 13:30 zegrep -rwxr-xr-x 1 root root 29 Mar 3 13:30 zfgrep -rwxr-xr-x 1 root root 2081 Mar 3 13:30 zforce -rwxr-xr-x 1 root root 7585 Mar 3 13:30 zgrep -rwxr-xr-x 1 root root 2206 Mar 3 13:30 zless -rwxr-xr-x 1 root root 1842 Mar 3 13:30 zmore -rwxr-xr-x 1 root root 4553 Mar 3 13:30 znew I: user script /srv/workspace/pbuilder/7057/tmp/hooks/D02_print_environment finished -> Attempting to satisfy build-dependencies -> Creating pbuilder-satisfydepends-dummy package Package: pbuilder-satisfydepends-dummy Version: 0.invalid.0 Architecture: armhf Maintainer: Debian Pbuilder Team Description: Dummy package to satisfy dependencies with aptitude - created by pbuilder This package was created automatically by pbuilder to satisfy the build-dependencies of the package being currently built. Depends: debhelper (>= 12~), dh-golang, golang-any, golang-github-armon-go-metrics-dev, golang-github-hashicorp-go-hclog-dev (>= 0.9.2~), golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~) dpkg-deb: building package 'pbuilder-satisfydepends-dummy' in '/tmp/satisfydepends-aptitude/pbuilder-satisfydepends-dummy.deb'. Selecting previously unselected package pbuilder-satisfydepends-dummy. (Reading database ... 19398 files and directories currently installed.) Preparing to unpack .../pbuilder-satisfydepends-dummy.deb ... Unpacking pbuilder-satisfydepends-dummy (0.invalid.0) ... dpkg: pbuilder-satisfydepends-dummy: dependency problems, but configuring anyway as you requested: pbuilder-satisfydepends-dummy depends on debhelper (>= 12~); however: Package debhelper is not installed. pbuilder-satisfydepends-dummy depends on dh-golang; however: Package dh-golang is not installed. pbuilder-satisfydepends-dummy depends on golang-any; however: Package golang-any is not installed. pbuilder-satisfydepends-dummy depends on golang-github-armon-go-metrics-dev; however: Package golang-github-armon-go-metrics-dev is not installed. pbuilder-satisfydepends-dummy depends on golang-github-hashicorp-go-hclog-dev (>= 0.9.2~); however: Package golang-github-hashicorp-go-hclog-dev is not installed. pbuilder-satisfydepends-dummy depends on golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~); however: Package golang-github-hashicorp-go-msgpack-dev is not installed. Setting up pbuilder-satisfydepends-dummy (0.invalid.0) ... Reading package lists... Building dependency tree... Reading state information... Initializing package states... Writing extended state information... Building tag database... pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) The following NEW packages will be installed: autoconf{a} automake{a} autopoint{a} autotools-dev{a} bsdextrautils{a} debhelper{a} dh-autoreconf{a} dh-golang{a} dh-strip-nondeterminism{a} dwz{a} file{a} gettext{a} gettext-base{a} golang-1.15-go{a} golang-1.15-src{a} golang-any{a} golang-github-alecthomas-units-dev{a} golang-github-armon-go-metrics-dev{a} golang-github-beorn7-perks-dev{a} golang-github-cespare-xxhash-dev{a} golang-github-circonus-labs-circonus-gometrics-dev{a} golang-github-circonus-labs-circonusllhist-dev{a} golang-github-creack-pty-dev{a} golang-github-datadog-datadog-go-dev{a} golang-github-davecgh-go-spew-dev{a} golang-github-dgrijalva-jwt-go-dev{a} golang-github-fatih-color-dev{a} golang-github-go-kit-kit-dev{a} golang-github-go-logfmt-logfmt-dev{a} golang-github-go-stack-stack-dev{a} golang-github-golang-mock-dev{a} golang-github-google-go-cmp-dev{a} golang-github-hashicorp-go-cleanhttp-dev{a} golang-github-hashicorp-go-hclog-dev{a} golang-github-hashicorp-go-immutable-radix-dev{a} golang-github-hashicorp-go-msgpack-dev{a} golang-github-hashicorp-go-retryablehttp-dev{a} golang-github-hashicorp-go-uuid-dev{a} golang-github-hashicorp-golang-lru-dev{a} golang-github-jpillora-backoff-dev{a} golang-github-json-iterator-go-dev{a} golang-github-julienschmidt-httprouter-dev{a} golang-github-kr-pretty-dev{a} golang-github-kr-text-dev{a} golang-github-mattn-go-colorable-dev{a} golang-github-mattn-go-isatty-dev{a} golang-github-modern-go-concurrent-dev{a} golang-github-modern-go-reflect2-dev{a} golang-github-mwitkow-go-conntrack-dev{a} golang-github-opentracing-opentracing-go-dev{a} golang-github-pascaldekloe-goe-dev{a} golang-github-pkg-errors-dev{a} golang-github-pmezard-go-difflib-dev{a} golang-github-prometheus-client-golang-dev{a} golang-github-prometheus-client-model-dev{a} golang-github-prometheus-common-dev{a} golang-github-prometheus-procfs-dev{a} golang-github-sirupsen-logrus-dev{a} golang-github-stretchr-objx-dev{a} golang-github-stretchr-testify-dev{a} golang-github-tv42-httpunix-dev{a} golang-github-vmihailenco-tagparser-dev{a} golang-glog-dev{a} golang-go{a} golang-golang-x-net-dev{a} golang-golang-x-oauth2-dev{a} golang-golang-x-oauth2-google-dev{a} golang-golang-x-sync-dev{a} golang-golang-x-sys-dev{a} golang-golang-x-text-dev{a} golang-golang-x-time-dev{a} golang-golang-x-xerrors-dev{a} golang-google-cloud-compute-metadata-dev{a} golang-google-genproto-dev{a} golang-google-grpc-dev{a} golang-gopkg-alecthomas-kingpin.v2-dev{a} golang-gopkg-check.v1-dev{a} golang-gopkg-mgo.v2-dev{a} golang-gopkg-tomb.v2-dev{a} golang-gopkg-vmihailenco-msgpack.v2-dev{a} golang-gopkg-yaml.v2-dev{a} golang-gopkg-yaml.v3-dev{a} golang-goprotobuf-dev{a} golang-protobuf-extensions-dev{a} golang-src{a} groff-base{a} intltool-debian{a} libarchive-zip-perl{a} libdebhelper-perl{a} libelf1{a} libfile-stripnondeterminism-perl{a} libicu67{a} libmagic-mgc{a} libmagic1{a} libpipeline1{a} libprotobuf-dev{a} libprotobuf-lite23{a} libprotobuf23{a} libprotoc23{a} libsasl2-2{a} libsasl2-dev{a} libsasl2-modules-db{a} libsigsegv2{a} libsub-override-perl{a} libtool{a} libuchardet0{a} libxml2{a} m4{a} man-db{a} po-debconf{a} protobuf-compiler{a} sensible-utils{a} zlib1g-dev{a} The following packages are RECOMMENDED but will NOT be installed: curl libarchive-cpio-perl libltdl-dev libmail-sendmail-perl libsasl2-modules lynx pkg-config pkgconf wget 0 packages upgraded, 113 newly installed, 0 to remove and 0 not upgraded. Need to get 91.1 MB of archives. After unpacking 528 MB will be used. Writing extended state information... Get: 1 http://deb.debian.org/debian bullseye/main armhf bsdextrautils armhf 2.36.1-8 [138 kB] Get: 2 http://deb.debian.org/debian bullseye/main armhf libuchardet0 armhf 0.0.7-1 [65.0 kB] Get: 3 http://deb.debian.org/debian bullseye/main armhf groff-base armhf 1.22.4-6 [847 kB] Get: 4 http://deb.debian.org/debian bullseye/main armhf libpipeline1 armhf 1.5.3-1 [30.1 kB] Get: 5 http://deb.debian.org/debian bullseye/main armhf man-db armhf 2.9.4-2 [1319 kB] Get: 6 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-sys-dev all 0.0~git20210124.22da62e-1 [308 kB] Get: 7 http://deb.debian.org/debian bullseye/main armhf golang-github-sirupsen-logrus-dev all 1.7.0-2 [45.3 kB] Get: 8 http://deb.debian.org/debian bullseye/main armhf sensible-utils all 0.0.14 [14.8 kB] Get: 9 http://deb.debian.org/debian bullseye/main armhf libmagic-mgc armhf 1:5.39-3 [273 kB] Get: 10 http://deb.debian.org/debian bullseye/main armhf libmagic1 armhf 1:5.39-3 [117 kB] Get: 11 http://deb.debian.org/debian bullseye/main armhf file armhf 1:5.39-3 [68.1 kB] Get: 12 http://deb.debian.org/debian bullseye/main armhf gettext-base armhf 0.21-4 [171 kB] Get: 13 http://deb.debian.org/debian bullseye/main armhf libsigsegv2 armhf 2.13-1 [34.0 kB] Get: 14 http://deb.debian.org/debian bullseye/main armhf m4 armhf 1.4.18-5 [192 kB] Get: 15 http://deb.debian.org/debian bullseye/main armhf autoconf all 2.69-14 [313 kB] Get: 16 http://deb.debian.org/debian bullseye/main armhf autotools-dev all 20180224.1+nmu1 [77.1 kB] Get: 17 http://deb.debian.org/debian bullseye/main armhf automake all 1:1.16.3-2 [814 kB] Get: 18 http://deb.debian.org/debian bullseye/main armhf autopoint all 0.21-4 [510 kB] Get: 19 http://deb.debian.org/debian bullseye/main armhf libdebhelper-perl all 13.3.4 [189 kB] Get: 20 http://deb.debian.org/debian bullseye/main armhf libtool all 2.4.6-15 [513 kB] Get: 21 http://deb.debian.org/debian bullseye/main armhf dh-autoreconf all 20 [17.1 kB] Get: 22 http://deb.debian.org/debian bullseye/main armhf libarchive-zip-perl all 1.68-1 [104 kB] Get: 23 http://deb.debian.org/debian bullseye/main armhf libsub-override-perl all 0.09-2 [10.2 kB] Get: 24 http://deb.debian.org/debian bullseye/main armhf libfile-stripnondeterminism-perl all 1.12.0-1 [26.3 kB] Get: 25 http://deb.debian.org/debian bullseye/main armhf dh-strip-nondeterminism all 1.12.0-1 [15.4 kB] Get: 26 http://deb.debian.org/debian bullseye/main armhf libelf1 armhf 0.183-1 [161 kB] Get: 27 http://deb.debian.org/debian bullseye/main armhf dwz armhf 0.13+20210201-1 [179 kB] Get: 28 http://deb.debian.org/debian bullseye/main armhf libicu67 armhf 67.1-7 [8319 kB] Get: 29 http://deb.debian.org/debian bullseye/main armhf libxml2 armhf 2.9.10+dfsg-6.7 [602 kB] Get: 30 http://deb.debian.org/debian bullseye/main armhf gettext armhf 0.21-4 [1243 kB] Get: 31 http://deb.debian.org/debian bullseye/main armhf intltool-debian all 0.35.0+20060710.5 [26.8 kB] Get: 32 http://deb.debian.org/debian bullseye/main armhf po-debconf all 1.0.21+nmu1 [248 kB] Get: 33 http://deb.debian.org/debian bullseye/main armhf debhelper all 13.3.4 [1049 kB] Get: 34 http://deb.debian.org/debian bullseye/main armhf dh-golang all 1.51 [24.6 kB] Get: 35 http://deb.debian.org/debian bullseye/main armhf golang-1.15-src armhf 1.15.9-6 [13.8 MB] Get: 36 http://deb.debian.org/debian bullseye/main armhf golang-1.15-go armhf 1.15.9-6 [43.1 MB] Get: 37 http://deb.debian.org/debian bullseye/main armhf golang-src armhf 2:1.15~1 [4856 B] Get: 38 http://deb.debian.org/debian bullseye/main armhf golang-go armhf 2:1.15~1 [23.9 kB] Get: 39 http://deb.debian.org/debian bullseye/main armhf golang-any armhf 2:1.15~1 [4976 B] Get: 40 http://deb.debian.org/debian bullseye/main armhf golang-github-alecthomas-units-dev all 0.0~git20201120.1786d5e-1 [7504 B] Get: 41 http://deb.debian.org/debian bullseye/main armhf golang-github-circonus-labs-circonusllhist-dev all 0.0~git20191022.ec08cde-1 [11.1 kB] Get: 42 http://deb.debian.org/debian bullseye/main armhf golang-github-hashicorp-go-cleanhttp-dev all 0.5.2-1 [10.8 kB] Get: 43 http://deb.debian.org/debian bullseye/main armhf golang-github-mattn-go-isatty-dev all 0.0.12-1 [6472 B] Get: 44 http://deb.debian.org/debian bullseye/main armhf golang-github-mattn-go-colorable-dev all 0.1.7-1 [9936 B] Get: 45 http://deb.debian.org/debian bullseye/main armhf golang-github-fatih-color-dev all 1.7.0-1 [11.4 kB] Get: 46 http://deb.debian.org/debian bullseye/main armhf golang-github-davecgh-go-spew-dev all 1.1.1-2 [29.7 kB] Get: 47 http://deb.debian.org/debian bullseye/main armhf golang-github-pmezard-go-difflib-dev all 1.0.0-3 [12.3 kB] Get: 48 http://deb.debian.org/debian bullseye/main armhf golang-github-stretchr-objx-dev all 0.3.0-1 [25.4 kB] Get: 49 http://deb.debian.org/debian bullseye/main armhf golang-gopkg-yaml.v3-dev all 3.0.0~git20200121.a6ecf24-3 [70.5 kB] Get: 50 http://deb.debian.org/debian bullseye/main armhf golang-github-stretchr-testify-dev all 1.6.1-2 [60.4 kB] Get: 51 http://deb.debian.org/debian bullseye/main armhf golang-github-hashicorp-go-hclog-dev all 0.11.0-1 [17.9 kB] Get: 52 http://deb.debian.org/debian bullseye/main armhf golang-github-hashicorp-go-retryablehttp-dev all 0.6.4-2 [17.5 kB] Get: 53 http://deb.debian.org/debian bullseye/main armhf golang-github-pkg-errors-dev all 0.9.1-1 [13.0 kB] Get: 54 http://deb.debian.org/debian bullseye/main armhf golang-github-tv42-httpunix-dev all 0.0~git20150427.b75d861-2 [3744 B] Get: 55 http://deb.debian.org/debian bullseye/main armhf golang-github-circonus-labs-circonus-gometrics-dev all 2.3.1-3 [65.9 kB] Get: 56 http://deb.debian.org/debian bullseye/main armhf golang-github-datadog-datadog-go-dev all 2.1.0-3 [14.9 kB] Get: 57 http://deb.debian.org/debian bullseye/main armhf golang-github-hashicorp-go-uuid-dev all 1.0.2-1 [8764 B] Get: 58 http://deb.debian.org/debian bullseye/main armhf golang-github-hashicorp-golang-lru-dev all 0.5.4-2 [14.7 kB] Get: 59 http://deb.debian.org/debian bullseye/main armhf golang-github-hashicorp-go-immutable-radix-dev all 1.3.0-1 [25.0 kB] Get: 60 http://deb.debian.org/debian bullseye/main armhf golang-github-pascaldekloe-goe-dev all 0.1.0-2 [21.7 kB] Get: 61 http://deb.debian.org/debian bullseye/main armhf golang-github-beorn7-perks-dev all 1.0.1-1 [12.1 kB] Get: 62 http://deb.debian.org/debian bullseye/main armhf golang-github-cespare-xxhash-dev all 2.1.1-1 [8748 B] Get: 63 http://deb.debian.org/debian bullseye/main armhf golang-github-modern-go-concurrent-dev all 1.0.3-1.1 [4668 B] Get: 64 http://deb.debian.org/debian bullseye/main armhf golang-github-modern-go-reflect2-dev all 1.0.1-1 [10.6 kB] Get: 65 http://deb.debian.org/debian bullseye/main armhf golang-github-json-iterator-go-dev all 1.1.10-2 [69.4 kB] Get: 66 http://deb.debian.org/debian bullseye/main armhf zlib1g-dev armhf 1:1.2.11.dfsg-2 [185 kB] Get: 67 http://deb.debian.org/debian bullseye/main armhf libprotobuf23 armhf 3.12.4-1 [777 kB] Get: 68 http://deb.debian.org/debian bullseye/main armhf libprotobuf-lite23 armhf 3.12.4-1 [217 kB] Get: 69 http://deb.debian.org/debian bullseye/main armhf libprotobuf-dev armhf 3.12.4-1 [1192 kB] Get: 70 http://deb.debian.org/debian bullseye/main armhf libprotoc23 armhf 3.12.4-1 [704 kB] Get: 71 http://deb.debian.org/debian bullseye/main armhf protobuf-compiler armhf 3.12.4-1 [74.7 kB] Get: 72 http://deb.debian.org/debian bullseye/main armhf golang-goprotobuf-dev armhf 1.3.4-2+b6 [1334 kB] Get: 73 http://deb.debian.org/debian bullseye/main armhf golang-github-prometheus-client-model-dev all 0.2.0-2 [8476 B] Get: 74 http://deb.debian.org/debian bullseye/main armhf golang-github-dgrijalva-jwt-go-dev all 3.2.0-3 [33.6 kB] Get: 75 http://deb.debian.org/debian bullseye/main armhf golang-github-go-logfmt-logfmt-dev all 0.5.0-2 [13.4 kB] Get: 76 http://deb.debian.org/debian bullseye/main armhf golang-github-go-stack-stack-dev all 1.8.0-1 [8696 B] Get: 77 http://deb.debian.org/debian bullseye/main armhf golang-github-opentracing-opentracing-go-dev all 1.2.0-1 [29.4 kB] Get: 78 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-text-dev all 0.3.6-1 [3857 kB] Get: 79 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-net-dev all 1:0.0+git20210119.5f4716e+dfsg-4 [659 kB] Get: 80 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-time-dev all 0.0+git20200630.3af7569-1 [10.1 kB] Get: 81 http://deb.debian.org/debian bullseye/main armhf golang-github-golang-mock-dev all 1.3.1-2 [35.1 kB] Get: 82 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-xerrors-dev all 0.0~git20191204.9bdfabe-1 [14.2 kB] Get: 83 http://deb.debian.org/debian bullseye/main armhf golang-github-google-go-cmp-dev all 0.5.4-1 [83.7 kB] Get: 84 http://deb.debian.org/debian bullseye/main armhf golang-glog-dev all 0.0~git20160126.23def4e-3 [17.3 kB] Get: 85 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-oauth2-dev all 0.0~git20190604.0f29369-2 [31.9 kB] Get: 86 http://deb.debian.org/debian bullseye/main armhf golang-google-cloud-compute-metadata-dev all 0.56.0-1 [34.0 kB] Get: 87 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-oauth2-google-dev all 0.0~git20190604.0f29369-2 [13.2 kB] Get: 88 http://deb.debian.org/debian bullseye/main armhf golang-google-genproto-dev all 0.0~git20200413.b5235f6-1 [4037 kB] Get: 89 http://deb.debian.org/debian bullseye/main armhf golang-google-grpc-dev all 1.27.1-1 [478 kB] Get: 90 http://deb.debian.org/debian bullseye/main armhf golang-github-go-kit-kit-dev all 0.6.0-3 [103 kB] Get: 91 http://deb.debian.org/debian bullseye/main armhf golang-github-julienschmidt-httprouter-dev all 1.3.0-1 [19.3 kB] Get: 92 http://deb.debian.org/debian bullseye/main armhf golang-github-jpillora-backoff-dev all 1.0.0-1.1 [3712 B] Get: 93 http://deb.debian.org/debian bullseye/main armhf golang-github-mwitkow-go-conntrack-dev all 0.0~git20190716.2f06839-2 [14.6 kB] Get: 94 http://deb.debian.org/debian bullseye/main armhf golang-gopkg-alecthomas-kingpin.v2-dev all 2.2.6-3 [42.8 kB] Get: 95 http://deb.debian.org/debian bullseye/main armhf golang-gopkg-yaml.v2-dev all 2.4.0-1 [61.3 kB] Get: 96 http://deb.debian.org/debian bullseye/main armhf golang-protobuf-extensions-dev all 1.0.1-1 [29.6 kB] Get: 97 http://deb.debian.org/debian bullseye/main armhf golang-github-prometheus-common-dev all 0.15.0-2 [102 kB] Get: 98 http://deb.debian.org/debian bullseye/main armhf golang-golang-x-sync-dev all 0.0~git20210220.036812b-1 [19.1 kB] Get: 99 http://deb.debian.org/debian bullseye/main armhf golang-github-prometheus-procfs-dev all 0.3.0-2 [110 kB] Get: 100 http://deb.debian.org/debian bullseye/main armhf golang-github-prometheus-client-golang-dev all 1.9.0-2 [134 kB] Get: 101 http://deb.debian.org/debian bullseye/main armhf golang-github-armon-go-metrics-dev all 0.3.4-1 [30.5 kB] Get: 102 http://deb.debian.org/debian bullseye/main armhf golang-github-creack-pty-dev all 1.1.11-1 [8380 B] Get: 103 http://deb.debian.org/debian bullseye/main armhf golang-github-vmihailenco-tagparser-dev all 0.1.1-2 [4440 B] Get: 104 http://deb.debian.org/debian bullseye/main armhf golang-gopkg-vmihailenco-msgpack.v2-dev all 4.3.1-2 [27.5 kB] Get: 105 http://deb.debian.org/debian bullseye/main armhf golang-gopkg-tomb.v2-dev all 0.0~git20161208.d5d1b58-3 [6840 B] Get: 106 http://deb.debian.org/debian bullseye/main armhf libsasl2-modules-db armhf 2.1.27+dfsg-2.1 [67.6 kB] Get: 107 http://deb.debian.org/debian bullseye/main armhf libsasl2-2 armhf 2.1.27+dfsg-2.1 [99.1 kB] Get: 108 http://deb.debian.org/debian bullseye/main armhf libsasl2-dev armhf 2.1.27+dfsg-2.1 [267 kB] Get: 109 http://deb.debian.org/debian bullseye/main armhf golang-github-kr-text-dev all 0.2.0-1 [11.1 kB] Get: 110 http://deb.debian.org/debian bullseye/main armhf golang-github-kr-pretty-dev all 0.2.1+git20200831.59b4212-1 [14.6 kB] Get: 111 http://deb.debian.org/debian bullseye/main armhf golang-gopkg-check.v1-dev all 0.0+git20200902.038fdea-1 [32.1 kB] Get: 112 http://deb.debian.org/debian bullseye/main armhf golang-gopkg-mgo.v2-dev all 2016.08.01-7 [316 kB] Get: 113 http://deb.debian.org/debian bullseye/main armhf golang-github-hashicorp-go-msgpack-dev all 0.5.5-1 [43.3 kB] Fetched 91.1 MB in 9s (9830 kB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package bsdextrautils. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 19398 files and directories currently installed.) Preparing to unpack .../000-bsdextrautils_2.36.1-8_armhf.deb ... Unpacking bsdextrautils (2.36.1-8) ... Selecting previously unselected package libuchardet0:armhf. Preparing to unpack .../001-libuchardet0_0.0.7-1_armhf.deb ... Unpacking libuchardet0:armhf (0.0.7-1) ... Selecting previously unselected package groff-base. Preparing to unpack .../002-groff-base_1.22.4-6_armhf.deb ... Unpacking groff-base (1.22.4-6) ... Selecting previously unselected package libpipeline1:armhf. Preparing to unpack .../003-libpipeline1_1.5.3-1_armhf.deb ... Unpacking libpipeline1:armhf (1.5.3-1) ... Selecting previously unselected package man-db. Preparing to unpack .../004-man-db_2.9.4-2_armhf.deb ... Unpacking man-db (2.9.4-2) ... Selecting previously unselected package golang-golang-x-sys-dev. Preparing to unpack .../005-golang-golang-x-sys-dev_0.0~git20210124.22da62e-1_all.deb ... Unpacking golang-golang-x-sys-dev (0.0~git20210124.22da62e-1) ... Selecting previously unselected package golang-github-sirupsen-logrus-dev. Preparing to unpack .../006-golang-github-sirupsen-logrus-dev_1.7.0-2_all.deb ... Unpacking golang-github-sirupsen-logrus-dev (1.7.0-2) ... Selecting previously unselected package sensible-utils. Preparing to unpack .../007-sensible-utils_0.0.14_all.deb ... Unpacking sensible-utils (0.0.14) ... Selecting previously unselected package libmagic-mgc. Preparing to unpack .../008-libmagic-mgc_1%3a5.39-3_armhf.deb ... Unpacking libmagic-mgc (1:5.39-3) ... Selecting previously unselected package libmagic1:armhf. Preparing to unpack .../009-libmagic1_1%3a5.39-3_armhf.deb ... Unpacking libmagic1:armhf (1:5.39-3) ... Selecting previously unselected package file. Preparing to unpack .../010-file_1%3a5.39-3_armhf.deb ... Unpacking file (1:5.39-3) ... Selecting previously unselected package gettext-base. Preparing to unpack .../011-gettext-base_0.21-4_armhf.deb ... Unpacking gettext-base (0.21-4) ... Selecting previously unselected package libsigsegv2:armhf. Preparing to unpack .../012-libsigsegv2_2.13-1_armhf.deb ... Unpacking libsigsegv2:armhf (2.13-1) ... Selecting previously unselected package m4. Preparing to unpack .../013-m4_1.4.18-5_armhf.deb ... Unpacking m4 (1.4.18-5) ... Selecting previously unselected package autoconf. Preparing to unpack .../014-autoconf_2.69-14_all.deb ... Unpacking autoconf (2.69-14) ... Selecting previously unselected package autotools-dev. Preparing to unpack .../015-autotools-dev_20180224.1+nmu1_all.deb ... Unpacking autotools-dev (20180224.1+nmu1) ... Selecting previously unselected package automake. Preparing to unpack .../016-automake_1%3a1.16.3-2_all.deb ... Unpacking automake (1:1.16.3-2) ... Selecting previously unselected package autopoint. Preparing to unpack .../017-autopoint_0.21-4_all.deb ... Unpacking autopoint (0.21-4) ... Selecting previously unselected package libdebhelper-perl. Preparing to unpack .../018-libdebhelper-perl_13.3.4_all.deb ... Unpacking libdebhelper-perl (13.3.4) ... Selecting previously unselected package libtool. Preparing to unpack .../019-libtool_2.4.6-15_all.deb ... Unpacking libtool (2.4.6-15) ... Selecting previously unselected package dh-autoreconf. Preparing to unpack .../020-dh-autoreconf_20_all.deb ... Unpacking dh-autoreconf (20) ... Selecting previously unselected package libarchive-zip-perl. Preparing to unpack .../021-libarchive-zip-perl_1.68-1_all.deb ... Unpacking libarchive-zip-perl (1.68-1) ... Selecting previously unselected package libsub-override-perl. Preparing to unpack .../022-libsub-override-perl_0.09-2_all.deb ... Unpacking libsub-override-perl (0.09-2) ... Selecting previously unselected package libfile-stripnondeterminism-perl. Preparing to unpack .../023-libfile-stripnondeterminism-perl_1.12.0-1_all.deb ... Unpacking libfile-stripnondeterminism-perl (1.12.0-1) ... Selecting previously unselected package dh-strip-nondeterminism. Preparing to unpack .../024-dh-strip-nondeterminism_1.12.0-1_all.deb ... Unpacking dh-strip-nondeterminism (1.12.0-1) ... Selecting previously unselected package libelf1:armhf. Preparing to unpack .../025-libelf1_0.183-1_armhf.deb ... Unpacking libelf1:armhf (0.183-1) ... Selecting previously unselected package dwz. Preparing to unpack .../026-dwz_0.13+20210201-1_armhf.deb ... Unpacking dwz (0.13+20210201-1) ... Selecting previously unselected package libicu67:armhf. Preparing to unpack .../027-libicu67_67.1-7_armhf.deb ... Unpacking libicu67:armhf (67.1-7) ... Selecting previously unselected package libxml2:armhf. Preparing to unpack .../028-libxml2_2.9.10+dfsg-6.7_armhf.deb ... Unpacking libxml2:armhf (2.9.10+dfsg-6.7) ... Selecting previously unselected package gettext. Preparing to unpack .../029-gettext_0.21-4_armhf.deb ... Unpacking gettext (0.21-4) ... Selecting previously unselected package intltool-debian. Preparing to unpack .../030-intltool-debian_0.35.0+20060710.5_all.deb ... Unpacking intltool-debian (0.35.0+20060710.5) ... Selecting previously unselected package po-debconf. Preparing to unpack .../031-po-debconf_1.0.21+nmu1_all.deb ... Unpacking po-debconf (1.0.21+nmu1) ... Selecting previously unselected package debhelper. Preparing to unpack .../032-debhelper_13.3.4_all.deb ... Unpacking debhelper (13.3.4) ... Selecting previously unselected package dh-golang. Preparing to unpack .../033-dh-golang_1.51_all.deb ... Unpacking dh-golang (1.51) ... Selecting previously unselected package golang-1.15-src. Preparing to unpack .../034-golang-1.15-src_1.15.9-6_armhf.deb ... Unpacking golang-1.15-src (1.15.9-6) ... Selecting previously unselected package golang-1.15-go. Preparing to unpack .../035-golang-1.15-go_1.15.9-6_armhf.deb ... Unpacking golang-1.15-go (1.15.9-6) ... Selecting previously unselected package golang-src:armhf. Preparing to unpack .../036-golang-src_2%3a1.15~1_armhf.deb ... Unpacking golang-src:armhf (2:1.15~1) ... Selecting previously unselected package golang-go. Preparing to unpack .../037-golang-go_2%3a1.15~1_armhf.deb ... Unpacking golang-go (2:1.15~1) ... Selecting previously unselected package golang-any:armhf. Preparing to unpack .../038-golang-any_2%3a1.15~1_armhf.deb ... Unpacking golang-any:armhf (2:1.15~1) ... Selecting previously unselected package golang-github-alecthomas-units-dev. Preparing to unpack .../039-golang-github-alecthomas-units-dev_0.0~git20201120.1786d5e-1_all.deb ... Unpacking golang-github-alecthomas-units-dev (0.0~git20201120.1786d5e-1) ... Selecting previously unselected package golang-github-circonus-labs-circonusllhist-dev. Preparing to unpack .../040-golang-github-circonus-labs-circonusllhist-dev_0.0~git20191022.ec08cde-1_all.deb ... Unpacking golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ... Selecting previously unselected package golang-github-hashicorp-go-cleanhttp-dev. Preparing to unpack .../041-golang-github-hashicorp-go-cleanhttp-dev_0.5.2-1_all.deb ... Unpacking golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ... Selecting previously unselected package golang-github-mattn-go-isatty-dev. Preparing to unpack .../042-golang-github-mattn-go-isatty-dev_0.0.12-1_all.deb ... Unpacking golang-github-mattn-go-isatty-dev (0.0.12-1) ... Selecting previously unselected package golang-github-mattn-go-colorable-dev. Preparing to unpack .../043-golang-github-mattn-go-colorable-dev_0.1.7-1_all.deb ... Unpacking golang-github-mattn-go-colorable-dev (0.1.7-1) ... Selecting previously unselected package golang-github-fatih-color-dev. Preparing to unpack .../044-golang-github-fatih-color-dev_1.7.0-1_all.deb ... Unpacking golang-github-fatih-color-dev (1.7.0-1) ... Selecting previously unselected package golang-github-davecgh-go-spew-dev. Preparing to unpack .../045-golang-github-davecgh-go-spew-dev_1.1.1-2_all.deb ... Unpacking golang-github-davecgh-go-spew-dev (1.1.1-2) ... Selecting previously unselected package golang-github-pmezard-go-difflib-dev. Preparing to unpack .../046-golang-github-pmezard-go-difflib-dev_1.0.0-3_all.deb ... Unpacking golang-github-pmezard-go-difflib-dev (1.0.0-3) ... Selecting previously unselected package golang-github-stretchr-objx-dev. Preparing to unpack .../047-golang-github-stretchr-objx-dev_0.3.0-1_all.deb ... Unpacking golang-github-stretchr-objx-dev (0.3.0-1) ... Selecting previously unselected package golang-gopkg-yaml.v3-dev. Preparing to unpack .../048-golang-gopkg-yaml.v3-dev_3.0.0~git20200121.a6ecf24-3_all.deb ... Unpacking golang-gopkg-yaml.v3-dev (3.0.0~git20200121.a6ecf24-3) ... Selecting previously unselected package golang-github-stretchr-testify-dev. Preparing to unpack .../049-golang-github-stretchr-testify-dev_1.6.1-2_all.deb ... Unpacking golang-github-stretchr-testify-dev (1.6.1-2) ... Selecting previously unselected package golang-github-hashicorp-go-hclog-dev. Preparing to unpack .../050-golang-github-hashicorp-go-hclog-dev_0.11.0-1_all.deb ... Unpacking golang-github-hashicorp-go-hclog-dev (0.11.0-1) ... Selecting previously unselected package golang-github-hashicorp-go-retryablehttp-dev. Preparing to unpack .../051-golang-github-hashicorp-go-retryablehttp-dev_0.6.4-2_all.deb ... Unpacking golang-github-hashicorp-go-retryablehttp-dev (0.6.4-2) ... Selecting previously unselected package golang-github-pkg-errors-dev. Preparing to unpack .../052-golang-github-pkg-errors-dev_0.9.1-1_all.deb ... Unpacking golang-github-pkg-errors-dev (0.9.1-1) ... Selecting previously unselected package golang-github-tv42-httpunix-dev. Preparing to unpack .../053-golang-github-tv42-httpunix-dev_0.0~git20150427.b75d861-2_all.deb ... Unpacking golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-2) ... Selecting previously unselected package golang-github-circonus-labs-circonus-gometrics-dev. Preparing to unpack .../054-golang-github-circonus-labs-circonus-gometrics-dev_2.3.1-3_all.deb ... Unpacking golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-3) ... Selecting previously unselected package golang-github-datadog-datadog-go-dev. Preparing to unpack .../055-golang-github-datadog-datadog-go-dev_2.1.0-3_all.deb ... Unpacking golang-github-datadog-datadog-go-dev (2.1.0-3) ... Selecting previously unselected package golang-github-hashicorp-go-uuid-dev. Preparing to unpack .../056-golang-github-hashicorp-go-uuid-dev_1.0.2-1_all.deb ... Unpacking golang-github-hashicorp-go-uuid-dev (1.0.2-1) ... Selecting previously unselected package golang-github-hashicorp-golang-lru-dev. Preparing to unpack .../057-golang-github-hashicorp-golang-lru-dev_0.5.4-2_all.deb ... Unpacking golang-github-hashicorp-golang-lru-dev (0.5.4-2) ... Selecting previously unselected package golang-github-hashicorp-go-immutable-radix-dev. Preparing to unpack .../058-golang-github-hashicorp-go-immutable-radix-dev_1.3.0-1_all.deb ... Unpacking golang-github-hashicorp-go-immutable-radix-dev (1.3.0-1) ... Selecting previously unselected package golang-github-pascaldekloe-goe-dev. Preparing to unpack .../059-golang-github-pascaldekloe-goe-dev_0.1.0-2_all.deb ... Unpacking golang-github-pascaldekloe-goe-dev (0.1.0-2) ... Selecting previously unselected package golang-github-beorn7-perks-dev. Preparing to unpack .../060-golang-github-beorn7-perks-dev_1.0.1-1_all.deb ... Unpacking golang-github-beorn7-perks-dev (1.0.1-1) ... Selecting previously unselected package golang-github-cespare-xxhash-dev. Preparing to unpack .../061-golang-github-cespare-xxhash-dev_2.1.1-1_all.deb ... Unpacking golang-github-cespare-xxhash-dev (2.1.1-1) ... Selecting previously unselected package golang-github-modern-go-concurrent-dev. Preparing to unpack .../062-golang-github-modern-go-concurrent-dev_1.0.3-1.1_all.deb ... Unpacking golang-github-modern-go-concurrent-dev (1.0.3-1.1) ... Selecting previously unselected package golang-github-modern-go-reflect2-dev. Preparing to unpack .../063-golang-github-modern-go-reflect2-dev_1.0.1-1_all.deb ... Unpacking golang-github-modern-go-reflect2-dev (1.0.1-1) ... Selecting previously unselected package golang-github-json-iterator-go-dev. Preparing to unpack .../064-golang-github-json-iterator-go-dev_1.1.10-2_all.deb ... Unpacking golang-github-json-iterator-go-dev (1.1.10-2) ... Selecting previously unselected package zlib1g-dev:armhf. Preparing to unpack .../065-zlib1g-dev_1%3a1.2.11.dfsg-2_armhf.deb ... Unpacking zlib1g-dev:armhf (1:1.2.11.dfsg-2) ... Selecting previously unselected package libprotobuf23:armhf. Preparing to unpack .../066-libprotobuf23_3.12.4-1_armhf.deb ... Unpacking libprotobuf23:armhf (3.12.4-1) ... Selecting previously unselected package libprotobuf-lite23:armhf. Preparing to unpack .../067-libprotobuf-lite23_3.12.4-1_armhf.deb ... Unpacking libprotobuf-lite23:armhf (3.12.4-1) ... Selecting previously unselected package libprotobuf-dev:armhf. Preparing to unpack .../068-libprotobuf-dev_3.12.4-1_armhf.deb ... Unpacking libprotobuf-dev:armhf (3.12.4-1) ... Selecting previously unselected package libprotoc23:armhf. Preparing to unpack .../069-libprotoc23_3.12.4-1_armhf.deb ... Unpacking libprotoc23:armhf (3.12.4-1) ... Selecting previously unselected package protobuf-compiler. Preparing to unpack .../070-protobuf-compiler_3.12.4-1_armhf.deb ... Unpacking protobuf-compiler (3.12.4-1) ... Selecting previously unselected package golang-goprotobuf-dev. Preparing to unpack .../071-golang-goprotobuf-dev_1.3.4-2+b6_armhf.deb ... Unpacking golang-goprotobuf-dev (1.3.4-2+b6) ... Selecting previously unselected package golang-github-prometheus-client-model-dev. Preparing to unpack .../072-golang-github-prometheus-client-model-dev_0.2.0-2_all.deb ... Unpacking golang-github-prometheus-client-model-dev (0.2.0-2) ... Selecting previously unselected package golang-github-dgrijalva-jwt-go-dev. Preparing to unpack .../073-golang-github-dgrijalva-jwt-go-dev_3.2.0-3_all.deb ... Unpacking golang-github-dgrijalva-jwt-go-dev (3.2.0-3) ... Selecting previously unselected package golang-github-go-logfmt-logfmt-dev. Preparing to unpack .../074-golang-github-go-logfmt-logfmt-dev_0.5.0-2_all.deb ... Unpacking golang-github-go-logfmt-logfmt-dev (0.5.0-2) ... Selecting previously unselected package golang-github-go-stack-stack-dev. Preparing to unpack .../075-golang-github-go-stack-stack-dev_1.8.0-1_all.deb ... Unpacking golang-github-go-stack-stack-dev (1.8.0-1) ... Selecting previously unselected package golang-github-opentracing-opentracing-go-dev. Preparing to unpack .../076-golang-github-opentracing-opentracing-go-dev_1.2.0-1_all.deb ... Unpacking golang-github-opentracing-opentracing-go-dev (1.2.0-1) ... Selecting previously unselected package golang-golang-x-text-dev. Preparing to unpack .../077-golang-golang-x-text-dev_0.3.6-1_all.deb ... Unpacking golang-golang-x-text-dev (0.3.6-1) ... Selecting previously unselected package golang-golang-x-net-dev. Preparing to unpack .../078-golang-golang-x-net-dev_1%3a0.0+git20210119.5f4716e+dfsg-4_all.deb ... Unpacking golang-golang-x-net-dev (1:0.0+git20210119.5f4716e+dfsg-4) ... Selecting previously unselected package golang-golang-x-time-dev. Preparing to unpack .../079-golang-golang-x-time-dev_0.0+git20200630.3af7569-1_all.deb ... Unpacking golang-golang-x-time-dev (0.0+git20200630.3af7569-1) ... Selecting previously unselected package golang-github-golang-mock-dev. Preparing to unpack .../080-golang-github-golang-mock-dev_1.3.1-2_all.deb ... Unpacking golang-github-golang-mock-dev (1.3.1-2) ... Selecting previously unselected package golang-golang-x-xerrors-dev. Preparing to unpack .../081-golang-golang-x-xerrors-dev_0.0~git20191204.9bdfabe-1_all.deb ... Unpacking golang-golang-x-xerrors-dev (0.0~git20191204.9bdfabe-1) ... Selecting previously unselected package golang-github-google-go-cmp-dev. Preparing to unpack .../082-golang-github-google-go-cmp-dev_0.5.4-1_all.deb ... Unpacking golang-github-google-go-cmp-dev (0.5.4-1) ... Selecting previously unselected package golang-glog-dev. Preparing to unpack .../083-golang-glog-dev_0.0~git20160126.23def4e-3_all.deb ... Unpacking golang-glog-dev (0.0~git20160126.23def4e-3) ... Selecting previously unselected package golang-golang-x-oauth2-dev. Preparing to unpack .../084-golang-golang-x-oauth2-dev_0.0~git20190604.0f29369-2_all.deb ... Unpacking golang-golang-x-oauth2-dev (0.0~git20190604.0f29369-2) ... Selecting previously unselected package golang-google-cloud-compute-metadata-dev. Preparing to unpack .../085-golang-google-cloud-compute-metadata-dev_0.56.0-1_all.deb ... Unpacking golang-google-cloud-compute-metadata-dev (0.56.0-1) ... Selecting previously unselected package golang-golang-x-oauth2-google-dev. Preparing to unpack .../086-golang-golang-x-oauth2-google-dev_0.0~git20190604.0f29369-2_all.deb ... Unpacking golang-golang-x-oauth2-google-dev (0.0~git20190604.0f29369-2) ... Selecting previously unselected package golang-google-genproto-dev. Preparing to unpack .../087-golang-google-genproto-dev_0.0~git20200413.b5235f6-1_all.deb ... Unpacking golang-google-genproto-dev (0.0~git20200413.b5235f6-1) ... Selecting previously unselected package golang-google-grpc-dev. Preparing to unpack .../088-golang-google-grpc-dev_1.27.1-1_all.deb ... Unpacking golang-google-grpc-dev (1.27.1-1) ... Selecting previously unselected package golang-github-go-kit-kit-dev. Preparing to unpack .../089-golang-github-go-kit-kit-dev_0.6.0-3_all.deb ... Unpacking golang-github-go-kit-kit-dev (0.6.0-3) ... Selecting previously unselected package golang-github-julienschmidt-httprouter-dev. Preparing to unpack .../090-golang-github-julienschmidt-httprouter-dev_1.3.0-1_all.deb ... Unpacking golang-github-julienschmidt-httprouter-dev (1.3.0-1) ... Selecting previously unselected package golang-github-jpillora-backoff-dev. Preparing to unpack .../091-golang-github-jpillora-backoff-dev_1.0.0-1.1_all.deb ... Unpacking golang-github-jpillora-backoff-dev (1.0.0-1.1) ... Selecting previously unselected package golang-github-mwitkow-go-conntrack-dev. Preparing to unpack .../092-golang-github-mwitkow-go-conntrack-dev_0.0~git20190716.2f06839-2_all.deb ... Unpacking golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-2) ... Selecting previously unselected package golang-gopkg-alecthomas-kingpin.v2-dev. Preparing to unpack .../093-golang-gopkg-alecthomas-kingpin.v2-dev_2.2.6-3_all.deb ... Unpacking golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-3) ... Selecting previously unselected package golang-gopkg-yaml.v2-dev. Preparing to unpack .../094-golang-gopkg-yaml.v2-dev_2.4.0-1_all.deb ... Unpacking golang-gopkg-yaml.v2-dev (2.4.0-1) ... Selecting previously unselected package golang-protobuf-extensions-dev. Preparing to unpack .../095-golang-protobuf-extensions-dev_1.0.1-1_all.deb ... Unpacking golang-protobuf-extensions-dev (1.0.1-1) ... Selecting previously unselected package golang-github-prometheus-common-dev. Preparing to unpack .../096-golang-github-prometheus-common-dev_0.15.0-2_all.deb ... Unpacking golang-github-prometheus-common-dev (0.15.0-2) ... Selecting previously unselected package golang-golang-x-sync-dev. Preparing to unpack .../097-golang-golang-x-sync-dev_0.0~git20210220.036812b-1_all.deb ... Unpacking golang-golang-x-sync-dev (0.0~git20210220.036812b-1) ... Selecting previously unselected package golang-github-prometheus-procfs-dev. Preparing to unpack .../098-golang-github-prometheus-procfs-dev_0.3.0-2_all.deb ... Unpacking golang-github-prometheus-procfs-dev (0.3.0-2) ... Selecting previously unselected package golang-github-prometheus-client-golang-dev. Preparing to unpack .../099-golang-github-prometheus-client-golang-dev_1.9.0-2_all.deb ... Unpacking golang-github-prometheus-client-golang-dev (1.9.0-2) ... Selecting previously unselected package golang-github-armon-go-metrics-dev. Preparing to unpack .../100-golang-github-armon-go-metrics-dev_0.3.4-1_all.deb ... Unpacking golang-github-armon-go-metrics-dev (0.3.4-1) ... Selecting previously unselected package golang-github-creack-pty-dev. Preparing to unpack .../101-golang-github-creack-pty-dev_1.1.11-1_all.deb ... Unpacking golang-github-creack-pty-dev (1.1.11-1) ... Selecting previously unselected package golang-github-vmihailenco-tagparser-dev. Preparing to unpack .../102-golang-github-vmihailenco-tagparser-dev_0.1.1-2_all.deb ... Unpacking golang-github-vmihailenco-tagparser-dev (0.1.1-2) ... Selecting previously unselected package golang-gopkg-vmihailenco-msgpack.v2-dev. Preparing to unpack .../103-golang-gopkg-vmihailenco-msgpack.v2-dev_4.3.1-2_all.deb ... Unpacking golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ... Selecting previously unselected package golang-gopkg-tomb.v2-dev. Preparing to unpack .../104-golang-gopkg-tomb.v2-dev_0.0~git20161208.d5d1b58-3_all.deb ... Unpacking golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ... Selecting previously unselected package libsasl2-modules-db:armhf. Preparing to unpack .../105-libsasl2-modules-db_2.1.27+dfsg-2.1_armhf.deb ... Unpacking libsasl2-modules-db:armhf (2.1.27+dfsg-2.1) ... Selecting previously unselected package libsasl2-2:armhf. Preparing to unpack .../106-libsasl2-2_2.1.27+dfsg-2.1_armhf.deb ... Unpacking libsasl2-2:armhf (2.1.27+dfsg-2.1) ... Selecting previously unselected package libsasl2-dev. Preparing to unpack .../107-libsasl2-dev_2.1.27+dfsg-2.1_armhf.deb ... Unpacking libsasl2-dev (2.1.27+dfsg-2.1) ... Selecting previously unselected package golang-github-kr-text-dev. Preparing to unpack .../108-golang-github-kr-text-dev_0.2.0-1_all.deb ... Unpacking golang-github-kr-text-dev (0.2.0-1) ... Selecting previously unselected package golang-github-kr-pretty-dev. Preparing to unpack .../109-golang-github-kr-pretty-dev_0.2.1+git20200831.59b4212-1_all.deb ... Unpacking golang-github-kr-pretty-dev (0.2.1+git20200831.59b4212-1) ... Selecting previously unselected package golang-gopkg-check.v1-dev. Preparing to unpack .../110-golang-gopkg-check.v1-dev_0.0+git20200902.038fdea-1_all.deb ... Unpacking golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ... Selecting previously unselected package golang-gopkg-mgo.v2-dev. Preparing to unpack .../111-golang-gopkg-mgo.v2-dev_2016.08.01-7_all.deb ... Unpacking golang-gopkg-mgo.v2-dev (2016.08.01-7) ... Selecting previously unselected package golang-github-hashicorp-go-msgpack-dev. Preparing to unpack .../112-golang-github-hashicorp-go-msgpack-dev_0.5.5-1_all.deb ... Unpacking golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ... Setting up libpipeline1:armhf (1.5.3-1) ... Setting up golang-gopkg-yaml.v2-dev (2.4.0-1) ... Setting up golang-github-hashicorp-go-uuid-dev (1.0.2-1) ... Setting up bsdextrautils (2.36.1-8) ... update-alternatives: using /usr/bin/write.ul to provide /usr/bin/write (write) in auto mode Setting up golang-github-creack-pty-dev (1.1.11-1) ... Setting up libicu67:armhf (67.1-7) ... Setting up golang-github-golang-mock-dev (1.3.1-2) ... Setting up golang-github-stretchr-objx-dev (0.3.0-1) ... Setting up libmagic-mgc (1:5.39-3) ... Setting up golang-github-pkg-errors-dev (0.9.1-1) ... Setting up golang-github-hashicorp-golang-lru-dev (0.5.4-2) ... Setting up golang-1.15-src (1.15.9-6) ... Setting up libarchive-zip-perl (1.68-1) ... Setting up golang-gopkg-yaml.v3-dev (3.0.0~git20200121.a6ecf24-3) ... Setting up libdebhelper-perl (13.3.4) ... Setting up golang-golang-x-sys-dev (0.0~git20210124.22da62e-1) ... Setting up libmagic1:armhf (1:5.39-3) ... Setting up golang-github-pmezard-go-difflib-dev (1.0.0-3) ... Setting up golang-github-modern-go-concurrent-dev (1.0.3-1.1) ... Setting up gettext-base (0.21-4) ... Setting up golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ... Setting up libprotobuf23:armhf (3.12.4-1) ... Setting up file (1:5.39-3) ... Setting up golang-github-go-stack-stack-dev (1.8.0-1) ... Setting up golang-github-beorn7-perks-dev (1.0.1-1) ... Setting up golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ... Setting up libsasl2-modules-db:armhf (2.1.27+dfsg-2.1) ... Setting up golang-github-cespare-xxhash-dev (2.1.1-1) ... Setting up golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ... Setting up golang-github-jpillora-backoff-dev (1.0.0-1.1) ... Setting up golang-github-davecgh-go-spew-dev (1.1.1-2) ... Setting up libprotobuf-lite23:armhf (3.12.4-1) ... Setting up autotools-dev (20180224.1+nmu1) ... Setting up golang-github-pascaldekloe-goe-dev (0.1.0-2) ... Setting up golang-github-go-logfmt-logfmt-dev (0.5.0-2) ... Setting up golang-github-kr-text-dev (0.2.0-1) ... Setting up golang-github-stretchr-testify-dev (1.6.1-2) ... Setting up golang-github-hashicorp-go-immutable-radix-dev (1.3.0-1) ... Setting up libprotoc23:armhf (3.12.4-1) ... Setting up libsigsegv2:armhf (2.13-1) ... Setting up golang-golang-x-sync-dev (0.0~git20210220.036812b-1) ... Setting up autopoint (0.21-4) ... Setting up golang-glog-dev (0.0~git20160126.23def4e-3) ... Setting up libsasl2-2:armhf (2.1.27+dfsg-2.1) ... Setting up golang-github-julienschmidt-httprouter-dev (1.3.0-1) ... Setting up zlib1g-dev:armhf (1:1.2.11.dfsg-2) ... Setting up golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-2) ... Setting up sensible-utils (0.0.14) ... Setting up golang-golang-x-text-dev (0.3.6-1) ... Setting up libuchardet0:armhf (0.0.7-1) ... Setting up golang-github-prometheus-procfs-dev (0.3.0-2) ... Setting up golang-github-alecthomas-units-dev (0.0~git20201120.1786d5e-1) ... Setting up golang-github-modern-go-reflect2-dev (1.0.1-1) ... Setting up libsub-override-perl (0.09-2) ... Setting up golang-github-dgrijalva-jwt-go-dev (3.2.0-3) ... Setting up golang-github-datadog-datadog-go-dev (2.1.0-3) ... Setting up golang-golang-x-xerrors-dev (0.0~git20191204.9bdfabe-1) ... Setting up golang-src:armhf (2:1.15~1) ... Setting up golang-1.15-go (1.15.9-6) ... Setting up libelf1:armhf (0.183-1) ... Setting up golang-github-vmihailenco-tagparser-dev (0.1.1-2) ... Setting up libxml2:armhf (2.9.10+dfsg-6.7) ... Setting up golang-github-kr-pretty-dev (0.2.1+git20200831.59b4212-1) ... Setting up libfile-stripnondeterminism-perl (1.12.0-1) ... Setting up golang-github-google-go-cmp-dev (0.5.4-1) ... Setting up golang-github-mattn-go-isatty-dev (0.0.12-1) ... Setting up golang-github-sirupsen-logrus-dev (1.7.0-2) ... Setting up golang-github-opentracing-opentracing-go-dev (1.2.0-1) ... Setting up gettext (0.21-4) ... Setting up protobuf-compiler (3.12.4-1) ... Setting up golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ... Setting up golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-3) ... Setting up libtool (2.4.6-15) ... Setting up golang-go (2:1.15~1) ... Setting up golang-github-mattn-go-colorable-dev (0.1.7-1) ... Setting up m4 (1.4.18-5) ... Setting up golang-github-json-iterator-go-dev (1.1.10-2) ... Setting up golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ... Setting up golang-any:armhf (2:1.15~1) ... Setting up intltool-debian (0.35.0+20060710.5) ... Setting up libprotobuf-dev:armhf (3.12.4-1) ... Setting up libsasl2-dev (2.1.27+dfsg-2.1) ... Setting up golang-gopkg-mgo.v2-dev (2016.08.01-7) ... Setting up golang-goprotobuf-dev (1.3.4-2+b6) ... Setting up golang-golang-x-net-dev (1:0.0+git20210119.5f4716e+dfsg-4) ... Setting up autoconf (2.69-14) ... Setting up dh-strip-nondeterminism (1.12.0-1) ... Setting up dwz (0.13+20210201-1) ... Setting up groff-base (1.22.4-6) ... Setting up golang-github-prometheus-client-model-dev (0.2.0-2) ... Setting up golang-golang-x-oauth2-dev (0.0~git20190604.0f29369-2) ... Setting up automake (1:1.16.3-2) ... update-alternatives: using /usr/bin/automake-1.16 to provide /usr/bin/automake (automake) in auto mode Setting up golang-golang-x-time-dev (0.0+git20200630.3af7569-1) ... Setting up golang-github-fatih-color-dev (1.7.0-1) ... Setting up golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ... Setting up po-debconf (1.0.21+nmu1) ... Setting up golang-protobuf-extensions-dev (1.0.1-1) ... Setting up golang-google-cloud-compute-metadata-dev (0.56.0-1) ... Setting up man-db (2.9.4-2) ... Not building database; man-db/auto-update is not 'true'. Setting up dh-autoreconf (20) ... Setting up golang-golang-x-oauth2-google-dev (0.0~git20190604.0f29369-2) ... Setting up golang-github-hashicorp-go-hclog-dev (0.11.0-1) ... Setting up debhelper (13.3.4) ... Setting up golang-github-hashicorp-go-retryablehttp-dev (0.6.4-2) ... Setting up dh-golang (1.51) ... Setting up golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-3) ... Setting up golang-google-genproto-dev (0.0~git20200413.b5235f6-1) ... Setting up golang-github-prometheus-client-golang-dev (1.9.0-2) ... Setting up golang-github-armon-go-metrics-dev (0.3.4-1) ... Setting up golang-google-grpc-dev (1.27.1-1) ... Setting up golang-github-go-kit-kit-dev (0.6.0-3) ... Setting up golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-2) ... Setting up golang-github-prometheus-common-dev (0.15.0-2) ... Processing triggers for libc-bin (2.31-13) ... Reading package lists... Building dependency tree... Reading state information... Reading extended state information... Initializing package states... Writing extended state information... Building tag database... -> Finished parsing the build-deps Reading package lists... Building dependency tree... Reading state information... fakeroot is already the newest version (1.25.3-1.1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. I: Building the package hostname: Name or service not known I: Running cd /build/golang-github-hashicorp-raft-1.1.2/ && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-buildpackage -us -uc -b && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-genchanges -S > ../golang-github-hashicorp-raft_1.1.2-1_source.changes dpkg-buildpackage: info: source package golang-github-hashicorp-raft dpkg-buildpackage: info: source version 1.1.2-1 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Dmitry Smirnov dpkg-source --before-build . dpkg-buildpackage: info: host architecture armhf fakeroot debian/rules clean dh clean --buildsystem=golang --with=golang dh_auto_clean -O--buildsystem=golang dh_autoreconf_clean -O--buildsystem=golang dh_clean -O--buildsystem=golang debian/rules build dh build --buildsystem=golang --with=golang dh_update_autotools_config -O--buildsystem=golang dh_autoreconf -O--buildsystem=golang dh_auto_configure -O--buildsystem=golang dh_auto_build -O--buildsystem=golang cd obj-arm-linux-gnueabihf && go install -trimpath -v -p 4 github.com/hashicorp/raft github.com/hashicorp/raft/bench internal/unsafeheader internal/cpu runtime/internal/sys internal/race sync/atomic unicode runtime/internal/math internal/bytealg runtime/internal/atomic unicode/utf8 container/list crypto/internal/subtle crypto/subtle math/bits runtime internal/testlog encoding unicode/utf16 math internal/nettrace vendor/golang.org/x/crypto/cryptobyte/asn1 runtime/cgo vendor/golang.org/x/crypto/internal/subtle golang.org/x/sys/internal/unsafeheader internal/reflectlite sync internal/singleflight math/rand sort errors io internal/oserror strconv syscall github.com/hashicorp/golang-lru/simplelru vendor/golang.org/x/net/dns/dnsmessage bytes strings reflect bufio github.com/hashicorp/go-immutable-radix time internal/syscall/unix internal/syscall/execenv hash crypto/internal/randutil crypto/rc4 hash/crc32 crypto crypto/hmac vendor/golang.org/x/text/transform vendor/golang.org/x/crypto/hkdf path regexp/syntax html context internal/poll hash/crc64 internal/fmtsort encoding/binary os regexp crypto/cipher encoding/base64 crypto/sha512 fmt crypto/aes net crypto/des crypto/ed25519/internal/edwards25519 crypto/md5 crypto/sha1 math/big encoding/json log compress/flate compress/gzip crypto/sha256 encoding/hex encoding/pem path/filepath net/url io/ioutil crypto/rand crypto/elliptic encoding/asn1 crypto/ed25519 crypto/rsa crypto/dsa vendor/golang.org/x/crypto/cryptobyte crypto/x509/pkix vendor/golang.org/x/crypto/chacha20 vendor/golang.org/x/crypto/poly1305 vendor/golang.org/x/crypto/curve25519 vendor/golang.org/x/crypto/chacha20poly1305 crypto/ecdsa vendor/golang.org/x/text/unicode/bidi vendor/golang.org/x/text/unicode/norm vendor/golang.org/x/net/http2/hpack mime vendor/golang.org/x/text/secure/bidirule mime/quotedprintable net/http/internal os/signal golang.org/x/sys/unix vendor/golang.org/x/net/idna encoding/gob go/token text/template/parse crypto/x509 net/textproto vendor/golang.org/x/net/http/httpguts vendor/golang.org/x/net/http/httpproxy mime/multipart github.com/mattn/go-isatty text/template github.com/mattn/go-colorable flag github.com/fatih/color github.com/hashicorp/go-hclog crypto/tls runtime/debug runtime/trace testing html/template net/http/httptrace net/http net/rpc github.com/armon/go-metrics github.com/hashicorp/go-msgpack/codec github.com/hashicorp/raft github.com/hashicorp/raft/bench dh_auto_test -O--buildsystem=golang cd obj-arm-linux-gnueabihf && go test -vet=off -v -p 4 github.com/hashicorp/raft github.com/hashicorp/raft/bench === RUN TestCommitment_setVoters --- PASS: TestCommitment_setVoters (0.00s) === RUN TestCommitment_match_max --- PASS: TestCommitment_match_max (0.00s) === RUN TestCommitment_match_nonVoting --- PASS: TestCommitment_match_nonVoting (0.00s) === RUN TestCommitment_recalculate --- PASS: TestCommitment_recalculate (0.00s) === RUN TestCommitment_recalculate_startIndex --- PASS: TestCommitment_recalculate_startIndex (0.00s) === RUN TestCommitment_noVoterSanity --- PASS: TestCommitment_noVoterSanity (0.00s) === RUN TestCommitment_singleVoter --- PASS: TestCommitment_singleVoter (0.00s) === RUN TestConfiguration_Configuration_Clone --- PASS: TestConfiguration_Configuration_Clone (0.00s) === RUN TestConfiguration_configurations_Clone --- PASS: TestConfiguration_configurations_Clone (0.00s) === RUN TestConfiguration_hasVote --- PASS: TestConfiguration_hasVote (0.00s) === RUN TestConfiguration_checkConfiguration --- PASS: TestConfiguration_checkConfiguration (0.00s) === RUN TestConfiguration_nextConfiguration_table --- PASS: TestConfiguration_nextConfiguration_table (0.00s) === RUN TestConfiguration_nextConfiguration_prevIndex --- PASS: TestConfiguration_nextConfiguration_prevIndex (0.00s) === RUN TestConfiguration_nextConfiguration_checkConfiguration --- PASS: TestConfiguration_nextConfiguration_checkConfiguration (0.00s) === RUN TestConfiguration_encodeDecodePeers --- PASS: TestConfiguration_encodeDecodePeers (0.00s) === RUN TestConfiguration_encodeDecodeConfiguration --- PASS: TestConfiguration_encodeDecodeConfiguration (0.00s) === RUN TestDiscardSnapshotStoreImpl --- PASS: TestDiscardSnapshotStoreImpl (0.00s) === RUN TestDiscardSnapshotSinkImpl --- PASS: TestDiscardSnapshotSinkImpl (0.00s) === RUN TestFileSnapshotStoreImpl --- PASS: TestFileSnapshotStoreImpl (0.00s) === RUN TestFileSnapshotSinkImpl --- PASS: TestFileSnapshotSinkImpl (0.00s) === RUN TestFileSS_CreateSnapshotMissingParentDir testing.go:153: 2021-08-25T09:30:55.743Z [INFO] creating new snapshot: path=/tmp/raft214392515/raft220312390/snapshots/3-10-1629883855743.tmp --- PASS: TestFileSS_CreateSnapshotMissingParentDir (0.01s) === RUN TestFileSS_CreateSnapshot testing.go:153: 2021-08-25T09:30:55.754Z [INFO] creating new snapshot: path=/tmp/raft136735213/snapshots/3-10-1629883855754.tmp testing.go:153: 2021-08-25T09:30:55.760Z [WARN] found temporary snapshot: name=3-10-1629883855754.tmp --- PASS: TestFileSS_CreateSnapshot (0.02s) === RUN TestFileSS_CancelSnapshot testing.go:153: 2021-08-25T09:30:55.776Z [INFO] creating new snapshot: path=/tmp/raft121874024/snapshots/3-10-1629883855776.tmp --- PASS: TestFileSS_CancelSnapshot (0.01s) === RUN TestFileSS_Retention testing.go:153: 2021-08-25T09:30:55.791Z [INFO] creating new snapshot: path=/tmp/raft391945383/snapshots/3-10-1629883855791.tmp testing.go:153: 2021-08-25T09:30:55.817Z [INFO] creating new snapshot: path=/tmp/raft391945383/snapshots/3-11-1629883855817.tmp testing.go:153: 2021-08-25T09:30:55.840Z [INFO] creating new snapshot: path=/tmp/raft391945383/snapshots/3-12-1629883855839.tmp testing.go:153: 2021-08-25T09:30:55.859Z [INFO] reaping snapshot: path=/tmp/raft391945383/snapshots/3-10-1629883855791 testing.go:153: 2021-08-25T09:30:55.860Z [INFO] creating new snapshot: path=/tmp/raft391945383/snapshots/3-13-1629883855860.tmp testing.go:153: 2021-08-25T09:30:55.883Z [INFO] reaping snapshot: path=/tmp/raft391945383/snapshots/3-11-1629883855817 testing.go:153: 2021-08-25T09:30:55.884Z [INFO] creating new snapshot: path=/tmp/raft391945383/snapshots/3-14-1629883855884.tmp testing.go:153: 2021-08-25T09:30:55.910Z [INFO] reaping snapshot: path=/tmp/raft391945383/snapshots/3-12-1629883855839 --- PASS: TestFileSS_Retention (0.12s) === RUN TestFileSS_BadPerm --- PASS: TestFileSS_BadPerm (0.00s) === RUN TestFileSS_MissingParentDir --- PASS: TestFileSS_MissingParentDir (0.00s) === RUN TestFileSS_Ordering testing.go:153: 2021-08-25T09:30:55.914Z [INFO] creating new snapshot: path=/tmp/raft177964462/snapshots/5-130350-1629883855914.tmp testing.go:153: 2021-08-25T09:30:55.936Z [INFO] creating new snapshot: path=/tmp/raft177964462/snapshots/36-204917-1629883855936.tmp --- PASS: TestFileSS_Ordering (0.04s) === RUN TestDeferFutureSuccess --- PASS: TestDeferFutureSuccess (0.00s) === RUN TestDeferFutureError --- PASS: TestDeferFutureError (0.00s) === RUN TestDeferFutureConcurrent --- PASS: TestDeferFutureConcurrent (0.00s) === RUN TestInmemSnapshotStoreImpl --- PASS: TestInmemSnapshotStoreImpl (0.00s) === RUN TestInmemSnapshotSinkImpl --- PASS: TestInmemSnapshotSinkImpl (0.00s) === RUN TestInmemSS_CreateSnapshot --- PASS: TestInmemSS_CreateSnapshot (0.00s) === RUN TestInmemSS_OpenSnapshotTwice --- PASS: TestInmemSS_OpenSnapshotTwice (0.00s) === RUN TestInmemTransportImpl --- PASS: TestInmemTransportImpl (0.00s) === RUN TestInmemTransportWriteTimeout --- PASS: TestInmemTransportWriteTimeout (0.01s) === RUN TestRaft_Integ --- SKIP: TestRaft_Integ (0.00s) === RUN TestLogCache --- PASS: TestLogCache (0.00s) === RUN TestNetworkTransport_CloseStreams --- PASS: TestNetworkTransport_CloseStreams (0.00s) === RUN TestNetworkTransport_StartStop --- PASS: TestNetworkTransport_StartStop (0.00s) === RUN TestNetworkTransport_Heartbeat_FastPath --- PASS: TestNetworkTransport_Heartbeat_FastPath (0.00s) === RUN TestNetworkTransport_AppendEntries --- PASS: TestNetworkTransport_AppendEntries (0.00s) === RUN TestNetworkTransport_AppendEntriesPipeline --- PASS: TestNetworkTransport_AppendEntriesPipeline (0.00s) === RUN TestNetworkTransport_AppendEntriesPipeline_CloseStreams net_transport_test.go:339: Disabled unreliable test --- SKIP: TestNetworkTransport_AppendEntriesPipeline_CloseStreams (0.00s) === RUN TestNetworkTransport_RequestVote --- PASS: TestNetworkTransport_RequestVote (0.00s) === RUN TestNetworkTransport_InstallSnapshot --- PASS: TestNetworkTransport_InstallSnapshot (0.00s) === RUN TestNetworkTransport_EncodeDecode --- PASS: TestNetworkTransport_EncodeDecode (0.00s) === RUN TestNetworkTransport_EncodeDecode_AddressProvider --- PASS: TestNetworkTransport_EncodeDecode_AddressProvider (0.00s) === RUN TestNetworkTransport_PooledConn --- PASS: TestNetworkTransport_PooledConn (0.02s) === RUN TestNetworkTransport_ListenBackoff net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.002Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.008Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.019Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.039Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.079Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.159Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.320Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:56.640Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:57.281Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:58.281Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2021-08-25T09:30:59.282Z [ERROR] test: failed to accept connection: error="intentional error in test" --- PASS: TestNetworkTransport_ListenBackoff (4.00s) === RUN TestPeersJSON_BadConfiguration --- PASS: TestPeersJSON_BadConfiguration (0.00s) === RUN TestPeersJSON_ReadPeersJSON --- PASS: TestPeersJSON_ReadPeersJSON (0.00s) === RUN TestPeersJSON_ReadConfigJSON --- PASS: TestPeersJSON_ReadConfigJSON (0.00s) === RUN TestRaft_StartStop testLoggerAdapter verbose: server-5a945f62-a102-7777-ba33-5203822559bd: 2021-08-25T09:31:00.007Z [INFO] server-5a945f62-a102-7777-ba33-5203822559bd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5a945f62-a102-7777-ba33-5203822559bd Address:5a945f62-a102-7777-ba33-5203822559bd}]" testing.go:149: server-5a945f62-a102-7777-ba33-5203822559bd: 2021-08-25T09:31:00.007Z [INFO] server-5a945f62-a102-7777-ba33-5203822559bd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5a945f62-a102-7777-ba33-5203822559bd Address:5a945f62-a102-7777-ba33-5203822559bd}]" --- PASS: TestRaft_StartStop (0.00s) === RUN TestRaft_AfterShutdown testLoggerAdapter verbose: server-93db230f-e40e-c8f2-bce1-49bce958dad3: 2021-08-25T09:31:00.009Z [INFO] server-93db230f-e40e-c8f2-bce1-49bce958dad3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-93db230f-e40e-c8f2-bce1-49bce958dad3 Address:93db230f-e40e-c8f2-bce1-49bce958dad3}]" testing.go:149: server-93db230f-e40e-c8f2-bce1-49bce958dad3: 2021-08-25T09:31:00.009Z [INFO] server-93db230f-e40e-c8f2-bce1-49bce958dad3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-93db230f-e40e-c8f2-bce1-49bce958dad3 Address:93db230f-e40e-c8f2-bce1-49bce958dad3}]" --- PASS: TestRaft_AfterShutdown (0.00s) === RUN TestRaft_LiveBootstrap testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.012Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: initial configuration: index=0 servers=[] testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.012Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.012Z [INFO] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: initial configuration: index=0 servers=[] testing.go:149: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.012Z [INFO] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-801e9bb3-0387-ea88-77d9-fb7631183d54: 2021-08-25T09:31:00.012Z [INFO] server-801e9bb3-0387-ea88-77d9-fb7631183d54: initial configuration: index=0 servers=[] testing.go:149: server-801e9bb3-0387-ea88-77d9-fb7631183d54: 2021-08-25T09:31:00.012Z [INFO] server-801e9bb3-0387-ea88-77d9-fb7631183d54: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.012Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: entering follower state: follower="Node at e6dad585-8c02-5d30-9b8b-22e27aec3544 [Follower]" leader= testLoggerAdapter verbose: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.012Z [INFO] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: entering follower state: follower="Node at e6a50ae5-5126-81d4-adb0-402613fc9ca2 [Follower]" leader= testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.012Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: entering follower state: follower="Node at e6dad585-8c02-5d30-9b8b-22e27aec3544 [Follower]" leader= testLoggerAdapter verbose: server-801e9bb3-0387-ea88-77d9-fb7631183d54: 2021-08-25T09:31:00.013Z [INFO] server-801e9bb3-0387-ea88-77d9-fb7631183d54: entering follower state: follower="Node at 801e9bb3-0387-ea88-77d9-fb7631183d54 [Follower]" leader= testing.go:149: server-801e9bb3-0387-ea88-77d9-fb7631183d54: 2021-08-25T09:31:00.013Z [INFO] server-801e9bb3-0387-ea88-77d9-fb7631183d54: entering follower state: follower="Node at 801e9bb3-0387-ea88-77d9-fb7631183d54 [Follower]" leader= testing.go:149: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.012Z [INFO] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: entering follower state: follower="Node at e6a50ae5-5126-81d4-adb0-402613fc9ca2 [Follower]" leader= testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.013Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:00.013Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.084Z [WARN] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: no known peers, aborting election testing.go:149: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.084Z [WARN] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: no known peers, aborting election testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [WARN] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [WARN] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: entering candidate state: node="Node at e6dad585-8c02-5d30-9b8b-22e27aec3544 [Candidate]" term=2 testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: entering candidate state: node="Node at e6dad585-8c02-5d30-9b8b-22e27aec3544 [Candidate]" term=2 testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: election won: tally=2 testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: election won: tally=2 testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: entering leader state: leader="Node at e6dad585-8c02-5d30-9b8b-22e27aec3544 [Leader]" testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.103Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: entering leader state: leader="Node at e6dad585-8c02-5d30-9b8b-22e27aec3544 [Leader]" testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: added peer, starting replication: peer=server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: added peer, starting replication: peer=server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: added peer, starting replication: peer=server-801e9bb3-0387-ea88-77d9-fb7631183d54 testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: added peer, starting replication: peer=server-801e9bb3-0387-ea88-77d9-fb7631183d54 testLoggerAdapter verbose: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.104Z [WARN] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: failed to get previous log: previous-index=1 last-index=0 error="log not found" testing.go:149: server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: 2021-08-25T09:31:00.104Z [WARN] server-e6a50ae5-5126-81d4-adb0-402613fc9ca2: failed to get previous log: previous-index=1 last-index=0 error="log not found" testLoggerAdapter verbose: server-801e9bb3-0387-ea88-77d9-fb7631183d54: 2021-08-25T09:31:00.104Z [WARN] server-801e9bb3-0387-ea88-77d9-fb7631183d54: failed to get previous log: previous-index=1 last-index=0 error="log not found" testing.go:149: server-801e9bb3-0387-ea88-77d9-fb7631183d54: 2021-08-25T09:31:00.104Z [WARN] server-801e9bb3-0387-ea88-77d9-fb7631183d54: failed to get previous log: previous-index=1 last-index=0 error="log not found" testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [WARN] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: appendEntries rejected, sending older logs: peer="{Voter server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 e6a50ae5-5126-81d4-adb0-402613fc9ca2}" next=1 testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [WARN] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: appendEntries rejected, sending older logs: peer="{Voter server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 e6a50ae5-5126-81d4-adb0-402613fc9ca2}" next=1 testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [WARN] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: appendEntries rejected, sending older logs: peer="{Voter server-801e9bb3-0387-ea88-77d9-fb7631183d54 801e9bb3-0387-ea88-77d9-fb7631183d54}" next=1 testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [WARN] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: appendEntries rejected, sending older logs: peer="{Voter server-801e9bb3-0387-ea88-77d9-fb7631183d54 801e9bb3-0387-ea88-77d9-fb7631183d54}" next=1 testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: pipelining replication: peer="{Voter server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 e6a50ae5-5126-81d4-adb0-402613fc9ca2}" testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: pipelining replication: peer="{Voter server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 e6a50ae5-5126-81d4-adb0-402613fc9ca2}" testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: pipelining replication: peer="{Voter server-801e9bb3-0387-ea88-77d9-fb7631183d54 801e9bb3-0387-ea88-77d9-fb7631183d54}" testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.104Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: pipelining replication: peer="{Voter server-801e9bb3-0387-ea88-77d9-fb7631183d54 801e9bb3-0387-ea88-77d9-fb7631183d54}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.209Z [INFO] stable state for Follower reached at 2021-08-25 09:31:00.104072742 +0000 UTC m=+4.370289186 (2 nodes), 90.486564ms from start of poll, 91.673532ms from cluster start. Timeout at 2021-08-25 09:31:00.209229904 +0000 UTC m=+4.475446408, 105.157222ms after stability testing.go:149: cluster: 2021-08-25T09:31:00.209Z [INFO] stable state for Follower reached at 2021-08-25 09:31:00.104072742 +0000 UTC m=+4.370289186 (2 nodes), 90.486564ms from start of poll, 91.673532ms from cluster start. Timeout at 2021-08-25 09:31:00.209229904 +0000 UTC m=+4.475446408, 105.157222ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.209Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:00.209Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.314Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.209624627 +0000 UTC m=+4.475841031 (1 nodes), 2.92µs from start of poll, 197.225377ms from cluster start. Timeout at 2021-08-25 09:31:00.314728175 +0000 UTC m=+4.580944599, 105.103568ms after stability testing.go:149: cluster: 2021-08-25T09:31:00.314Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.209624627 +0000 UTC m=+4.475841031 (1 nodes), 2.92µs from start of poll, 197.225377ms from cluster start. Timeout at 2021-08-25 09:31:00.314728175 +0000 UTC m=+4.580944599, 105.103568ms after stability testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.325Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: aborting pipeline replication: peer="{Voter server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 e6a50ae5-5126-81d4-adb0-402613fc9ca2}" testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.325Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: aborting pipeline replication: peer="{Voter server-e6a50ae5-5126-81d4-adb0-402613fc9ca2 e6a50ae5-5126-81d4-adb0-402613fc9ca2}" testLoggerAdapter verbose: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.325Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: aborting pipeline replication: peer="{Voter server-801e9bb3-0387-ea88-77d9-fb7631183d54 801e9bb3-0387-ea88-77d9-fb7631183d54}" testing.go:149: server-e6dad585-8c02-5d30-9b8b-22e27aec3544: 2021-08-25T09:31:00.325Z [INFO] server-e6dad585-8c02-5d30-9b8b-22e27aec3544: aborting pipeline replication: peer="{Voter server-801e9bb3-0387-ea88-77d9-fb7631183d54 801e9bb3-0387-ea88-77d9-fb7631183d54}" --- PASS: TestRaft_LiveBootstrap (0.32s) === RUN TestRaft_RecoverCluster_NoState testLoggerAdapter verbose: server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: 2021-08-25T09:31:00.328Z [INFO] server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: initial configuration: index=0 servers=[] testing.go:149: server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: 2021-08-25T09:31:00.328Z [INFO] server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: 2021-08-25T09:31:00.328Z [INFO] server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: entering follower state: follower="Node at 8606353d-9f2c-7f8c-8e48-47fb87b81ca9 [Shutdown]" leader= testing.go:149: server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: 2021-08-25T09:31:00.328Z [INFO] server-8606353d-9f2c-7f8c-8e48-47fb87b81ca9: entering follower state: follower="Node at 8606353d-9f2c-7f8c-8e48-47fb87b81ca9 [Shutdown]" leader= --- PASS: TestRaft_RecoverCluster_NoState (0.00s) === RUN TestRaft_RecoverCluster raft_test.go:119: Disabled unreliable test --- SKIP: TestRaft_RecoverCluster (0.00s) === RUN TestRaft_HasExistingState testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.330Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f Address:a0ccecb6-c39b-654b-f606-75bb4aec1e9f} {Suffrage:Voter ID:server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 Address:f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}]" testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.330Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f Address:a0ccecb6-c39b-654b-f606-75bb4aec1e9f} {Suffrage:Voter ID:server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 Address:f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}]" testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.330Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: entering follower state: follower="Node at a0ccecb6-c39b-654b-f606-75bb4aec1e9f [Follower]" leader= testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.330Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: entering follower state: follower="Node at a0ccecb6-c39b-654b-f606-75bb4aec1e9f [Follower]" leader= testLoggerAdapter verbose: server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: 2021-08-25T09:31:00.331Z [INFO] server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f Address:a0ccecb6-c39b-654b-f606-75bb4aec1e9f} {Suffrage:Voter ID:server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 Address:f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}]" testing.go:149: server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: 2021-08-25T09:31:00.331Z [INFO] server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f Address:a0ccecb6-c39b-654b-f606-75bb4aec1e9f} {Suffrage:Voter ID:server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 Address:f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}]" testLoggerAdapter verbose: server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: 2021-08-25T09:31:00.331Z [INFO] server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: entering follower state: follower="Node at f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 [Follower]" leader= testing.go:149: server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: 2021-08-25T09:31:00.331Z [INFO] server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9: entering follower state: follower="Node at f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 [Follower]" leader= testLoggerAdapter verbose: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.331Z [INFO] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: initial configuration: index=0 servers=[] testing.go:149: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.331Z [INFO] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.332Z [INFO] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: entering follower state: follower="Node at 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d [Follower]" leader= testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.332Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.332Z [INFO] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: entering follower state: follower="Node at 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d [Follower]" leader= testing.go:149: cluster: 2021-08-25T09:31:00.332Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.403Z [WARN] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.403Z [WARN] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.403Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: entering candidate state: node="Node at a0ccecb6-c39b-654b-f606-75bb4aec1e9f [Candidate]" term=2 testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.403Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: entering candidate state: node="Node at a0ccecb6-c39b-654b-f606-75bb4aec1e9f [Candidate]" term=2 testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: election won: tally=2 testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: election won: tally=2 testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: entering leader state: leader="Node at a0ccecb6-c39b-654b-f606-75bb4aec1e9f [Leader]" testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: entering leader state: leader="Node at a0ccecb6-c39b-654b-f606-75bb4aec1e9f [Leader]" testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: added peer, starting replication: peer=server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: added peer, starting replication: peer=server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: pipelining replication: peer="{Voter server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}" testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.404Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: pipelining replication: peer="{Voter server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}" testLoggerAdapter verbose: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.414Z [WARN] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: no known peers, aborting election testing.go:149: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.414Z [WARN] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: no known peers, aborting election testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.509Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.404300004 +0000 UTC m=+4.670516448 (1 nodes), 72.001883ms from start of poll, 73.777856ms from cluster start. Timeout at 2021-08-25 09:31:00.509403792 +0000 UTC m=+4.775620256, 105.103808ms after stability testing.go:149: cluster: 2021-08-25T09:31:00.509Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.404300004 +0000 UTC m=+4.670516448 (1 nodes), 72.001883ms from start of poll, 73.777856ms from cluster start. Timeout at 2021-08-25 09:31:00.509403792 +0000 UTC m=+4.775620256, 105.103808ms after stability testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.509Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: updating configuration: command=AddStaging server-id=server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d server-addr=8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d servers="[{Suffrage:Voter ID:server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f Address:a0ccecb6-c39b-654b-f606-75bb4aec1e9f} {Suffrage:Voter ID:server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 Address:f988c234-2c4e-5b85-9d8f-ba864c3dc0d9} {Suffrage:Voter ID:server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d Address:8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}]" testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.509Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: updating configuration: command=AddStaging server-id=server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d server-addr=8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d servers="[{Suffrage:Voter ID:server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f Address:a0ccecb6-c39b-654b-f606-75bb4aec1e9f} {Suffrage:Voter ID:server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 Address:f988c234-2c4e-5b85-9d8f-ba864c3dc0d9} {Suffrage:Voter ID:server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d Address:8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}]" testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.510Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: added peer, starting replication: peer=server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.510Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: added peer, starting replication: peer=server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d testLoggerAdapter verbose: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.510Z [WARN] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: 2021-08-25T09:31:00.510Z [WARN] server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.510Z [WARN] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: appendEntries rejected, sending older logs: peer="{Voter server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}" next=1 testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.510Z [WARN] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: appendEntries rejected, sending older logs: peer="{Voter server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}" next=1 testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.510Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: pipelining replication: peer="{Voter server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}" testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.510Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: pipelining replication: peer="{Voter server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.515Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:00.515Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.620Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.515731096 +0000 UTC m=+4.781947480 (1 nodes), 1.919µs from start of poll, 185.208888ms from cluster start. Timeout at 2021-08-25 09:31:00.620829965 +0000 UTC m=+4.887046389, 105.098909ms after stability testing.go:149: cluster: 2021-08-25T09:31:00.620Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.515731096 +0000 UTC m=+4.781947480 (1 nodes), 1.919µs from start of poll, 185.208888ms from cluster start. Timeout at 2021-08-25 09:31:00.620829965 +0000 UTC m=+4.887046389, 105.098909ms after stability testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.621Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: aborting pipeline replication: peer="{Voter server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}" testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.621Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: aborting pipeline replication: peer="{Voter server-8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d 8f5a7bb6-e396-75c8-c28b-5487f1fc8f0d}" testLoggerAdapter verbose: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.621Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: aborting pipeline replication: peer="{Voter server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}" testing.go:149: server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: 2021-08-25T09:31:00.621Z [INFO] server-a0ccecb6-c39b-654b-f606-75bb4aec1e9f: aborting pipeline replication: peer="{Voter server-f988c234-2c4e-5b85-9d8f-ba864c3dc0d9 f988c234-2c4e-5b85-9d8f-ba864c3dc0d9}" --- PASS: TestRaft_HasExistingState (0.29s) === RUN TestRaft_SingleNode testLoggerAdapter verbose: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.623Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff2f47fd-a459-0a31-905d-6a101fe701b6 Address:ff2f47fd-a459-0a31-905d-6a101fe701b6}]" testing.go:149: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.623Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ff2f47fd-a459-0a31-905d-6a101fe701b6 Address:ff2f47fd-a459-0a31-905d-6a101fe701b6}]" testLoggerAdapter verbose: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.624Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: entering follower state: follower="Node at ff2f47fd-a459-0a31-905d-6a101fe701b6 [Follower]" leader= testing.go:149: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.624Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: entering follower state: follower="Node at ff2f47fd-a459-0a31-905d-6a101fe701b6 [Follower]" leader= testLoggerAdapter verbose: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.698Z [WARN] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.698Z [WARN] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.699Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: entering candidate state: node="Node at ff2f47fd-a459-0a31-905d-6a101fe701b6 [Candidate]" term=2 testing.go:149: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.699Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: entering candidate state: node="Node at ff2f47fd-a459-0a31-905d-6a101fe701b6 [Candidate]" term=2 testLoggerAdapter verbose: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.699Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: election won: tally=1 testing.go:149: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.699Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: election won: tally=1 testLoggerAdapter verbose: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.699Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: entering leader state: leader="Node at ff2f47fd-a459-0a31-905d-6a101fe701b6 [Leader]" testing.go:149: server-ff2f47fd-a459-0a31-905d-6a101fe701b6: 2021-08-25T09:31:00.699Z [INFO] server-ff2f47fd-a459-0a31-905d-6a101fe701b6: entering leader state: leader="Node at ff2f47fd-a459-0a31-905d-6a101fe701b6 [Leader]" --- PASS: TestRaft_SingleNode (0.08s) === RUN TestRaft_TripleNode testLoggerAdapter verbose: server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: 2021-08-25T09:31:00.702Z [INFO] server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a Address:8e4053e4-853c-c4d2-c2e9-5e2430ed666a} {Suffrage:Voter ID:server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 Address:0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9} {Suffrage:Voter ID:server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 Address:6bf51d3a-dee5-70cc-239c-82e0fe292dd9}]" testing.go:149: server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: 2021-08-25T09:31:00.702Z [INFO] server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a Address:8e4053e4-853c-c4d2-c2e9-5e2430ed666a} {Suffrage:Voter ID:server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 Address:0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9} {Suffrage:Voter ID:server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 Address:6bf51d3a-dee5-70cc-239c-82e0fe292dd9}]" testLoggerAdapter verbose: server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: 2021-08-25T09:31:00.702Z [INFO] server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: entering follower state: follower="Node at 8e4053e4-853c-c4d2-c2e9-5e2430ed666a [Follower]" leader= testing.go:149: server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: 2021-08-25T09:31:00.702Z [INFO] server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a: entering follower state: follower="Node at 8e4053e4-853c-c4d2-c2e9-5e2430ed666a [Follower]" leader= testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.705Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a Address:8e4053e4-853c-c4d2-c2e9-5e2430ed666a} {Suffrage:Voter ID:server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 Address:0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9} {Suffrage:Voter ID:server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 Address:6bf51d3a-dee5-70cc-239c-82e0fe292dd9}]" testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.705Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a Address:8e4053e4-853c-c4d2-c2e9-5e2430ed666a} {Suffrage:Voter ID:server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 Address:0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9} {Suffrage:Voter ID:server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 Address:6bf51d3a-dee5-70cc-239c-82e0fe292dd9}]" testLoggerAdapter verbose: server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: 2021-08-25T09:31:00.705Z [INFO] server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a Address:8e4053e4-853c-c4d2-c2e9-5e2430ed666a} {Suffrage:Voter ID:server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 Address:0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9} {Suffrage:Voter ID:server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 Address:6bf51d3a-dee5-70cc-239c-82e0fe292dd9}]" testing.go:149: server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: 2021-08-25T09:31:00.705Z [INFO] server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a Address:8e4053e4-853c-c4d2-c2e9-5e2430ed666a} {Suffrage:Voter ID:server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 Address:0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9} {Suffrage:Voter ID:server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 Address:6bf51d3a-dee5-70cc-239c-82e0fe292dd9}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.705Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:00.705Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.705Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: entering follower state: follower="Node at 0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 [Follower]" leader= testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.705Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: entering follower state: follower="Node at 0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 [Follower]" leader= testLoggerAdapter verbose: server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: 2021-08-25T09:31:00.705Z [INFO] server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: entering follower state: follower="Node at 6bf51d3a-dee5-70cc-239c-82e0fe292dd9 [Follower]" leader= testing.go:149: server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: 2021-08-25T09:31:00.705Z [INFO] server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9: entering follower state: follower="Node at 6bf51d3a-dee5-70cc-239c-82e0fe292dd9 [Follower]" leader= testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.765Z [WARN] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.765Z [WARN] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.765Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: entering candidate state: node="Node at 0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 [Candidate]" term=2 testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.765Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: entering candidate state: node="Node at 0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 [Candidate]" term=2 testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: election won: tally=2 testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: election won: tally=2 testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: entering leader state: leader="Node at 0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 [Leader]" testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: entering leader state: leader="Node at 0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9 [Leader]" testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: added peer, starting replication: peer=server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: added peer, starting replication: peer=server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: added peer, starting replication: peer=server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: added peer, starting replication: peer=server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: pipelining replication: peer="{Voter server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a 8e4053e4-853c-c4d2-c2e9-5e2430ed666a}" testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: pipelining replication: peer="{Voter server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a 8e4053e4-853c-c4d2-c2e9-5e2430ed666a}" testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: pipelining replication: peer="{Voter server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 6bf51d3a-dee5-70cc-239c-82e0fe292dd9}" testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.766Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: pipelining replication: peer="{Voter server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 6bf51d3a-dee5-70cc-239c-82e0fe292dd9}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.871Z [INFO] stable state for Follower reached at 2021-08-25 09:31:00.766412802 +0000 UTC m=+5.032629226 (2 nodes), 60.900668ms from start of poll, 64.434355ms from cluster start. Timeout at 2021-08-25 09:31:00.871475454 +0000 UTC m=+5.137691898, 105.062672ms after stability testing.go:149: cluster: 2021-08-25T09:31:00.871Z [INFO] stable state for Follower reached at 2021-08-25 09:31:00.766412802 +0000 UTC m=+5.032629226 (2 nodes), 60.900668ms from start of poll, 64.434355ms from cluster start. Timeout at 2021-08-25 09:31:00.871475454 +0000 UTC m=+5.137691898, 105.062672ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.871Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:00.871Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:00.977Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.87183084 +0000 UTC m=+5.138047224 (1 nodes), 2.559µs from start of poll, 169.852353ms from cluster start. Timeout at 2021-08-25 09:31:00.976961665 +0000 UTC m=+5.243178129, 105.130905ms after stability testing.go:149: cluster: 2021-08-25T09:31:00.977Z [INFO] stable state for Leader reached at 2021-08-25 09:31:00.87183084 +0000 UTC m=+5.138047224 (1 nodes), 2.559µs from start of poll, 169.852353ms from cluster start. Timeout at 2021-08-25 09:31:00.976961665 +0000 UTC m=+5.243178129, 105.130905ms after stability testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.987Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: aborting pipeline replication: peer="{Voter server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 6bf51d3a-dee5-70cc-239c-82e0fe292dd9}" testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.987Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: aborting pipeline replication: peer="{Voter server-6bf51d3a-dee5-70cc-239c-82e0fe292dd9 6bf51d3a-dee5-70cc-239c-82e0fe292dd9}" testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.987Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: aborting pipeline replication: peer="{Voter server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a 8e4053e4-853c-c4d2-c2e9-5e2430ed666a}" testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:00.987Z [INFO] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: aborting pipeline replication: peer="{Voter server-8e4053e4-853c-c4d2-c2e9-5e2430ed666a 8e4053e4-853c-c4d2-c2e9-5e2430ed666a}" testLoggerAdapter verbose: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:01.038Z [ERROR] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: failed to heartbeat to: peer=8e4053e4-853c-c4d2-c2e9-5e2430ed666a error="command timed out" testing.go:149: server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: 2021-08-25T09:31:01.038Z [ERROR] server-0e93c0c4-0766-b6b7-26e7-bdc19a4a07f9: failed to heartbeat to: peer=8e4053e4-853c-c4d2-c2e9-5e2430ed666a error="command timed out" --- PASS: TestRaft_TripleNode (0.34s) === RUN TestRaft_LeaderFail testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.043Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c880f81a-325f-f987-5534-6efe79a4aa0d Address:c880f81a-325f-f987-5534-6efe79a4aa0d} {Suffrage:Voter ID:server-58633671-e0f2-ffc4-96f8-80811db6bec1 Address:58633671-e0f2-ffc4-96f8-80811db6bec1} {Suffrage:Voter ID:server-c728053a-382d-a4a2-d470-cf8beb8767be Address:c728053a-382d-a4a2-d470-cf8beb8767be}]" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.043Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c880f81a-325f-f987-5534-6efe79a4aa0d Address:c880f81a-325f-f987-5534-6efe79a4aa0d} {Suffrage:Voter ID:server-58633671-e0f2-ffc4-96f8-80811db6bec1 Address:58633671-e0f2-ffc4-96f8-80811db6bec1} {Suffrage:Voter ID:server-c728053a-382d-a4a2-d470-cf8beb8767be Address:c728053a-382d-a4a2-d470-cf8beb8767be}]" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.043Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering follower state: follower="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Follower]" leader= testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.043Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering follower state: follower="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Follower]" leader= testLoggerAdapter verbose: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.044Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c880f81a-325f-f987-5534-6efe79a4aa0d Address:c880f81a-325f-f987-5534-6efe79a4aa0d} {Suffrage:Voter ID:server-58633671-e0f2-ffc4-96f8-80811db6bec1 Address:58633671-e0f2-ffc4-96f8-80811db6bec1} {Suffrage:Voter ID:server-c728053a-382d-a4a2-d470-cf8beb8767be Address:c728053a-382d-a4a2-d470-cf8beb8767be}]" testing.go:149: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.044Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c880f81a-325f-f987-5534-6efe79a4aa0d Address:c880f81a-325f-f987-5534-6efe79a4aa0d} {Suffrage:Voter ID:server-58633671-e0f2-ffc4-96f8-80811db6bec1 Address:58633671-e0f2-ffc4-96f8-80811db6bec1} {Suffrage:Voter ID:server-c728053a-382d-a4a2-d470-cf8beb8767be Address:c728053a-382d-a4a2-d470-cf8beb8767be}]" testLoggerAdapter verbose: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.044Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: entering follower state: follower="Node at 58633671-e0f2-ffc4-96f8-80811db6bec1 [Follower]" leader= testing.go:149: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.044Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: entering follower state: follower="Node at 58633671-e0f2-ffc4-96f8-80811db6bec1 [Follower]" leader= testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.044Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c880f81a-325f-f987-5534-6efe79a4aa0d Address:c880f81a-325f-f987-5534-6efe79a4aa0d} {Suffrage:Voter ID:server-58633671-e0f2-ffc4-96f8-80811db6bec1 Address:58633671-e0f2-ffc4-96f8-80811db6bec1} {Suffrage:Voter ID:server-c728053a-382d-a4a2-d470-cf8beb8767be Address:c728053a-382d-a4a2-d470-cf8beb8767be}]" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.044Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c880f81a-325f-f987-5534-6efe79a4aa0d Address:c880f81a-325f-f987-5534-6efe79a4aa0d} {Suffrage:Voter ID:server-58633671-e0f2-ffc4-96f8-80811db6bec1 Address:58633671-e0f2-ffc4-96f8-80811db6bec1} {Suffrage:Voter ID:server-c728053a-382d-a4a2-d470-cf8beb8767be Address:c728053a-382d-a4a2-d470-cf8beb8767be}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.044Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:01.044Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.044Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering follower state: follower="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Follower]" leader= testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.044Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering follower state: follower="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Follower]" leader= testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.107Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.107Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.107Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering candidate state: node="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Candidate]" term=2 testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.107Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering candidate state: node="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Candidate]" term=2 testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.107Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: election won: tally=2 testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.107Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: election won: tally=2 testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering leader state: leader="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Leader]" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering leader state: leader="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Leader]" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: added peer, starting replication: peer=server-58633671-e0f2-ffc4-96f8-80811db6bec1 testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: added peer, starting replication: peer=server-58633671-e0f2-ffc4-96f8-80811db6bec1 testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: added peer, starting replication: peer=server-c728053a-382d-a4a2-d470-cf8beb8767be testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: added peer, starting replication: peer=server-c728053a-382d-a4a2-d470-cf8beb8767be testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: pipelining replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: pipelining replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: pipelining replication: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.108Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: pipelining replication: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.213Z [INFO] stable state for Follower reached at 2021-08-25 09:31:01.10823033 +0000 UTC m=+5.374446774 (2 nodes), 63.548538ms from start of poll, 64.844496ms from cluster start. Timeout at 2021-08-25 09:31:01.213382354 +0000 UTC m=+5.479598818, 105.152044ms after stability testing.go:149: cluster: 2021-08-25T09:31:01.213Z [INFO] stable state for Follower reached at 2021-08-25 09:31:01.10823033 +0000 UTC m=+5.374446774 (2 nodes), 63.548538ms from start of poll, 64.844496ms from cluster start. Timeout at 2021-08-25 09:31:01.213382354 +0000 UTC m=+5.479598818, 105.152044ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.213Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:01.213Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.319Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.213788895 +0000 UTC m=+5.480005279 (1 nodes), 2.499µs from start of poll, 170.403001ms from cluster start. Timeout at 2021-08-25 09:31:01.318948198 +0000 UTC m=+5.585164662, 105.159383ms after stability testing.go:149: cluster: 2021-08-25T09:31:01.319Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.213788895 +0000 UTC m=+5.480005279 (1 nodes), 2.499µs from start of poll, 170.403001ms from cluster start. Timeout at 2021-08-25 09:31:01.318948198 +0000 UTC m=+5.585164662, 105.159383ms after stability raft_test.go:336: [INFO] Disconnecting Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Leader] testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.334Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.334Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.334Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.334Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.335Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:01.335Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.335Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to pipeline appendEntries: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="append pipeline closed" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.335Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to pipeline appendEntries: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="append pipeline closed" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.335Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: aborting pipeline replication: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.335Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: aborting pipeline replication: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.336Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to pipeline appendEntries: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="append pipeline closed" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.336Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to pipeline appendEntries: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="append pipeline closed" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.336Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: aborting pipeline replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.336Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: aborting pipeline replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.343Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.343Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.343Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.343Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.350Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.350Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.352Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.352Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.363Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.363Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.363Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.363Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.365Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.365Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.372Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.372Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.378Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact: server-id=server-c728053a-382d-a4a2-d470-cf8beb8767be time=50.128122ms testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.378Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact: server-id=server-c728053a-382d-a4a2-d470-cf8beb8767be time=50.128122ms testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.380Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.380Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.381Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.381Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.382Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: heartbeat timeout reached, starting election: last-leader=c880f81a-325f-f987-5534-6efe79a4aa0d testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.382Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: heartbeat timeout reached, starting election: last-leader=c880f81a-325f-f987-5534-6efe79a4aa0d testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.383Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering candidate state: node="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Candidate]" term=3 testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.383Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering candidate state: node="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Candidate]" term=3 testLoggerAdapter verbose: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.383Z [WARN] server-58633671-e0f2-ffc4-96f8-80811db6bec1: rejecting vote request since we have a leader: from=c728053a-382d-a4a2-d470-cf8beb8767be leader=c880f81a-325f-f987-5534-6efe79a4aa0d testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.383Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to make requestVote RPC: target="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.383Z [WARN] server-58633671-e0f2-ffc4-96f8-80811db6bec1: rejecting vote request since we have a leader: from=c728053a-382d-a4a2-d470-cf8beb8767be leader=c880f81a-325f-f987-5534-6efe79a4aa0d testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.383Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to make requestVote RPC: target="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact: server-id=server-58633671-e0f2-ffc4-96f8-80811db6bec1 time=59.911821ms testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact: server-id=server-58633671-e0f2-ffc4-96f8-80811db6bec1 time=59.911821ms testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact: server-id=server-c728053a-382d-a4a2-d470-cf8beb8767be time=60.420553ms testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact: server-id=server-c728053a-382d-a4a2-d470-cf8beb8767be time=60.420553ms testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact quorum of nodes, stepping down testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering follower state: follower="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Follower]" leader= testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.389Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering follower state: follower="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Follower]" leader= testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.391Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.391Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=c728053a-382d-a4a2-d470-cf8beb8767be error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.401Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.401Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to heartbeat to: peer=58633671-e0f2-ffc4-96f8-80811db6bec1 error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.406Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.406Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.408Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.408Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to appendEntries to: peer="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.433Z [WARN] server-58633671-e0f2-ffc4-96f8-80811db6bec1: heartbeat timeout reached, starting election: last-leader=c880f81a-325f-f987-5534-6efe79a4aa0d testing.go:149: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.433Z [WARN] server-58633671-e0f2-ffc4-96f8-80811db6bec1: heartbeat timeout reached, starting election: last-leader=c880f81a-325f-f987-5534-6efe79a4aa0d testLoggerAdapter verbose: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.433Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: entering candidate state: node="Node at 58633671-e0f2-ffc4-96f8-80811db6bec1 [Candidate]" term=3 testing.go:149: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.433Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: entering candidate state: node="Node at 58633671-e0f2-ffc4-96f8-80811db6bec1 [Candidate]" term=3 testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.433Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.433Z [ERROR] server-58633671-e0f2-ffc4-96f8-80811db6bec1: failed to make requestVote RPC: target="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.433Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: duplicate requestVote for same term: term=3 testing.go:149: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.433Z [ERROR] server-58633671-e0f2-ffc4-96f8-80811db6bec1: failed to make requestVote RPC: target="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.476Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: Election timeout reached, restarting election testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.476Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: Election timeout reached, restarting election testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering candidate state: node="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Candidate]" term=4 testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering candidate state: node="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Candidate]" term=4 testLoggerAdapter verbose: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.477Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: entering follower state: follower="Node at 58633671-e0f2-ffc4-96f8-80811db6bec1 [Follower]" leader= testing.go:149: server-58633671-e0f2-ffc4-96f8-80811db6bec1: 2021-08-25T09:31:01.477Z [INFO] server-58633671-e0f2-ffc4-96f8-80811db6bec1: entering follower state: follower="Node at 58633671-e0f2-ffc4-96f8-80811db6bec1 [Follower]" leader= testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to make requestVote RPC: target="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to make requestVote RPC: target="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: election won: tally=2 testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: election won: tally=2 testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering leader state: leader="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Leader]" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: entering leader state: leader="Node at c728053a-382d-a4a2-d470-cf8beb8767be [Leader]" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: added peer, starting replication: peer=server-c880f81a-325f-f987-5534-6efe79a4aa0d testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: added peer, starting replication: peer=server-c880f81a-325f-f987-5534-6efe79a4aa0d testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: added peer, starting replication: peer=server-58633671-e0f2-ffc4-96f8-80811db6bec1 testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: added peer, starting replication: peer=server-58633671-e0f2-ffc4-96f8-80811db6bec1 testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: pipelining replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.477Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: pipelining replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.482Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.482Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.483Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering candidate state: node="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Candidate]" term=3 testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.483Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering candidate state: node="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Candidate]" term=3 testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.483Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.483Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.483Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.483Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.486Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.486Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.494Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.494Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.506Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.506Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.512Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.512Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.525Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.525Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.528Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to contact: server-id=server-c880f81a-325f-f987-5534-6efe79a4aa0d time=50.290407ms testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.528Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to contact: server-id=server-c880f81a-325f-f987-5534-6efe79a4aa0d time=50.290407ms testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.541Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.541Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to appendEntries to: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.553Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.553Z [ERROR] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to heartbeat to: peer=c880f81a-325f-f987-5534-6efe79a4aa0d error="failed to connect to peer: c880f81a-325f-f987-5534-6efe79a4aa0d" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: Election timeout reached, restarting election testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [WARN] server-c880f81a-325f-f987-5534-6efe79a4aa0d: Election timeout reached, restarting election testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering candidate state: node="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Candidate]" term=4 testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering candidate state: node="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Candidate]" term=4 testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-c728053a-382d-a4a2-d470-cf8beb8767be c728053a-382d-a4a2-d470-cf8beb8767be}" error="failed to connect to peer: c728053a-382d-a4a2-d470-cf8beb8767be" testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.560Z [ERROR] server-c880f81a-325f-f987-5534-6efe79a4aa0d: failed to make requestVote RPC: target="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" error="failed to connect to peer: 58633671-e0f2-ffc4-96f8-80811db6bec1" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.577Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to contact: server-id=server-c880f81a-325f-f987-5534-6efe79a4aa0d time=100.123976ms testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.577Z [WARN] server-c728053a-382d-a4a2-d470-cf8beb8767be: failed to contact: server-id=server-c880f81a-325f-f987-5534-6efe79a4aa0d time=100.123976ms testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.588Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.483154463 +0000 UTC m=+5.749370887 (1 nodes), 148.023088ms from start of poll, 439.768609ms from cluster start. Timeout at 2021-08-25 09:31:01.588328404 +0000 UTC m=+5.854544888, 105.174001ms after stability testing.go:149: cluster: 2021-08-25T09:31:01.588Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.483154463 +0000 UTC m=+5.749370887 (1 nodes), 148.023088ms from start of poll, 439.768609ms from cluster start. Timeout at 2021-08-25 09:31:01.588328404 +0000 UTC m=+5.854544888, 105.174001ms after stability raft_test.go:371: [INFO] Reconnecting Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Candidate] testLoggerAdapter verbose: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.592Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering follower state: follower="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Follower]" leader=c728053a-382d-a4a2-d470-cf8beb8767be testing.go:149: server-c880f81a-325f-f987-5534-6efe79a4aa0d: 2021-08-25T09:31:01.592Z [INFO] server-c880f81a-325f-f987-5534-6efe79a4aa0d: entering follower state: follower="Node at c880f81a-325f-f987-5534-6efe79a4aa0d [Follower]" leader=c728053a-382d-a4a2-d470-cf8beb8767be testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.592Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: pipelining replication: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.592Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: pipelining replication: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.597Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: aborting pipeline replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.597Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: aborting pipeline replication: peer="{Voter server-58633671-e0f2-ffc4-96f8-80811db6bec1 58633671-e0f2-ffc4-96f8-80811db6bec1}" testLoggerAdapter verbose: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.597Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: aborting pipeline replication: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" testing.go:149: server-c728053a-382d-a4a2-d470-cf8beb8767be: 2021-08-25T09:31:01.597Z [INFO] server-c728053a-382d-a4a2-d470-cf8beb8767be: aborting pipeline replication: peer="{Voter server-c880f81a-325f-f987-5534-6efe79a4aa0d c880f81a-325f-f987-5534-6efe79a4aa0d}" --- PASS: TestRaft_LeaderFail (0.56s) === RUN TestRaft_BehindFollower testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.601Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4a07c722-1d1e-b639-1c47-528b0a9a382d Address:4a07c722-1d1e-b639-1c47-528b0a9a382d} {Suffrage:Voter ID:server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 Address:e7880f18-7dcf-d42f-06e9-7ae18e1d1354} {Suffrage:Voter ID:server-a190ce1e-00f4-9803-b86b-4b844d5efd7c Address:a190ce1e-00f4-9803-b86b-4b844d5efd7c}]" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.601Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4a07c722-1d1e-b639-1c47-528b0a9a382d Address:4a07c722-1d1e-b639-1c47-528b0a9a382d} {Suffrage:Voter ID:server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 Address:e7880f18-7dcf-d42f-06e9-7ae18e1d1354} {Suffrage:Voter ID:server-a190ce1e-00f4-9803-b86b-4b844d5efd7c Address:a190ce1e-00f4-9803-b86b-4b844d5efd7c}]" testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.601Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: entering follower state: follower="Node at 4a07c722-1d1e-b639-1c47-528b0a9a382d [Follower]" leader= testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.601Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: entering follower state: follower="Node at 4a07c722-1d1e-b639-1c47-528b0a9a382d [Follower]" leader= testLoggerAdapter verbose: server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: 2021-08-25T09:31:01.602Z [INFO] server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4a07c722-1d1e-b639-1c47-528b0a9a382d Address:4a07c722-1d1e-b639-1c47-528b0a9a382d} {Suffrage:Voter ID:server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 Address:e7880f18-7dcf-d42f-06e9-7ae18e1d1354} {Suffrage:Voter ID:server-a190ce1e-00f4-9803-b86b-4b844d5efd7c Address:a190ce1e-00f4-9803-b86b-4b844d5efd7c}]" testing.go:149: server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: 2021-08-25T09:31:01.602Z [INFO] server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4a07c722-1d1e-b639-1c47-528b0a9a382d Address:4a07c722-1d1e-b639-1c47-528b0a9a382d} {Suffrage:Voter ID:server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 Address:e7880f18-7dcf-d42f-06e9-7ae18e1d1354} {Suffrage:Voter ID:server-a190ce1e-00f4-9803-b86b-4b844d5efd7c Address:a190ce1e-00f4-9803-b86b-4b844d5efd7c}]" testLoggerAdapter verbose: server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: 2021-08-25T09:31:01.602Z [INFO] server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: entering follower state: follower="Node at e7880f18-7dcf-d42f-06e9-7ae18e1d1354 [Follower]" leader= testing.go:149: server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: 2021-08-25T09:31:01.602Z [INFO] server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354: entering follower state: follower="Node at e7880f18-7dcf-d42f-06e9-7ae18e1d1354 [Follower]" leader= testLoggerAdapter verbose: server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: 2021-08-25T09:31:01.602Z [INFO] server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4a07c722-1d1e-b639-1c47-528b0a9a382d Address:4a07c722-1d1e-b639-1c47-528b0a9a382d} {Suffrage:Voter ID:server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 Address:e7880f18-7dcf-d42f-06e9-7ae18e1d1354} {Suffrage:Voter ID:server-a190ce1e-00f4-9803-b86b-4b844d5efd7c Address:a190ce1e-00f4-9803-b86b-4b844d5efd7c}]" testing.go:149: server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: 2021-08-25T09:31:01.602Z [INFO] server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4a07c722-1d1e-b639-1c47-528b0a9a382d Address:4a07c722-1d1e-b639-1c47-528b0a9a382d} {Suffrage:Voter ID:server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 Address:e7880f18-7dcf-d42f-06e9-7ae18e1d1354} {Suffrage:Voter ID:server-a190ce1e-00f4-9803-b86b-4b844d5efd7c Address:a190ce1e-00f4-9803-b86b-4b844d5efd7c}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.602Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:01.602Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: 2021-08-25T09:31:01.602Z [INFO] server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: entering follower state: follower="Node at a190ce1e-00f4-9803-b86b-4b844d5efd7c [Follower]" leader= testing.go:149: server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: 2021-08-25T09:31:01.602Z [INFO] server-a190ce1e-00f4-9803-b86b-4b844d5efd7c: entering follower state: follower="Node at a190ce1e-00f4-9803-b86b-4b844d5efd7c [Follower]" leader= testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [WARN] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [WARN] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: entering candidate state: node="Node at 4a07c722-1d1e-b639-1c47-528b0a9a382d [Candidate]" term=2 testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: entering candidate state: node="Node at 4a07c722-1d1e-b639-1c47-528b0a9a382d [Candidate]" term=2 testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: election won: tally=2 testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: election won: tally=2 testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: entering leader state: leader="Node at 4a07c722-1d1e-b639-1c47-528b0a9a382d [Leader]" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: entering leader state: leader="Node at 4a07c722-1d1e-b639-1c47-528b0a9a382d [Leader]" testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: added peer, starting replication: peer=server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: added peer, starting replication: peer=server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: added peer, starting replication: peer=server-a190ce1e-00f4-9803-b86b-4b844d5efd7c testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: added peer, starting replication: peer=server-a190ce1e-00f4-9803-b86b-4b844d5efd7c testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: pipelining replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.663Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: pipelining replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.664Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: pipelining replication: peer="{Voter server-a190ce1e-00f4-9803-b86b-4b844d5efd7c a190ce1e-00f4-9803-b86b-4b844d5efd7c}" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.664Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: pipelining replication: peer="{Voter server-a190ce1e-00f4-9803-b86b-4b844d5efd7c a190ce1e-00f4-9803-b86b-4b844d5efd7c}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.769Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.663715648 +0000 UTC m=+5.929932092 (1 nodes), 61.143725ms from start of poll, 62.492698ms from cluster start. Timeout at 2021-08-25 09:31:01.76899416 +0000 UTC m=+6.035210624, 105.278532ms after stability testing.go:149: cluster: 2021-08-25T09:31:01.769Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.663715648 +0000 UTC m=+5.929932092 (1 nodes), 61.143725ms from start of poll, 62.492698ms from cluster start. Timeout at 2021-08-25 09:31:01.76899416 +0000 UTC m=+6.035210624, 105.278532ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.769Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:01.769Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.874Z [INFO] stable state for Follower reached at 2021-08-25 09:31:01.769317109 +0000 UTC m=+6.035533493 (2 nodes), 3.579µs from start of poll, 168.094099ms from cluster start. Timeout at 2021-08-25 09:31:01.874356003 +0000 UTC m=+6.140572407, 105.038914ms after stability testing.go:149: cluster: 2021-08-25T09:31:01.874Z [INFO] stable state for Follower reached at 2021-08-25 09:31:01.769317109 +0000 UTC m=+6.035533493 (2 nodes), 3.579µs from start of poll, 168.094099ms from cluster start. Timeout at 2021-08-25 09:31:01.874356003 +0000 UTC m=+6.140572407, 105.038914ms after stability testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.874Z [ERROR] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: failed to pipeline appendEntries: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" error="append pipeline closed" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.874Z [ERROR] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: failed to pipeline appendEntries: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" error="append pipeline closed" testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.874Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: aborting pipeline replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.874Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: aborting pipeline replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.874Z [ERROR] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: failed to appendEntries to: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" error="failed to connect to peer: e7880f18-7dcf-d42f-06e9-7ae18e1d1354" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.874Z [ERROR] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: failed to appendEntries to: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" error="failed to connect to peer: e7880f18-7dcf-d42f-06e9-7ae18e1d1354" raft_test.go:420: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.885Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: pipelining replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.885Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: pipelining replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.894Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:01.894Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:01.999Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.894405956 +0000 UTC m=+6.160622440 (1 nodes), 15.459µs from start of poll, 293.183046ms from cluster start. Timeout at 2021-08-25 09:31:01.999533881 +0000 UTC m=+6.265750325, 105.127885ms after stability testing.go:149: cluster: 2021-08-25T09:31:01.999Z [INFO] stable state for Leader reached at 2021-08-25 09:31:01.894405956 +0000 UTC m=+6.160622440 (1 nodes), 15.459µs from start of poll, 293.183046ms from cluster start. Timeout at 2021-08-25 09:31:01.999533881 +0000 UTC m=+6.265750325, 105.127885ms after stability testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.999Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: aborting pipeline replication: peer="{Voter server-a190ce1e-00f4-9803-b86b-4b844d5efd7c a190ce1e-00f4-9803-b86b-4b844d5efd7c}" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.999Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: aborting pipeline replication: peer="{Voter server-a190ce1e-00f4-9803-b86b-4b844d5efd7c a190ce1e-00f4-9803-b86b-4b844d5efd7c}" testLoggerAdapter verbose: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.999Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: aborting pipeline replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" testing.go:149: server-4a07c722-1d1e-b639-1c47-528b0a9a382d: 2021-08-25T09:31:01.999Z [INFO] server-4a07c722-1d1e-b639-1c47-528b0a9a382d: aborting pipeline replication: peer="{Voter server-e7880f18-7dcf-d42f-06e9-7ae18e1d1354 e7880f18-7dcf-d42f-06e9-7ae18e1d1354}" --- PASS: TestRaft_BehindFollower (0.40s) === RUN TestRaft_ApplyNonLeader testLoggerAdapter verbose: server-4bb98f98-907a-c904-eb90-f9587034a211: 2021-08-25T09:31:02.004Z [INFO] server-4bb98f98-907a-c904-eb90-f9587034a211: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4bb98f98-907a-c904-eb90-f9587034a211 Address:4bb98f98-907a-c904-eb90-f9587034a211} {Suffrage:Voter ID:server-625ecbd8-5b58-6828-6e89-ac680de13424 Address:625ecbd8-5b58-6828-6e89-ac680de13424} {Suffrage:Voter ID:server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 Address:4b1b7608-b90b-e9db-134a-6cbc2ea994d8}]" testing.go:149: server-4bb98f98-907a-c904-eb90-f9587034a211: 2021-08-25T09:31:02.004Z [INFO] server-4bb98f98-907a-c904-eb90-f9587034a211: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4bb98f98-907a-c904-eb90-f9587034a211 Address:4bb98f98-907a-c904-eb90-f9587034a211} {Suffrage:Voter ID:server-625ecbd8-5b58-6828-6e89-ac680de13424 Address:625ecbd8-5b58-6828-6e89-ac680de13424} {Suffrage:Voter ID:server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 Address:4b1b7608-b90b-e9db-134a-6cbc2ea994d8}]" testLoggerAdapter verbose: server-4bb98f98-907a-c904-eb90-f9587034a211: 2021-08-25T09:31:02.004Z [INFO] server-4bb98f98-907a-c904-eb90-f9587034a211: entering follower state: follower="Node at 4bb98f98-907a-c904-eb90-f9587034a211 [Follower]" leader= testing.go:149: server-4bb98f98-907a-c904-eb90-f9587034a211: 2021-08-25T09:31:02.004Z [INFO] server-4bb98f98-907a-c904-eb90-f9587034a211: entering follower state: follower="Node at 4bb98f98-907a-c904-eb90-f9587034a211 [Follower]" leader= testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.004Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4bb98f98-907a-c904-eb90-f9587034a211 Address:4bb98f98-907a-c904-eb90-f9587034a211} {Suffrage:Voter ID:server-625ecbd8-5b58-6828-6e89-ac680de13424 Address:625ecbd8-5b58-6828-6e89-ac680de13424} {Suffrage:Voter ID:server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 Address:4b1b7608-b90b-e9db-134a-6cbc2ea994d8}]" testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.004Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4bb98f98-907a-c904-eb90-f9587034a211 Address:4bb98f98-907a-c904-eb90-f9587034a211} {Suffrage:Voter ID:server-625ecbd8-5b58-6828-6e89-ac680de13424 Address:625ecbd8-5b58-6828-6e89-ac680de13424} {Suffrage:Voter ID:server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 Address:4b1b7608-b90b-e9db-134a-6cbc2ea994d8}]" testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.005Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: entering follower state: follower="Node at 625ecbd8-5b58-6828-6e89-ac680de13424 [Follower]" leader= testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.005Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: entering follower state: follower="Node at 625ecbd8-5b58-6828-6e89-ac680de13424 [Follower]" leader= testLoggerAdapter verbose: server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: 2021-08-25T09:31:02.005Z [INFO] server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4bb98f98-907a-c904-eb90-f9587034a211 Address:4bb98f98-907a-c904-eb90-f9587034a211} {Suffrage:Voter ID:server-625ecbd8-5b58-6828-6e89-ac680de13424 Address:625ecbd8-5b58-6828-6e89-ac680de13424} {Suffrage:Voter ID:server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 Address:4b1b7608-b90b-e9db-134a-6cbc2ea994d8}]" testing.go:149: server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: 2021-08-25T09:31:02.005Z [INFO] server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4bb98f98-907a-c904-eb90-f9587034a211 Address:4bb98f98-907a-c904-eb90-f9587034a211} {Suffrage:Voter ID:server-625ecbd8-5b58-6828-6e89-ac680de13424 Address:625ecbd8-5b58-6828-6e89-ac680de13424} {Suffrage:Voter ID:server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 Address:4b1b7608-b90b-e9db-134a-6cbc2ea994d8}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.005Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:02.005Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: 2021-08-25T09:31:02.005Z [INFO] server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: entering follower state: follower="Node at 4b1b7608-b90b-e9db-134a-6cbc2ea994d8 [Follower]" leader= testing.go:149: server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: 2021-08-25T09:31:02.005Z [INFO] server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8: entering follower state: follower="Node at 4b1b7608-b90b-e9db-134a-6cbc2ea994d8 [Follower]" leader= testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [WARN] server-625ecbd8-5b58-6828-6e89-ac680de13424: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [WARN] server-625ecbd8-5b58-6828-6e89-ac680de13424: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: entering candidate state: node="Node at 625ecbd8-5b58-6828-6e89-ac680de13424 [Candidate]" term=2 testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: entering candidate state: node="Node at 625ecbd8-5b58-6828-6e89-ac680de13424 [Candidate]" term=2 testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: election won: tally=2 testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: election won: tally=2 testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: entering leader state: leader="Node at 625ecbd8-5b58-6828-6e89-ac680de13424 [Leader]" testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: entering leader state: leader="Node at 625ecbd8-5b58-6828-6e89-ac680de13424 [Leader]" testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: added peer, starting replication: peer=server-4bb98f98-907a-c904-eb90-f9587034a211 testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: added peer, starting replication: peer=server-4bb98f98-907a-c904-eb90-f9587034a211 testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: added peer, starting replication: peer=server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.058Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: added peer, starting replication: peer=server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.059Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: pipelining replication: peer="{Voter server-4bb98f98-907a-c904-eb90-f9587034a211 4bb98f98-907a-c904-eb90-f9587034a211}" testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.059Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: pipelining replication: peer="{Voter server-4bb98f98-907a-c904-eb90-f9587034a211 4bb98f98-907a-c904-eb90-f9587034a211}" testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.059Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: pipelining replication: peer="{Voter server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 4b1b7608-b90b-e9db-134a-6cbc2ea994d8}" testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.059Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: pipelining replication: peer="{Voter server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 4b1b7608-b90b-e9db-134a-6cbc2ea994d8}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.164Z [INFO] stable state for Leader reached at 2021-08-25 09:31:02.058863137 +0000 UTC m=+6.325079541 (1 nodes), 53.340639ms from start of poll, 54.696412ms from cluster start. Timeout at 2021-08-25 09:31:02.163994942 +0000 UTC m=+6.430211426, 105.131885ms after stability testing.go:149: cluster: 2021-08-25T09:31:02.164Z [INFO] stable state for Leader reached at 2021-08-25 09:31:02.058863137 +0000 UTC m=+6.325079541 (1 nodes), 53.340639ms from start of poll, 54.696412ms from cluster start. Timeout at 2021-08-25 09:31:02.163994942 +0000 UTC m=+6.430211426, 105.131885ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.164Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:02.164Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.269Z [INFO] stable state for Follower reached at 2021-08-25 09:31:02.164266137 +0000 UTC m=+6.430482541 (2 nodes), 3.06µs from start of poll, 160.099412ms from cluster start. Timeout at 2021-08-25 09:31:02.269285412 +0000 UTC m=+6.535501896, 105.019355ms after stability testing.go:149: cluster: 2021-08-25T09:31:02.269Z [INFO] stable state for Follower reached at 2021-08-25 09:31:02.164266137 +0000 UTC m=+6.430482541 (2 nodes), 3.06µs from start of poll, 160.099412ms from cluster start. Timeout at 2021-08-25 09:31:02.269285412 +0000 UTC m=+6.535501896, 105.019355ms after stability testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.269Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: aborting pipeline replication: peer="{Voter server-4bb98f98-907a-c904-eb90-f9587034a211 4bb98f98-907a-c904-eb90-f9587034a211}" testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.269Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: aborting pipeline replication: peer="{Voter server-4bb98f98-907a-c904-eb90-f9587034a211 4bb98f98-907a-c904-eb90-f9587034a211}" testLoggerAdapter verbose: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.269Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: aborting pipeline replication: peer="{Voter server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 4b1b7608-b90b-e9db-134a-6cbc2ea994d8}" testing.go:149: server-625ecbd8-5b58-6828-6e89-ac680de13424: 2021-08-25T09:31:02.269Z [INFO] server-625ecbd8-5b58-6828-6e89-ac680de13424: aborting pipeline replication: peer="{Voter server-4b1b7608-b90b-e9db-134a-6cbc2ea994d8 4b1b7608-b90b-e9db-134a-6cbc2ea994d8}" --- PASS: TestRaft_ApplyNonLeader (0.27s) === RUN TestRaft_ApplyConcurrent testLoggerAdapter verbose: server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: 2021-08-25T09:31:02.274Z [INFO] server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0af17e88-12bf-32ef-b543-a3c70f4aa29d Address:0af17e88-12bf-32ef-b543-a3c70f4aa29d} {Suffrage:Voter ID:server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 Address:c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52} {Suffrage:Voter ID:server-ca44bb78-0870-8913-45b0-9e9a2a417d49 Address:ca44bb78-0870-8913-45b0-9e9a2a417d49}]" testing.go:149: server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: 2021-08-25T09:31:02.274Z [INFO] server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0af17e88-12bf-32ef-b543-a3c70f4aa29d Address:0af17e88-12bf-32ef-b543-a3c70f4aa29d} {Suffrage:Voter ID:server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 Address:c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52} {Suffrage:Voter ID:server-ca44bb78-0870-8913-45b0-9e9a2a417d49 Address:ca44bb78-0870-8913-45b0-9e9a2a417d49}]" testLoggerAdapter verbose: server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: 2021-08-25T09:31:02.274Z [INFO] server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: entering follower state: follower="Node at 0af17e88-12bf-32ef-b543-a3c70f4aa29d [Follower]" leader= testing.go:149: server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: 2021-08-25T09:31:02.274Z [INFO] server-0af17e88-12bf-32ef-b543-a3c70f4aa29d: entering follower state: follower="Node at 0af17e88-12bf-32ef-b543-a3c70f4aa29d [Follower]" leader= testLoggerAdapter verbose: server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: 2021-08-25T09:31:02.274Z [INFO] server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0af17e88-12bf-32ef-b543-a3c70f4aa29d Address:0af17e88-12bf-32ef-b543-a3c70f4aa29d} {Suffrage:Voter ID:server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 Address:c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52} {Suffrage:Voter ID:server-ca44bb78-0870-8913-45b0-9e9a2a417d49 Address:ca44bb78-0870-8913-45b0-9e9a2a417d49}]" testing.go:149: server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: 2021-08-25T09:31:02.274Z [INFO] server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0af17e88-12bf-32ef-b543-a3c70f4aa29d Address:0af17e88-12bf-32ef-b543-a3c70f4aa29d} {Suffrage:Voter ID:server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 Address:c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52} {Suffrage:Voter ID:server-ca44bb78-0870-8913-45b0-9e9a2a417d49 Address:ca44bb78-0870-8913-45b0-9e9a2a417d49}]" testLoggerAdapter verbose: server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: 2021-08-25T09:31:02.274Z [INFO] server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: entering follower state: follower="Node at c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 [Follower]" leader= testing.go:149: server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: 2021-08-25T09:31:02.274Z [INFO] server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52: entering follower state: follower="Node at c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 [Follower]" leader= testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.274Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0af17e88-12bf-32ef-b543-a3c70f4aa29d Address:0af17e88-12bf-32ef-b543-a3c70f4aa29d} {Suffrage:Voter ID:server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 Address:c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52} {Suffrage:Voter ID:server-ca44bb78-0870-8913-45b0-9e9a2a417d49 Address:ca44bb78-0870-8913-45b0-9e9a2a417d49}]" testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.274Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0af17e88-12bf-32ef-b543-a3c70f4aa29d Address:0af17e88-12bf-32ef-b543-a3c70f4aa29d} {Suffrage:Voter ID:server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 Address:c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52} {Suffrage:Voter ID:server-ca44bb78-0870-8913-45b0-9e9a2a417d49 Address:ca44bb78-0870-8913-45b0-9e9a2a417d49}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.275Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:02.275Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.275Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: entering follower state: follower="Node at ca44bb78-0870-8913-45b0-9e9a2a417d49 [Follower]" leader= testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.275Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: entering follower state: follower="Node at ca44bb78-0870-8913-45b0-9e9a2a417d49 [Follower]" leader= testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [WARN] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [WARN] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: entering candidate state: node="Node at ca44bb78-0870-8913-45b0-9e9a2a417d49 [Candidate]" term=2 testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: entering candidate state: node="Node at ca44bb78-0870-8913-45b0-9e9a2a417d49 [Candidate]" term=2 testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: election won: tally=2 testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: election won: tally=2 testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: entering leader state: leader="Node at ca44bb78-0870-8913-45b0-9e9a2a417d49 [Leader]" testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: entering leader state: leader="Node at ca44bb78-0870-8913-45b0-9e9a2a417d49 [Leader]" testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: added peer, starting replication: peer=server-0af17e88-12bf-32ef-b543-a3c70f4aa29d testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: added peer, starting replication: peer=server-0af17e88-12bf-32ef-b543-a3c70f4aa29d testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: added peer, starting replication: peer=server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: added peer, starting replication: peer=server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: pipelining replication: peer="{Voter server-0af17e88-12bf-32ef-b543-a3c70f4aa29d 0af17e88-12bf-32ef-b543-a3c70f4aa29d}" testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: pipelining replication: peer="{Voter server-0af17e88-12bf-32ef-b543-a3c70f4aa29d 0af17e88-12bf-32ef-b543-a3c70f4aa29d}" testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: pipelining replication: peer="{Voter server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52}" testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.390Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: pipelining replication: peer="{Voter server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.596Z [INFO] stable state for Leader reached at 2021-08-25 09:31:02.390785697 +0000 UTC m=+6.657002121 (1 nodes), 115.554064ms from start of poll, 116.988909ms from cluster start. Timeout at 2021-08-25 09:31:02.595973365 +0000 UTC m=+6.862189809, 205.187688ms after stability testing.go:149: cluster: 2021-08-25T09:31:02.596Z [INFO] stable state for Leader reached at 2021-08-25 09:31:02.390785697 +0000 UTC m=+6.657002121 (1 nodes), 115.554064ms from start of poll, 116.988909ms from cluster start. Timeout at 2021-08-25 09:31:02.595973365 +0000 UTC m=+6.862189809, 205.187688ms after stability testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.608Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: aborting pipeline replication: peer="{Voter server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52}" testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.608Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: aborting pipeline replication: peer="{Voter server-c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52 c65d1fb8-e8ef-f4a8-2a1d-9de701e1af52}" testLoggerAdapter verbose: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.608Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: aborting pipeline replication: peer="{Voter server-0af17e88-12bf-32ef-b543-a3c70f4aa29d 0af17e88-12bf-32ef-b543-a3c70f4aa29d}" testing.go:149: server-ca44bb78-0870-8913-45b0-9e9a2a417d49: 2021-08-25T09:31:02.608Z [INFO] server-ca44bb78-0870-8913-45b0-9e9a2a417d49: aborting pipeline replication: peer="{Voter server-0af17e88-12bf-32ef-b543-a3c70f4aa29d 0af17e88-12bf-32ef-b543-a3c70f4aa29d}" --- PASS: TestRaft_ApplyConcurrent (0.34s) === RUN TestRaft_ApplyConcurrent_Timeout testLoggerAdapter verbose: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.611Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-26e42226-8c40-0211-14dd-2e25486586eb Address:26e42226-8c40-0211-14dd-2e25486586eb}]" testing.go:149: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.611Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-26e42226-8c40-0211-14dd-2e25486586eb Address:26e42226-8c40-0211-14dd-2e25486586eb}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.611Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:02.611Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.611Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: entering follower state: follower="Node at 26e42226-8c40-0211-14dd-2e25486586eb [Follower]" leader= testing.go:149: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.611Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: entering follower state: follower="Node at 26e42226-8c40-0211-14dd-2e25486586eb [Follower]" leader= testLoggerAdapter verbose: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.786Z [WARN] server-26e42226-8c40-0211-14dd-2e25486586eb: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.786Z [WARN] server-26e42226-8c40-0211-14dd-2e25486586eb: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.786Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: entering candidate state: node="Node at 26e42226-8c40-0211-14dd-2e25486586eb [Candidate]" term=2 testing.go:149: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.786Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: entering candidate state: node="Node at 26e42226-8c40-0211-14dd-2e25486586eb [Candidate]" term=2 testLoggerAdapter verbose: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.786Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: election won: tally=1 testing.go:149: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.786Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: election won: tally=1 testLoggerAdapter verbose: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.787Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: entering leader state: leader="Node at 26e42226-8c40-0211-14dd-2e25486586eb [Leader]" testing.go:149: server-26e42226-8c40-0211-14dd-2e25486586eb: 2021-08-25T09:31:02.787Z [INFO] server-26e42226-8c40-0211-14dd-2e25486586eb: entering leader state: leader="Node at 26e42226-8c40-0211-14dd-2e25486586eb [Leader]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.988Z [INFO] stable state for Leader reached at 2021-08-25 09:31:02.787082398 +0000 UTC m=+7.053298822 (1 nodes), 175.446487ms from start of poll, 175.913983ms from cluster start. Timeout at 2021-08-25 09:31:02.988202548 +0000 UTC m=+7.254419012, 201.12019ms after stability testing.go:149: cluster: 2021-08-25T09:31:02.988Z [INFO] stable state for Leader reached at 2021-08-25 09:31:02.787082398 +0000 UTC m=+7.053298822 (1 nodes), 175.446487ms from start of poll, 175.913983ms from cluster start. Timeout at 2021-08-25 09:31:02.988202548 +0000 UTC m=+7.254419012, 201.12019ms after stability --- PASS: TestRaft_ApplyConcurrent_Timeout (0.38s) === RUN TestRaft_JoinNode testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:02.991Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c983cac5-6e9a-1659-243f-896ef4285906 Address:c983cac5-6e9a-1659-243f-896ef4285906} {Suffrage:Voter ID:server-6dc2cf86-8bae-cc4e-8127-b5b754df438e Address:6dc2cf86-8bae-cc4e-8127-b5b754df438e}]" testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:02.991Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c983cac5-6e9a-1659-243f-896ef4285906 Address:c983cac5-6e9a-1659-243f-896ef4285906} {Suffrage:Voter ID:server-6dc2cf86-8bae-cc4e-8127-b5b754df438e Address:6dc2cf86-8bae-cc4e-8127-b5b754df438e}]" testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:02.991Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: entering follower state: follower="Node at c983cac5-6e9a-1659-243f-896ef4285906 [Follower]" leader= testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:02.991Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: entering follower state: follower="Node at c983cac5-6e9a-1659-243f-896ef4285906 [Follower]" leader= testLoggerAdapter verbose: server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: 2021-08-25T09:31:02.991Z [INFO] server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c983cac5-6e9a-1659-243f-896ef4285906 Address:c983cac5-6e9a-1659-243f-896ef4285906} {Suffrage:Voter ID:server-6dc2cf86-8bae-cc4e-8127-b5b754df438e Address:6dc2cf86-8bae-cc4e-8127-b5b754df438e}]" testing.go:149: server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: 2021-08-25T09:31:02.991Z [INFO] server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c983cac5-6e9a-1659-243f-896ef4285906 Address:c983cac5-6e9a-1659-243f-896ef4285906} {Suffrage:Voter ID:server-6dc2cf86-8bae-cc4e-8127-b5b754df438e Address:6dc2cf86-8bae-cc4e-8127-b5b754df438e}]" testLoggerAdapter verbose: server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: 2021-08-25T09:31:02.991Z [INFO] server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: entering follower state: follower="Node at 6dc2cf86-8bae-cc4e-8127-b5b754df438e [Follower]" leader= testing.go:149: server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: 2021-08-25T09:31:02.991Z [INFO] server-6dc2cf86-8bae-cc4e-8127-b5b754df438e: entering follower state: follower="Node at 6dc2cf86-8bae-cc4e-8127-b5b754df438e [Follower]" leader= testLoggerAdapter verbose: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:02.992Z [INFO] server-5638b3ad-b602-3f2b-a735-5da284e5a868: initial configuration: index=0 servers=[] testing.go:149: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:02.992Z [INFO] server-5638b3ad-b602-3f2b-a735-5da284e5a868: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2021-08-25T09:31:02.992Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:02.992Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:02.992Z [INFO] server-5638b3ad-b602-3f2b-a735-5da284e5a868: entering follower state: follower="Node at 5638b3ad-b602-3f2b-a735-5da284e5a868 [Follower]" leader= testing.go:149: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:02.992Z [INFO] server-5638b3ad-b602-3f2b-a735-5da284e5a868: entering follower state: follower="Node at 5638b3ad-b602-3f2b-a735-5da284e5a868 [Follower]" leader= testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [WARN] server-c983cac5-6e9a-1659-243f-896ef4285906: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [WARN] server-c983cac5-6e9a-1659-243f-896ef4285906: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: entering candidate state: node="Node at c983cac5-6e9a-1659-243f-896ef4285906 [Candidate]" term=2 testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: entering candidate state: node="Node at c983cac5-6e9a-1659-243f-896ef4285906 [Candidate]" term=2 testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: election won: tally=2 testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: election won: tally=2 testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: entering leader state: leader="Node at c983cac5-6e9a-1659-243f-896ef4285906 [Leader]" testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: entering leader state: leader="Node at c983cac5-6e9a-1659-243f-896ef4285906 [Leader]" testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: added peer, starting replication: peer=server-6dc2cf86-8bae-cc4e-8127-b5b754df438e testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: added peer, starting replication: peer=server-6dc2cf86-8bae-cc4e-8127-b5b754df438e testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: pipelining replication: peer="{Voter server-6dc2cf86-8bae-cc4e-8127-b5b754df438e 6dc2cf86-8bae-cc4e-8127-b5b754df438e}" testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.050Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: pipelining replication: peer="{Voter server-6dc2cf86-8bae-cc4e-8127-b5b754df438e 6dc2cf86-8bae-cc4e-8127-b5b754df438e}" testLoggerAdapter verbose: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:03.090Z [WARN] server-5638b3ad-b602-3f2b-a735-5da284e5a868: no known peers, aborting election testing.go:149: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:03.090Z [WARN] server-5638b3ad-b602-3f2b-a735-5da284e5a868: no known peers, aborting election testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.155Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.050559339 +0000 UTC m=+7.316775783 (1 nodes), 57.925808ms from start of poll, 59.623728ms from cluster start. Timeout at 2021-08-25 09:31:03.155705643 +0000 UTC m=+7.421922087, 105.146304ms after stability testing.go:149: cluster: 2021-08-25T09:31:03.155Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.050559339 +0000 UTC m=+7.316775783 (1 nodes), 57.925808ms from start of poll, 59.623728ms from cluster start. Timeout at 2021-08-25 09:31:03.155705643 +0000 UTC m=+7.421922087, 105.146304ms after stability testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.155Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: updating configuration: command=AddStaging server-id=server-5638b3ad-b602-3f2b-a735-5da284e5a868 server-addr=5638b3ad-b602-3f2b-a735-5da284e5a868 servers="[{Suffrage:Voter ID:server-c983cac5-6e9a-1659-243f-896ef4285906 Address:c983cac5-6e9a-1659-243f-896ef4285906} {Suffrage:Voter ID:server-6dc2cf86-8bae-cc4e-8127-b5b754df438e Address:6dc2cf86-8bae-cc4e-8127-b5b754df438e} {Suffrage:Voter ID:server-5638b3ad-b602-3f2b-a735-5da284e5a868 Address:5638b3ad-b602-3f2b-a735-5da284e5a868}]" testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.155Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: updating configuration: command=AddStaging server-id=server-5638b3ad-b602-3f2b-a735-5da284e5a868 server-addr=5638b3ad-b602-3f2b-a735-5da284e5a868 servers="[{Suffrage:Voter ID:server-c983cac5-6e9a-1659-243f-896ef4285906 Address:c983cac5-6e9a-1659-243f-896ef4285906} {Suffrage:Voter ID:server-6dc2cf86-8bae-cc4e-8127-b5b754df438e Address:6dc2cf86-8bae-cc4e-8127-b5b754df438e} {Suffrage:Voter ID:server-5638b3ad-b602-3f2b-a735-5da284e5a868 Address:5638b3ad-b602-3f2b-a735-5da284e5a868}]" testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.156Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: added peer, starting replication: peer=server-5638b3ad-b602-3f2b-a735-5da284e5a868 testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.156Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: added peer, starting replication: peer=server-5638b3ad-b602-3f2b-a735-5da284e5a868 testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.156Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:03.156Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:03.156Z [WARN] server-5638b3ad-b602-3f2b-a735-5da284e5a868: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-5638b3ad-b602-3f2b-a735-5da284e5a868: 2021-08-25T09:31:03.156Z [WARN] server-5638b3ad-b602-3f2b-a735-5da284e5a868: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.156Z [WARN] server-c983cac5-6e9a-1659-243f-896ef4285906: appendEntries rejected, sending older logs: peer="{Voter server-5638b3ad-b602-3f2b-a735-5da284e5a868 5638b3ad-b602-3f2b-a735-5da284e5a868}" next=1 testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.156Z [WARN] server-c983cac5-6e9a-1659-243f-896ef4285906: appendEntries rejected, sending older logs: peer="{Voter server-5638b3ad-b602-3f2b-a735-5da284e5a868 5638b3ad-b602-3f2b-a735-5da284e5a868}" next=1 testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.156Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: pipelining replication: peer="{Voter server-5638b3ad-b602-3f2b-a735-5da284e5a868 5638b3ad-b602-3f2b-a735-5da284e5a868}" testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.156Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: pipelining replication: peer="{Voter server-5638b3ad-b602-3f2b-a735-5da284e5a868 5638b3ad-b602-3f2b-a735-5da284e5a868}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.261Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.156431615 +0000 UTC m=+7.422648019 (1 nodes), 1.84µs from start of poll, 165.495964ms from cluster start. Timeout at 2021-08-25 09:31:03.261601696 +0000 UTC m=+7.527818160, 105.170141ms after stability testing.go:149: cluster: 2021-08-25T09:31:03.261Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.156431615 +0000 UTC m=+7.422648019 (1 nodes), 1.84µs from start of poll, 165.495964ms from cluster start. Timeout at 2021-08-25 09:31:03.261601696 +0000 UTC m=+7.527818160, 105.170141ms after stability testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.261Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: aborting pipeline replication: peer="{Voter server-6dc2cf86-8bae-cc4e-8127-b5b754df438e 6dc2cf86-8bae-cc4e-8127-b5b754df438e}" testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.261Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: aborting pipeline replication: peer="{Voter server-6dc2cf86-8bae-cc4e-8127-b5b754df438e 6dc2cf86-8bae-cc4e-8127-b5b754df438e}" testLoggerAdapter verbose: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.261Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: aborting pipeline replication: peer="{Voter server-5638b3ad-b602-3f2b-a735-5da284e5a868 5638b3ad-b602-3f2b-a735-5da284e5a868}" testing.go:149: server-c983cac5-6e9a-1659-243f-896ef4285906: 2021-08-25T09:31:03.261Z [INFO] server-c983cac5-6e9a-1659-243f-896ef4285906: aborting pipeline replication: peer="{Voter server-5638b3ad-b602-3f2b-a735-5da284e5a868 5638b3ad-b602-3f2b-a735-5da284e5a868}" --- PASS: TestRaft_JoinNode (0.27s) === RUN TestRaft_JoinNode_ConfigStore testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.264Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-63c344e4-4e0d-987f-e671-609be6b63c40 Address:63c344e4-4e0d-987f-e671-609be6b63c40}]" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.264Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-63c344e4-4e0d-987f-e671-609be6b63c40 Address:63c344e4-4e0d-987f-e671-609be6b63c40}]" testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.264Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: entering follower state: follower="Node at 63c344e4-4e0d-987f-e671-609be6b63c40 [Follower]" leader= testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.264Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: entering follower state: follower="Node at 63c344e4-4e0d-987f-e671-609be6b63c40 [Follower]" leader= testLoggerAdapter verbose: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.265Z [INFO] server-77035e64-5963-ae15-507b-66131657cc44: initial configuration: index=0 servers=[] testing.go:149: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.265Z [INFO] server-77035e64-5963-ae15-507b-66131657cc44: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.265Z [INFO] server-77035e64-5963-ae15-507b-66131657cc44: entering follower state: follower="Node at 77035e64-5963-ae15-507b-66131657cc44 [Follower]" leader= testing.go:149: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.265Z [INFO] server-77035e64-5963-ae15-507b-66131657cc44: entering follower state: follower="Node at 77035e64-5963-ae15-507b-66131657cc44 [Follower]" leader= testLoggerAdapter verbose: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.266Z [INFO] server-7da55b48-5800-9531-12b8-e97106fd93b0: initial configuration: index=0 servers=[] testing.go:149: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.266Z [INFO] server-7da55b48-5800-9531-12b8-e97106fd93b0: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.267Z [INFO] server-7da55b48-5800-9531-12b8-e97106fd93b0: entering follower state: follower="Node at 7da55b48-5800-9531-12b8-e97106fd93b0 [Follower]" leader= testing.go:149: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.267Z [INFO] server-7da55b48-5800-9531-12b8-e97106fd93b0: entering follower state: follower="Node at 7da55b48-5800-9531-12b8-e97106fd93b0 [Follower]" leader= testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.268Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:03.268Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.325Z [WARN] server-77035e64-5963-ae15-507b-66131657cc44: no known peers, aborting election testing.go:149: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.325Z [WARN] server-77035e64-5963-ae15-507b-66131657cc44: no known peers, aborting election testLoggerAdapter verbose: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.336Z [WARN] server-7da55b48-5800-9531-12b8-e97106fd93b0: no known peers, aborting election testing.go:149: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.336Z [WARN] server-7da55b48-5800-9531-12b8-e97106fd93b0: no known peers, aborting election testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [WARN] server-63c344e4-4e0d-987f-e671-609be6b63c40: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [WARN] server-63c344e4-4e0d-987f-e671-609be6b63c40: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: entering candidate state: node="Node at 63c344e4-4e0d-987f-e671-609be6b63c40 [Candidate]" term=2 testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: entering candidate state: node="Node at 63c344e4-4e0d-987f-e671-609be6b63c40 [Candidate]" term=2 testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: election won: tally=1 testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: election won: tally=1 testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: entering leader state: leader="Node at 63c344e4-4e0d-987f-e671-609be6b63c40 [Leader]" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.358Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: entering leader state: leader="Node at 63c344e4-4e0d-987f-e671-609be6b63c40 [Leader]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.464Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.358806847 +0000 UTC m=+7.625023251 (1 nodes), 90.664306ms from start of poll, 94.3415ms from cluster start. Timeout at 2021-08-25 09:31:03.463946272 +0000 UTC m=+7.730162776, 105.139525ms after stability testing.go:149: cluster: 2021-08-25T09:31:03.464Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.358806847 +0000 UTC m=+7.625023251 (1 nodes), 90.664306ms from start of poll, 94.3415ms from cluster start. Timeout at 2021-08-25 09:31:03.463946272 +0000 UTC m=+7.730162776, 105.139525ms after stability testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.464Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: updating configuration: command=AddStaging server-id=server-77035e64-5963-ae15-507b-66131657cc44 server-addr=77035e64-5963-ae15-507b-66131657cc44 servers="[{Suffrage:Voter ID:server-63c344e4-4e0d-987f-e671-609be6b63c40 Address:63c344e4-4e0d-987f-e671-609be6b63c40} {Suffrage:Voter ID:server-77035e64-5963-ae15-507b-66131657cc44 Address:77035e64-5963-ae15-507b-66131657cc44}]" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.464Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: updating configuration: command=AddStaging server-id=server-77035e64-5963-ae15-507b-66131657cc44 server-addr=77035e64-5963-ae15-507b-66131657cc44 servers="[{Suffrage:Voter ID:server-63c344e4-4e0d-987f-e671-609be6b63c40 Address:63c344e4-4e0d-987f-e671-609be6b63c40} {Suffrage:Voter ID:server-77035e64-5963-ae15-507b-66131657cc44 Address:77035e64-5963-ae15-507b-66131657cc44}]" testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.464Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: added peer, starting replication: peer=server-77035e64-5963-ae15-507b-66131657cc44 testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.464Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: added peer, starting replication: peer=server-77035e64-5963-ae15-507b-66131657cc44 testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.464Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.464Z [WARN] server-77035e64-5963-ae15-507b-66131657cc44: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: cluster: 2021-08-25T09:31:03.464Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-77035e64-5963-ae15-507b-66131657cc44: 2021-08-25T09:31:03.464Z [WARN] server-77035e64-5963-ae15-507b-66131657cc44: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.464Z [WARN] server-63c344e4-4e0d-987f-e671-609be6b63c40: appendEntries rejected, sending older logs: peer="{Voter server-77035e64-5963-ae15-507b-66131657cc44 77035e64-5963-ae15-507b-66131657cc44}" next=1 testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.464Z [WARN] server-63c344e4-4e0d-987f-e671-609be6b63c40: appendEntries rejected, sending older logs: peer="{Voter server-77035e64-5963-ae15-507b-66131657cc44 77035e64-5963-ae15-507b-66131657cc44}" next=1 testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.465Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: pipelining replication: peer="{Voter server-77035e64-5963-ae15-507b-66131657cc44 77035e64-5963-ae15-507b-66131657cc44}" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.465Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: pipelining replication: peer="{Voter server-77035e64-5963-ae15-507b-66131657cc44 77035e64-5963-ae15-507b-66131657cc44}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.570Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.464793772 +0000 UTC m=+7.731010176 (1 nodes), 1.98µs from start of poll, 200.328425ms from cluster start. Timeout at 2021-08-25 09:31:03.569904599 +0000 UTC m=+7.836121003, 105.110827ms after stability testing.go:149: cluster: 2021-08-25T09:31:03.570Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.464793772 +0000 UTC m=+7.731010176 (1 nodes), 1.98µs from start of poll, 200.328425ms from cluster start. Timeout at 2021-08-25 09:31:03.569904599 +0000 UTC m=+7.836121003, 105.110827ms after stability testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.570Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: updating configuration: command=AddStaging server-id=server-7da55b48-5800-9531-12b8-e97106fd93b0 server-addr=7da55b48-5800-9531-12b8-e97106fd93b0 servers="[{Suffrage:Voter ID:server-63c344e4-4e0d-987f-e671-609be6b63c40 Address:63c344e4-4e0d-987f-e671-609be6b63c40} {Suffrage:Voter ID:server-77035e64-5963-ae15-507b-66131657cc44 Address:77035e64-5963-ae15-507b-66131657cc44} {Suffrage:Voter ID:server-7da55b48-5800-9531-12b8-e97106fd93b0 Address:7da55b48-5800-9531-12b8-e97106fd93b0}]" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.570Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: updating configuration: command=AddStaging server-id=server-7da55b48-5800-9531-12b8-e97106fd93b0 server-addr=7da55b48-5800-9531-12b8-e97106fd93b0 servers="[{Suffrage:Voter ID:server-63c344e4-4e0d-987f-e671-609be6b63c40 Address:63c344e4-4e0d-987f-e671-609be6b63c40} {Suffrage:Voter ID:server-77035e64-5963-ae15-507b-66131657cc44 Address:77035e64-5963-ae15-507b-66131657cc44} {Suffrage:Voter ID:server-7da55b48-5800-9531-12b8-e97106fd93b0 Address:7da55b48-5800-9531-12b8-e97106fd93b0}]" testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.570Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: added peer, starting replication: peer=server-7da55b48-5800-9531-12b8-e97106fd93b0 testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.570Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: added peer, starting replication: peer=server-7da55b48-5800-9531-12b8-e97106fd93b0 testLoggerAdapter verbose: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.570Z [WARN] server-7da55b48-5800-9531-12b8-e97106fd93b0: failed to get previous log: previous-index=4 last-index=0 error="log not found" testing.go:149: server-7da55b48-5800-9531-12b8-e97106fd93b0: 2021-08-25T09:31:03.570Z [WARN] server-7da55b48-5800-9531-12b8-e97106fd93b0: failed to get previous log: previous-index=4 last-index=0 error="log not found" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.570Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.570Z [WARN] server-63c344e4-4e0d-987f-e671-609be6b63c40: appendEntries rejected, sending older logs: peer="{Voter server-7da55b48-5800-9531-12b8-e97106fd93b0 7da55b48-5800-9531-12b8-e97106fd93b0}" next=1 testing.go:149: cluster: 2021-08-25T09:31:03.570Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.570Z [WARN] server-63c344e4-4e0d-987f-e671-609be6b63c40: appendEntries rejected, sending older logs: peer="{Voter server-7da55b48-5800-9531-12b8-e97106fd93b0 7da55b48-5800-9531-12b8-e97106fd93b0}" next=1 testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.571Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: pipelining replication: peer="{Voter server-7da55b48-5800-9531-12b8-e97106fd93b0 7da55b48-5800-9531-12b8-e97106fd93b0}" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.571Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: pipelining replication: peer="{Voter server-7da55b48-5800-9531-12b8-e97106fd93b0 7da55b48-5800-9531-12b8-e97106fd93b0}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.676Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.570873908 +0000 UTC m=+7.837090312 (1 nodes), 2.02µs from start of poll, 306.408561ms from cluster start. Timeout at 2021-08-25 09:31:03.676022131 +0000 UTC m=+7.942238575, 105.148263ms after stability testing.go:149: cluster: 2021-08-25T09:31:03.676Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.570873908 +0000 UTC m=+7.837090312 (1 nodes), 2.02µs from start of poll, 306.408561ms from cluster start. Timeout at 2021-08-25 09:31:03.676022131 +0000 UTC m=+7.942238575, 105.148263ms after stability testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.676Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: aborting pipeline replication: peer="{Voter server-7da55b48-5800-9531-12b8-e97106fd93b0 7da55b48-5800-9531-12b8-e97106fd93b0}" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.676Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: aborting pipeline replication: peer="{Voter server-7da55b48-5800-9531-12b8-e97106fd93b0 7da55b48-5800-9531-12b8-e97106fd93b0}" testLoggerAdapter verbose: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.676Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: aborting pipeline replication: peer="{Voter server-77035e64-5963-ae15-507b-66131657cc44 77035e64-5963-ae15-507b-66131657cc44}" testing.go:149: server-63c344e4-4e0d-987f-e671-609be6b63c40: 2021-08-25T09:31:03.676Z [INFO] server-63c344e4-4e0d-987f-e671-609be6b63c40: aborting pipeline replication: peer="{Voter server-77035e64-5963-ae15-507b-66131657cc44 77035e64-5963-ae15-507b-66131657cc44}" --- PASS: TestRaft_JoinNode_ConfigStore (0.41s) === RUN TestRaft_RemoveFollower testLoggerAdapter verbose: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:03.680Z [INFO] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 Address:1b839e59-bc0e-bf81-3aa2-7d9fe09243e8} {Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testing.go:149: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:03.680Z [INFO] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 Address:1b839e59-bc0e-bf81-3aa2-7d9fe09243e8} {Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testLoggerAdapter verbose: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:03.680Z [INFO] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: entering follower state: follower="Node at 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 [Follower]" leader= testing.go:149: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:03.680Z [INFO] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: entering follower state: follower="Node at 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 [Follower]" leader= testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.680Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 Address:1b839e59-bc0e-bf81-3aa2-7d9fe09243e8} {Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.680Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 Address:1b839e59-bc0e-bf81-3aa2-7d9fe09243e8} {Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.680Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: entering follower state: follower="Node at f874226f-fe5a-d360-de96-f971dbb7204f [Follower]" leader= testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.680Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: entering follower state: follower="Node at f874226f-fe5a-d360-de96-f971dbb7204f [Follower]" leader= testLoggerAdapter verbose: server-23d1332c-166c-a241-6eec-467a9e6a71df: 2021-08-25T09:31:03.680Z [INFO] server-23d1332c-166c-a241-6eec-467a9e6a71df: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 Address:1b839e59-bc0e-bf81-3aa2-7d9fe09243e8} {Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testing.go:149: server-23d1332c-166c-a241-6eec-467a9e6a71df: 2021-08-25T09:31:03.680Z [INFO] server-23d1332c-166c-a241-6eec-467a9e6a71df: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 Address:1b839e59-bc0e-bf81-3aa2-7d9fe09243e8} {Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.681Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:03.681Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-23d1332c-166c-a241-6eec-467a9e6a71df: 2021-08-25T09:31:03.681Z [INFO] server-23d1332c-166c-a241-6eec-467a9e6a71df: entering follower state: follower="Node at 23d1332c-166c-a241-6eec-467a9e6a71df [Follower]" leader= testing.go:149: server-23d1332c-166c-a241-6eec-467a9e6a71df: 2021-08-25T09:31:03.681Z [INFO] server-23d1332c-166c-a241-6eec-467a9e6a71df: entering follower state: follower="Node at 23d1332c-166c-a241-6eec-467a9e6a71df [Follower]" leader= testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [WARN] server-f874226f-fe5a-d360-de96-f971dbb7204f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [WARN] server-f874226f-fe5a-d360-de96-f971dbb7204f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: entering candidate state: node="Node at f874226f-fe5a-d360-de96-f971dbb7204f [Candidate]" term=2 testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: entering candidate state: node="Node at f874226f-fe5a-d360-de96-f971dbb7204f [Candidate]" term=2 testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: election won: tally=2 testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: election won: tally=2 testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: entering leader state: leader="Node at f874226f-fe5a-d360-de96-f971dbb7204f [Leader]" testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: entering leader state: leader="Node at f874226f-fe5a-d360-de96-f971dbb7204f [Leader]" testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: added peer, starting replication: peer=server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: added peer, starting replication: peer=server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: added peer, starting replication: peer=server-23d1332c-166c-a241-6eec-467a9e6a71df testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.747Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: added peer, starting replication: peer=server-23d1332c-166c-a241-6eec-467a9e6a71df testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.748Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: pipelining replication: peer="{Voter server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8}" testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.748Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: pipelining replication: peer="{Voter server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8}" testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.748Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: pipelining replication: peer="{Voter server-23d1332c-166c-a241-6eec-467a9e6a71df 23d1332c-166c-a241-6eec-467a9e6a71df}" testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.748Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: pipelining replication: peer="{Voter server-23d1332c-166c-a241-6eec-467a9e6a71df 23d1332c-166c-a241-6eec-467a9e6a71df}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.852Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.747844432 +0000 UTC m=+8.014060836 (1 nodes), 66.738039ms from start of poll, 67.835496ms from cluster start. Timeout at 2021-08-25 09:31:03.852871027 +0000 UTC m=+8.119087471, 105.026635ms after stability testing.go:149: cluster: 2021-08-25T09:31:03.852Z [INFO] stable state for Leader reached at 2021-08-25 09:31:03.747844432 +0000 UTC m=+8.014060836 (1 nodes), 66.738039ms from start of poll, 67.835496ms from cluster start. Timeout at 2021-08-25 09:31:03.852871027 +0000 UTC m=+8.119087471, 105.026635ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.858Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:03.858Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:03.963Z [INFO] stable state for Follower reached at 2021-08-25 09:31:03.858427804 +0000 UTC m=+8.124644188 (2 nodes), 2.7µs from start of poll, 178.418848ms from cluster start. Timeout at 2021-08-25 09:31:03.963558529 +0000 UTC m=+8.229775033, 105.130845ms after stability testing.go:149: cluster: 2021-08-25T09:31:03.963Z [INFO] stable state for Follower reached at 2021-08-25 09:31:03.858427804 +0000 UTC m=+8.124644188 (2 nodes), 2.7µs from start of poll, 178.418848ms from cluster start. Timeout at 2021-08-25 09:31:03.963558529 +0000 UTC m=+8.229775033, 105.130845ms after stability testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.963Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: updating configuration: command=RemoveServer server-id=server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 server-addr= servers="[{Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.963Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: updating configuration: command=RemoveServer server-id=server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 server-addr= servers="[{Suffrage:Voter ID:server-f874226f-fe5a-d360-de96-f971dbb7204f Address:f874226f-fe5a-d360-de96-f971dbb7204f} {Suffrage:Voter ID:server-23d1332c-166c-a241-6eec-467a9e6a71df Address:23d1332c-166c-a241-6eec-467a9e6a71df}]" testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.964Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: removed peer, stopping replication: peer=server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 last-index=3 testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.964Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: removed peer, stopping replication: peer=server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 last-index=3 testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.964Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: aborting pipeline replication: peer="{Voter server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8}" testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:03.964Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: aborting pipeline replication: peer="{Voter server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8}" testLoggerAdapter verbose: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:04.054Z [WARN] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: heartbeat timeout reached, starting election: last-leader=f874226f-fe5a-d360-de96-f971dbb7204f testing.go:149: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:04.054Z [WARN] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: heartbeat timeout reached, starting election: last-leader=f874226f-fe5a-d360-de96-f971dbb7204f testLoggerAdapter verbose: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:04.054Z [INFO] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: entering candidate state: node="Node at 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 [Candidate]" term=3 testing.go:149: server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: 2021-08-25T09:31:04.054Z [INFO] server-1b839e59-bc0e-bf81-3aa2-7d9fe09243e8: entering candidate state: node="Node at 1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 [Candidate]" term=3 testLoggerAdapter verbose: server-23d1332c-166c-a241-6eec-467a9e6a71df: 2021-08-25T09:31:04.054Z [WARN] server-23d1332c-166c-a241-6eec-467a9e6a71df: rejecting vote request since we have a leader: from=1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 leader=f874226f-fe5a-d360-de96-f971dbb7204f testing.go:149: server-23d1332c-166c-a241-6eec-467a9e6a71df: 2021-08-25T09:31:04.054Z [WARN] server-23d1332c-166c-a241-6eec-467a9e6a71df: rejecting vote request since we have a leader: from=1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 leader=f874226f-fe5a-d360-de96-f971dbb7204f testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:04.054Z [WARN] server-f874226f-fe5a-d360-de96-f971dbb7204f: rejecting vote request since we have a leader: from=1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 leader=f874226f-fe5a-d360-de96-f971dbb7204f testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:04.054Z [WARN] server-f874226f-fe5a-d360-de96-f971dbb7204f: rejecting vote request since we have a leader: from=1b839e59-bc0e-bf81-3aa2-7d9fe09243e8 leader=f874226f-fe5a-d360-de96-f971dbb7204f testLoggerAdapter verbose: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:04.069Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: aborting pipeline replication: peer="{Voter server-23d1332c-166c-a241-6eec-467a9e6a71df 23d1332c-166c-a241-6eec-467a9e6a71df}" testing.go:149: server-f874226f-fe5a-d360-de96-f971dbb7204f: 2021-08-25T09:31:04.069Z [INFO] server-f874226f-fe5a-d360-de96-f971dbb7204f: aborting pipeline replication: peer="{Voter server-23d1332c-166c-a241-6eec-467a9e6a71df 23d1332c-166c-a241-6eec-467a9e6a71df}" --- PASS: TestRaft_RemoveFollower (0.39s) === RUN TestRaft_RemoveLeader testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.073Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb} {Suffrage:Voter ID:server-ecfd4bd9-12df-2443-487c-4ed94def9de8 Address:ecfd4bd9-12df-2443-487c-4ed94def9de8}]" testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.073Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb} {Suffrage:Voter ID:server-ecfd4bd9-12df-2443-487c-4ed94def9de8 Address:ecfd4bd9-12df-2443-487c-4ed94def9de8}]" testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.073Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering follower state: follower="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Follower]" leader= testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.073Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering follower state: follower="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Follower]" leader= testLoggerAdapter verbose: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.074Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb} {Suffrage:Voter ID:server-ecfd4bd9-12df-2443-487c-4ed94def9de8 Address:ecfd4bd9-12df-2443-487c-4ed94def9de8}]" testing.go:149: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.074Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb} {Suffrage:Voter ID:server-ecfd4bd9-12df-2443-487c-4ed94def9de8 Address:ecfd4bd9-12df-2443-487c-4ed94def9de8}]" testLoggerAdapter verbose: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.074Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: entering follower state: follower="Node at 54ca903a-4ab2-420b-6088-a1afdededecb [Follower]" leader= testing.go:149: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.074Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: entering follower state: follower="Node at 54ca903a-4ab2-420b-6088-a1afdededecb [Follower]" leader= testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.074Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb} {Suffrage:Voter ID:server-ecfd4bd9-12df-2443-487c-4ed94def9de8 Address:ecfd4bd9-12df-2443-487c-4ed94def9de8}]" testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.074Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb} {Suffrage:Voter ID:server-ecfd4bd9-12df-2443-487c-4ed94def9de8 Address:ecfd4bd9-12df-2443-487c-4ed94def9de8}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.074Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:04.074Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.074Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: entering follower state: follower="Node at ecfd4bd9-12df-2443-487c-4ed94def9de8 [Follower]" leader= testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.074Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: entering follower state: follower="Node at ecfd4bd9-12df-2443-487c-4ed94def9de8 [Follower]" leader= testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.140Z [WARN] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.140Z [WARN] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.140Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: entering candidate state: node="Node at ecfd4bd9-12df-2443-487c-4ed94def9de8 [Candidate]" term=2 testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.140Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: entering candidate state: node="Node at ecfd4bd9-12df-2443-487c-4ed94def9de8 [Candidate]" term=2 testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.140Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: election won: tally=2 testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.140Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: election won: tally=2 testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: entering leader state: leader="Node at ecfd4bd9-12df-2443-487c-4ed94def9de8 [Leader]" testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: entering leader state: leader="Node at ecfd4bd9-12df-2443-487c-4ed94def9de8 [Leader]" testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: added peer, starting replication: peer=server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: added peer, starting replication: peer=server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: added peer, starting replication: peer=server-54ca903a-4ab2-420b-6088-a1afdededecb testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: added peer, starting replication: peer=server-54ca903a-4ab2-420b-6088-a1afdededecb testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: pipelining replication: peer="{Voter server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232}" testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: pipelining replication: peer="{Voter server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232}" testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: pipelining replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.141Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: pipelining replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.246Z [INFO] stable state for Leader reached at 2021-08-25 09:31:04.141148774 +0000 UTC m=+8.407365178 (1 nodes), 66.51212ms from start of poll, 67.758942ms from cluster start. Timeout at 2021-08-25 09:31:04.24627084 +0000 UTC m=+8.512487264, 105.122086ms after stability testing.go:149: cluster: 2021-08-25T09:31:04.246Z [INFO] stable state for Leader reached at 2021-08-25 09:31:04.141148774 +0000 UTC m=+8.407365178 (1 nodes), 66.51212ms from start of poll, 67.758942ms from cluster start. Timeout at 2021-08-25 09:31:04.24627084 +0000 UTC m=+8.512487264, 105.122086ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.251Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:04.251Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.356Z [INFO] stable state for Follower reached at 2021-08-25 09:31:04.251818038 +0000 UTC m=+8.518034422 (2 nodes), 2.9µs from start of poll, 178.428186ms from cluster start. Timeout at 2021-08-25 09:31:04.356820775 +0000 UTC m=+8.623037179, 105.002757ms after stability testing.go:149: cluster: 2021-08-25T09:31:04.356Z [INFO] stable state for Follower reached at 2021-08-25 09:31:04.251818038 +0000 UTC m=+8.518034422 (2 nodes), 2.9µs from start of poll, 178.428186ms from cluster start. Timeout at 2021-08-25 09:31:04.356820775 +0000 UTC m=+8.623037179, 105.002757ms after stability testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: updating configuration: command=RemoveServer server-id=server-ecfd4bd9-12df-2443-487c-4ed94def9de8 server-addr= servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb}]" testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: updating configuration: command=RemoveServer server-id=server-ecfd4bd9-12df-2443-487c-4ed94def9de8 server-addr= servers="[{Suffrage:Voter ID:server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 Address:6ed4721a-7e76-0bd4-50d1-bcb0de4b2232} {Suffrage:Voter ID:server-54ca903a-4ab2-420b-6088-a1afdededecb Address:54ca903a-4ab2-420b-6088-a1afdededecb}]" testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: removed ourself, shutting down testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: removed ourself, shutting down testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: aborting pipeline replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: aborting pipeline replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" testLoggerAdapter verbose: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: aborting pipeline replication: peer="{Voter server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232}" testing.go:149: server-ecfd4bd9-12df-2443-487c-4ed94def9de8: 2021-08-25T09:31:04.357Z [INFO] server-ecfd4bd9-12df-2443-487c-4ed94def9de8: aborting pipeline replication: peer="{Voter server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232}" testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.420Z [WARN] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: heartbeat timeout reached, starting election: last-leader=ecfd4bd9-12df-2443-487c-4ed94def9de8 testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.420Z [WARN] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: heartbeat timeout reached, starting election: last-leader=ecfd4bd9-12df-2443-487c-4ed94def9de8 testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.420Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering candidate state: node="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Candidate]" term=3 testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.420Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering candidate state: node="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Candidate]" term=3 testLoggerAdapter verbose: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.420Z [WARN] server-54ca903a-4ab2-420b-6088-a1afdededecb: rejecting vote request since we have a leader: from=6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 leader=ecfd4bd9-12df-2443-487c-4ed94def9de8 testing.go:149: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.420Z [WARN] server-54ca903a-4ab2-420b-6088-a1afdededecb: rejecting vote request since we have a leader: from=6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 leader=ecfd4bd9-12df-2443-487c-4ed94def9de8 testLoggerAdapter verbose: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.460Z [WARN] server-54ca903a-4ab2-420b-6088-a1afdededecb: heartbeat timeout reached, starting election: last-leader=ecfd4bd9-12df-2443-487c-4ed94def9de8 testing.go:149: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.460Z [WARN] server-54ca903a-4ab2-420b-6088-a1afdededecb: heartbeat timeout reached, starting election: last-leader=ecfd4bd9-12df-2443-487c-4ed94def9de8 testLoggerAdapter verbose: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.460Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: entering candidate state: node="Node at 54ca903a-4ab2-420b-6088-a1afdededecb [Candidate]" term=3 testing.go:149: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.460Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: entering candidate state: node="Node at 54ca903a-4ab2-420b-6088-a1afdededecb [Candidate]" term=3 testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.460Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: duplicate requestVote for same term: term=3 testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.460Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [WARN] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: Election timeout reached, restarting election testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [WARN] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: Election timeout reached, restarting election testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering candidate state: node="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Candidate]" term=4 testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering candidate state: node="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Candidate]" term=4 testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: election won: tally=2 testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: election won: tally=2 testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering leader state: leader="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Leader]" testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: entering leader state: leader="Node at 6ed4721a-7e76-0bd4-50d1-bcb0de4b2232 [Leader]" testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: added peer, starting replication: peer=server-54ca903a-4ab2-420b-6088-a1afdededecb testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: added peer, starting replication: peer=server-54ca903a-4ab2-420b-6088-a1afdededecb testLoggerAdapter verbose: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.479Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: entering follower state: follower="Node at 54ca903a-4ab2-420b-6088-a1afdededecb [Follower]" leader= testing.go:149: server-54ca903a-4ab2-420b-6088-a1afdededecb: 2021-08-25T09:31:04.479Z [INFO] server-54ca903a-4ab2-420b-6088-a1afdededecb: entering follower state: follower="Node at 54ca903a-4ab2-420b-6088-a1afdededecb [Follower]" leader= testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: pipelining replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.479Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: pipelining replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.567Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:04.567Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.673Z [INFO] stable state for Leader reached at 2021-08-25 09:31:04.568237557 +0000 UTC m=+8.834453961 (1 nodes), 100.59µs from start of poll, 494.847725ms from cluster start. Timeout at 2021-08-25 09:31:04.67328701 +0000 UTC m=+8.939503494, 105.049533ms after stability testing.go:149: cluster: 2021-08-25T09:31:04.673Z [INFO] stable state for Leader reached at 2021-08-25 09:31:04.568237557 +0000 UTC m=+8.834453961 (1 nodes), 100.59µs from start of poll, 494.847725ms from cluster start. Timeout at 2021-08-25 09:31:04.67328701 +0000 UTC m=+8.939503494, 105.049533ms after stability testLoggerAdapter verbose: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.673Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: aborting pipeline replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" testing.go:149: server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: 2021-08-25T09:31:04.673Z [INFO] server-6ed4721a-7e76-0bd4-50d1-bcb0de4b2232: aborting pipeline replication: peer="{Voter server-54ca903a-4ab2-420b-6088-a1afdededecb 54ca903a-4ab2-420b-6088-a1afdededecb}" --- PASS: TestRaft_RemoveLeader (0.60s) === RUN TestRaft_RemoveLeader_NoShutdown testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.677Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 Address:4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9} {Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.677Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 Address:4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9} {Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.677Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering follower state: follower="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Follower]" leader= testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.677Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering follower state: follower="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Follower]" leader= testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:04.677Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 Address:4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9} {Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:04.677Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 Address:4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9} {Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:04.678Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering follower state: follower="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Follower]" leader= testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:04.678Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering follower state: follower="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Follower]" leader= testLoggerAdapter verbose: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:04.678Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 Address:4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9} {Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testing.go:149: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:04.678Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 Address:4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9} {Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.678Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:04.678Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:04.678Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: entering follower state: follower="Node at e56fef11-7326-8fbd-73a1-b576a906494c [Follower]" leader= testing.go:149: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:04.678Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: entering follower state: follower="Node at e56fef11-7326-8fbd-73a1-b576a906494c [Follower]" leader= testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.736Z [WARN] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.736Z [WARN] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.736Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering candidate state: node="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Candidate]" term=2 testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.736Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering candidate state: node="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Candidate]" term=2 testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.736Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: election won: tally=2 testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.736Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: election won: tally=2 testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering leader state: leader="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Leader]" testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering leader state: leader="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Leader]" testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: added peer, starting replication: peer=server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: added peer, starting replication: peer=server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: added peer, starting replication: peer=server-e56fef11-7326-8fbd-73a1-b576a906494c testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: added peer, starting replication: peer=server-e56fef11-7326-8fbd-73a1-b576a906494c testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: pipelining replication: peer="{Voter server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 fe6ed2ae-4788-f9f5-8990-73aa1ad45c74}" testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: pipelining replication: peer="{Voter server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 fe6ed2ae-4788-f9f5-8990-73aa1ad45c74}" testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: pipelining replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.737Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: pipelining replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.842Z [INFO] stable state for Follower reached at 2021-08-25 09:31:04.737044469 +0000 UTC m=+9.003260873 (2 nodes), 58.686876ms from start of poll, 59.767195ms from cluster start. Timeout at 2021-08-25 09:31:04.842186753 +0000 UTC m=+9.108403157, 105.142284ms after stability testing.go:149: cluster: 2021-08-25T09:31:04.842Z [INFO] stable state for Follower reached at 2021-08-25 09:31:04.737044469 +0000 UTC m=+9.003260873 (2 nodes), 58.686876ms from start of poll, 59.767195ms from cluster start. Timeout at 2021-08-25 09:31:04.842186753 +0000 UTC m=+9.108403157, 105.142284ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.842Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:04.842Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:04.947Z [INFO] stable state for Leader reached at 2021-08-25 09:31:04.842468167 +0000 UTC m=+9.108684571 (1 nodes), 2.6µs from start of poll, 165.190893ms from cluster start. Timeout at 2021-08-25 09:31:04.947577194 +0000 UTC m=+9.213793598, 105.109027ms after stability testing.go:149: cluster: 2021-08-25T09:31:04.947Z [INFO] stable state for Leader reached at 2021-08-25 09:31:04.842468167 +0000 UTC m=+9.108684571 (1 nodes), 2.6µs from start of poll, 165.190893ms from cluster start. Timeout at 2021-08-25 09:31:04.947577194 +0000 UTC m=+9.213793598, 105.109027ms after stability testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: updating configuration: command=RemoveServer server-id=server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 server-addr= servers="[{Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: updating configuration: command=RemoveServer server-id=server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 server-addr= servers="[{Suffrage:Voter ID:server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 Address:fe6ed2ae-4788-f9f5-8990-73aa1ad45c74} {Suffrage:Voter ID:server-e56fef11-7326-8fbd-73a1-b576a906494c Address:e56fef11-7326-8fbd-73a1-b576a906494c}]" testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: removed ourself, transitioning to follower testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: removed ourself, transitioning to follower testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering follower state: follower="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Follower]" leader= testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: entering follower state: follower="Node at 4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 [Follower]" leader= testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: aborting pipeline replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: aborting pipeline replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: aborting pipeline replication: peer="{Voter server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 fe6ed2ae-4788-f9f5-8990-73aa1ad45c74}" testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:04.950Z [INFO] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: aborting pipeline replication: peer="{Voter server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 fe6ed2ae-4788-f9f5-8990-73aa1ad45c74}" testLoggerAdapter verbose: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.019Z [WARN] server-e56fef11-7326-8fbd-73a1-b576a906494c: heartbeat timeout reached, starting election: last-leader=4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 testing.go:149: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.019Z [WARN] server-e56fef11-7326-8fbd-73a1-b576a906494c: heartbeat timeout reached, starting election: last-leader=4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 testLoggerAdapter verbose: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.019Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: entering candidate state: node="Node at e56fef11-7326-8fbd-73a1-b576a906494c [Candidate]" term=3 testing.go:149: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.019Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: entering candidate state: node="Node at e56fef11-7326-8fbd-73a1-b576a906494c [Candidate]" term=3 testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.020Z [WARN] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: rejecting vote request since we have a leader: from=e56fef11-7326-8fbd-73a1-b576a906494c leader=4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.020Z [WARN] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: rejecting vote request since we have a leader: from=e56fef11-7326-8fbd-73a1-b576a906494c leader=4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 testLoggerAdapter verbose: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:05.030Z [WARN] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: not part of stable configuration, aborting election testing.go:149: server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: 2021-08-25T09:31:05.030Z [WARN] server-4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9: not part of stable configuration, aborting election testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.037Z [WARN] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: heartbeat timeout reached, starting election: last-leader=4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.037Z [WARN] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: heartbeat timeout reached, starting election: last-leader=4abb3925-2bc0-5dd0-2cb6-eb7a96035fc9 testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.037Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering candidate state: node="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Candidate]" term=3 testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.037Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering candidate state: node="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Candidate]" term=3 testLoggerAdapter verbose: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.037Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: duplicate requestVote for same term: term=3 testing.go:149: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.037Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.056Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:05.056Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [WARN] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: Election timeout reached, restarting election testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [WARN] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: Election timeout reached, restarting election testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering candidate state: node="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Candidate]" term=4 testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering candidate state: node="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Candidate]" term=4 testLoggerAdapter verbose: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.091Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: entering follower state: follower="Node at e56fef11-7326-8fbd-73a1-b576a906494c [Follower]" leader= testing.go:149: server-e56fef11-7326-8fbd-73a1-b576a906494c: 2021-08-25T09:31:05.091Z [INFO] server-e56fef11-7326-8fbd-73a1-b576a906494c: entering follower state: follower="Node at e56fef11-7326-8fbd-73a1-b576a906494c [Follower]" leader= testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: election won: tally=2 testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: election won: tally=2 testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering leader state: leader="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Leader]" testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: entering leader state: leader="Node at fe6ed2ae-4788-f9f5-8990-73aa1ad45c74 [Leader]" testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: added peer, starting replication: peer=server-e56fef11-7326-8fbd-73a1-b576a906494c testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.091Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: added peer, starting replication: peer=server-e56fef11-7326-8fbd-73a1-b576a906494c testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.092Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: pipelining replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.092Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: pipelining replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.197Z [INFO] stable state for Leader reached at 2021-08-25 09:31:05.092163926 +0000 UTC m=+9.358380390 (1 nodes), 35.764934ms from start of poll, 414.886712ms from cluster start. Timeout at 2021-08-25 09:31:05.197275413 +0000 UTC m=+9.463491817, 105.111427ms after stability testing.go:149: cluster: 2021-08-25T09:31:05.197Z [INFO] stable state for Leader reached at 2021-08-25 09:31:05.092163926 +0000 UTC m=+9.358380390 (1 nodes), 35.764934ms from start of poll, 414.886712ms from cluster start. Timeout at 2021-08-25 09:31:05.197275413 +0000 UTC m=+9.463491817, 105.111427ms after stability testLoggerAdapter verbose: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.302Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: aborting pipeline replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" testing.go:149: server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: 2021-08-25T09:31:05.302Z [INFO] server-fe6ed2ae-4788-f9f5-8990-73aa1ad45c74: aborting pipeline replication: peer="{Voter server-e56fef11-7326-8fbd-73a1-b576a906494c e56fef11-7326-8fbd-73a1-b576a906494c}" --- PASS: TestRaft_RemoveLeader_NoShutdown (0.63s) === RUN TestRaft_RemoveFollower_SplitCluster testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.308Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.308Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.308Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering follower state: follower="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Follower]" leader= testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.308Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering follower state: follower="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Follower]" leader= testLoggerAdapter verbose: server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: 2021-08-25T09:31:05.308Z [INFO] server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testing.go:149: server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: 2021-08-25T09:31:05.308Z [INFO] server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testLoggerAdapter verbose: server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: 2021-08-25T09:31:05.308Z [INFO] server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: entering follower state: follower="Node at c778aa7e-ec4c-add6-15f9-47dc1da58f15 [Follower]" leader= testing.go:149: server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: 2021-08-25T09:31:05.308Z [INFO] server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: entering follower state: follower="Node at c778aa7e-ec4c-add6-15f9-47dc1da58f15 [Follower]" leader= testLoggerAdapter verbose: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.309Z [INFO] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testing.go:149: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.309Z [INFO] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testLoggerAdapter verbose: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.309Z [INFO] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: entering follower state: follower="Node at 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d [Follower]" leader= testing.go:149: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.309Z [INFO] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: entering follower state: follower="Node at 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d [Follower]" leader= testLoggerAdapter verbose: server-aaaaae65-4033-e76b-a58c-653a480cbac8: 2021-08-25T09:31:05.309Z [INFO] server-aaaaae65-4033-e76b-a58c-653a480cbac8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testing.go:149: server-aaaaae65-4033-e76b-a58c-653a480cbac8: 2021-08-25T09:31:05.309Z [INFO] server-aaaaae65-4033-e76b-a58c-653a480cbac8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d} {Suffrage:Voter ID:server-aaaaae65-4033-e76b-a58c-653a480cbac8 Address:aaaaae65-4033-e76b-a58c-653a480cbac8}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.309Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:05.309Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-aaaaae65-4033-e76b-a58c-653a480cbac8: 2021-08-25T09:31:05.309Z [INFO] server-aaaaae65-4033-e76b-a58c-653a480cbac8: entering follower state: follower="Node at aaaaae65-4033-e76b-a58c-653a480cbac8 [Follower]" leader= testing.go:149: server-aaaaae65-4033-e76b-a58c-653a480cbac8: 2021-08-25T09:31:05.309Z [INFO] server-aaaaae65-4033-e76b-a58c-653a480cbac8: entering follower state: follower="Node at aaaaae65-4033-e76b-a58c-653a480cbac8 [Follower]" leader= testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.360Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.360Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering candidate state: node="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Candidate]" term=2 testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering candidate state: node="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Candidate]" term=2 testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: election won: tally=3 testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: election won: tally=3 testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering leader state: leader="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Leader]" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering leader state: leader="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Leader]" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: added peer, starting replication: peer=server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: added peer, starting replication: peer=server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: added peer, starting replication: peer=server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: added peer, starting replication: peer=server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: added peer, starting replication: peer=server-aaaaae65-4033-e76b-a58c-653a480cbac8 testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: added peer, starting replication: peer=server-aaaaae65-4033-e76b-a58c-653a480cbac8 testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: pipelining replication: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: pipelining replication: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: pipelining replication: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.361Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: pipelining replication: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.362Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: pipelining replication: peer="{Voter server-aaaaae65-4033-e76b-a58c-653a480cbac8 aaaaae65-4033-e76b-a58c-653a480cbac8}" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.362Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: pipelining replication: peer="{Voter server-aaaaae65-4033-e76b-a58c-653a480cbac8 aaaaae65-4033-e76b-a58c-653a480cbac8}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.466Z [INFO] stable state for Leader reached at 2021-08-25 09:31:05.361711237 +0000 UTC m=+9.627927681 (1 nodes), 51.996067ms from start of poll, 53.76548ms from cluster start. Timeout at 2021-08-25 09:31:05.466881378 +0000 UTC m=+9.733097842, 105.170161ms after stability testing.go:149: cluster: 2021-08-25T09:31:05.466Z [INFO] stable state for Leader reached at 2021-08-25 09:31:05.361711237 +0000 UTC m=+9.627927681 (1 nodes), 51.996067ms from start of poll, 53.76548ms from cluster start. Timeout at 2021-08-25 09:31:05.466881378 +0000 UTC m=+9.733097842, 105.170161ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.572Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:05.572Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.677Z [INFO] stable state for Follower reached at 2021-08-25 09:31:05.572568931 +0000 UTC m=+9.838785315 (3 nodes), 3.48µs from start of poll, 264.623114ms from cluster start. Timeout at 2021-08-25 09:31:05.677706476 +0000 UTC m=+9.943922960, 105.137645ms after stability testing.go:149: cluster: 2021-08-25T09:31:05.677Z [INFO] stable state for Follower reached at 2021-08-25 09:31:05.572568931 +0000 UTC m=+9.838785315 (3 nodes), 3.48µs from start of poll, 264.623114ms from cluster start. Timeout at 2021-08-25 09:31:05.677706476 +0000 UTC m=+9.943922960, 105.137645ms after stability testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: updating configuration: command=RemoveServer server-id=server-aaaaae65-4033-e76b-a58c-653a480cbac8 server-addr= servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}]" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: updating configuration: command=RemoveServer server-id=server-aaaaae65-4033-e76b-a58c-653a480cbac8 server-addr= servers="[{Suffrage:Voter ID:server-207f0be7-9230-744d-939c-5efa7e5a0a8e Address:207f0be7-9230-744d-939c-5efa7e5a0a8e} {Suffrage:Voter ID:server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 Address:c778aa7e-ec4c-add6-15f9-47dc1da58f15} {Suffrage:Voter ID:server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d Address:17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}]" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: removed peer, stopping replication: peer=server-aaaaae65-4033-e76b-a58c-653a480cbac8 last-index=3 testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: removed peer, stopping replication: peer=server-aaaaae65-4033-e76b-a58c-653a480cbac8 last-index=3 testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: aborting pipeline replication: peer="{Voter server-aaaaae65-4033-e76b-a58c-653a480cbac8 aaaaae65-4033-e76b-a58c-653a480cbac8}" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: aborting pipeline replication: peer="{Voter server-aaaaae65-4033-e76b-a58c-653a480cbac8 aaaaae65-4033-e76b-a58c-653a480cbac8}" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to pipeline appendEntries: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="append pipeline closed" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to pipeline appendEntries: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="append pipeline closed" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: aborting pipeline replication: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: aborting pipeline replication: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to pipeline appendEntries: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="append pipeline closed" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to pipeline appendEntries: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="append pipeline closed" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: aborting pipeline replication: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: aborting pipeline replication: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=c778aa7e-ec4c-add6-15f9-47dc1da58f15 error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.678Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=c778aa7e-ec4c-add6-15f9-47dc1da58f15 error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.682Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.682Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.684Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.684Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.686Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.686Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.696Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=c778aa7e-ec4c-add6-15f9-47dc1da58f15 error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.696Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=c778aa7e-ec4c-add6-15f9-47dc1da58f15 error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.699Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.699Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.702Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.702Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.703Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.703Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.715Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=c778aa7e-ec4c-add6-15f9-47dc1da58f15 error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.715Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=c778aa7e-ec4c-add6-15f9-47dc1da58f15 error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.718Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.718Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to heartbeat to: peer=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.719Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.719Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.719Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.719Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.723Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact: server-id=server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 time=50.109284ms testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.723Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact: server-id=server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 time=50.109284ms testLoggerAdapter verbose: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [WARN] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: heartbeat timeout reached, starting election: last-leader=207f0be7-9230-744d-939c-5efa7e5a0a8e testing.go:149: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [WARN] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: heartbeat timeout reached, starting election: last-leader=207f0be7-9230-744d-939c-5efa7e5a0a8e testLoggerAdapter verbose: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [INFO] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: entering candidate state: node="Node at 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d [Candidate]" term=3 testing.go:149: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [INFO] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: entering candidate state: node="Node at 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d [Candidate]" term=3 testLoggerAdapter verbose: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [ERROR] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: failed to make requestVote RPC: target="{Voter server-aaaaae65-4033-e76b-a58c-653a480cbac8 aaaaae65-4033-e76b-a58c-653a480cbac8}" error="failed to connect to peer: aaaaae65-4033-e76b-a58c-653a480cbac8" testing.go:149: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [ERROR] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: failed to make requestVote RPC: target="{Voter server-aaaaae65-4033-e76b-a58c-653a480cbac8 aaaaae65-4033-e76b-a58c-653a480cbac8}" error="failed to connect to peer: aaaaae65-4033-e76b-a58c-653a480cbac8" testLoggerAdapter verbose: server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: 2021-08-25T09:31:05.732Z [WARN] server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: rejecting vote request since we have a leader: from=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d leader=207f0be7-9230-744d-939c-5efa7e5a0a8e testing.go:149: server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: 2021-08-25T09:31:05.732Z [WARN] server-c778aa7e-ec4c-add6-15f9-47dc1da58f15: rejecting vote request since we have a leader: from=17f9739f-bdf2-fc3a-d284-e52a21ee6f7d leader=207f0be7-9230-744d-939c-5efa7e5a0a8e testLoggerAdapter verbose: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [ERROR] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: failed to make requestVote RPC: target="{Voter server-207f0be7-9230-744d-939c-5efa7e5a0a8e 207f0be7-9230-744d-939c-5efa7e5a0a8e}" error="failed to connect to peer: 207f0be7-9230-744d-939c-5efa7e5a0a8e" testing.go:149: server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: 2021-08-25T09:31:05.732Z [ERROR] server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d: failed to make requestVote RPC: target="{Voter server-207f0be7-9230-744d-939c-5efa7e5a0a8e 207f0be7-9230-744d-939c-5efa7e5a0a8e}" error="failed to connect to peer: 207f0be7-9230-744d-939c-5efa7e5a0a8e" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact: server-id=server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 time=60.447371ms testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact: server-id=server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 time=60.447371ms testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact: server-id=server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d time=58.0063ms testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact: server-id=server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d time=58.0063ms testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact quorum of nodes, stepping down testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [WARN] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering follower state: follower="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Follower]" leader= testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [INFO] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: entering follower state: follower="Node at 207f0be7-9230-744d-939c-5efa7e5a0a8e [Follower]" leader= testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-17f9739f-bdf2-fc3a-d284-e52a21ee6f7d 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d}" error="failed to connect to peer: 17f9739f-bdf2-fc3a-d284-e52a21ee6f7d" testLoggerAdapter verbose: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" testing.go:149: server-207f0be7-9230-744d-939c-5efa7e5a0a8e: 2021-08-25T09:31:05.734Z [ERROR] server-207f0be7-9230-744d-939c-5efa7e5a0a8e: failed to appendEntries to: peer="{Voter server-c778aa7e-ec4c-add6-15f9-47dc1da58f15 c778aa7e-ec4c-add6-15f9-47dc1da58f15}" error="failed to connect to peer: c778aa7e-ec4c-add6-15f9-47dc1da58f15" --- PASS: TestRaft_RemoveFollower_SplitCluster (0.43s) === RUN TestRaft_AddKnownPeer testLoggerAdapter verbose: server-178aa703-72b6-fd56-e030-6e845a8d3e97: 2021-08-25T09:31:05.738Z [INFO] server-178aa703-72b6-fd56-e030-6e845a8d3e97: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testing.go:149: server-178aa703-72b6-fd56-e030-6e845a8d3e97: 2021-08-25T09:31:05.738Z [INFO] server-178aa703-72b6-fd56-e030-6e845a8d3e97: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testLoggerAdapter verbose: server-178aa703-72b6-fd56-e030-6e845a8d3e97: 2021-08-25T09:31:05.739Z [INFO] server-178aa703-72b6-fd56-e030-6e845a8d3e97: entering follower state: follower="Node at 178aa703-72b6-fd56-e030-6e845a8d3e97 [Follower]" leader= testing.go:149: server-178aa703-72b6-fd56-e030-6e845a8d3e97: 2021-08-25T09:31:05.739Z [INFO] server-178aa703-72b6-fd56-e030-6e845a8d3e97: entering follower state: follower="Node at 178aa703-72b6-fd56-e030-6e845a8d3e97 [Follower]" leader= testLoggerAdapter verbose: server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: 2021-08-25T09:31:05.739Z [INFO] server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testing.go:149: server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: 2021-08-25T09:31:05.739Z [INFO] server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testLoggerAdapter verbose: server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: 2021-08-25T09:31:05.739Z [INFO] server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: entering follower state: follower="Node at 931fe7dc-09f5-2a29-e0b1-3e9ac5532433 [Follower]" leader= testing.go:149: server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: 2021-08-25T09:31:05.739Z [INFO] server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433: entering follower state: follower="Node at 931fe7dc-09f5-2a29-e0b1-3e9ac5532433 [Follower]" leader= testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.739Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.739Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.739Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:05.739Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.739Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: entering follower state: follower="Node at 204c7154-9551-7527-7b60-f402666ad003 [Follower]" leader= testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.739Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: entering follower state: follower="Node at 204c7154-9551-7527-7b60-f402666ad003 [Follower]" leader= testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.794Z [WARN] server-204c7154-9551-7527-7b60-f402666ad003: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.794Z [WARN] server-204c7154-9551-7527-7b60-f402666ad003: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.794Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: entering candidate state: node="Node at 204c7154-9551-7527-7b60-f402666ad003 [Candidate]" term=2 testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.794Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: entering candidate state: node="Node at 204c7154-9551-7527-7b60-f402666ad003 [Candidate]" term=2 testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: election won: tally=2 testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: election won: tally=2 testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: entering leader state: leader="Node at 204c7154-9551-7527-7b60-f402666ad003 [Leader]" testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: entering leader state: leader="Node at 204c7154-9551-7527-7b60-f402666ad003 [Leader]" testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: added peer, starting replication: peer=server-178aa703-72b6-fd56-e030-6e845a8d3e97 testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: added peer, starting replication: peer=server-178aa703-72b6-fd56-e030-6e845a8d3e97 testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: added peer, starting replication: peer=server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: added peer, starting replication: peer=server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: pipelining replication: peer="{Voter server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 931fe7dc-09f5-2a29-e0b1-3e9ac5532433}" testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: pipelining replication: peer="{Voter server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 931fe7dc-09f5-2a29-e0b1-3e9ac5532433}" testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: pipelining replication: peer="{Voter server-178aa703-72b6-fd56-e030-6e845a8d3e97 178aa703-72b6-fd56-e030-6e845a8d3e97}" testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:05.795Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: pipelining replication: peer="{Voter server-178aa703-72b6-fd56-e030-6e845a8d3e97 178aa703-72b6-fd56-e030-6e845a8d3e97}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.900Z [INFO] stable state for Leader reached at 2021-08-25 09:31:05.795240614 +0000 UTC m=+10.061457018 (1 nodes), 55.490998ms from start of poll, 56.560777ms from cluster start. Timeout at 2021-08-25 09:31:05.900372279 +0000 UTC m=+10.166588683, 105.131665ms after stability testing.go:149: cluster: 2021-08-25T09:31:05.900Z [INFO] stable state for Leader reached at 2021-08-25 09:31:05.795240614 +0000 UTC m=+10.061457018 (1 nodes), 55.490998ms from start of poll, 56.560777ms from cluster start. Timeout at 2021-08-25 09:31:05.900372279 +0000 UTC m=+10.166588683, 105.131665ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:05.900Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:05.900Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.005Z [INFO] stable state for Follower reached at 2021-08-25 09:31:05.900636954 +0000 UTC m=+10.166853338 (2 nodes), 2.74µs from start of poll, 161.957097ms from cluster start. Timeout at 2021-08-25 09:31:06.005687627 +0000 UTC m=+10.271904071, 105.050733ms after stability testing.go:149: cluster: 2021-08-25T09:31:06.005Z [INFO] stable state for Follower reached at 2021-08-25 09:31:05.900636954 +0000 UTC m=+10.166853338 (2 nodes), 2.74µs from start of poll, 161.957097ms from cluster start. Timeout at 2021-08-25 09:31:06.005687627 +0000 UTC m=+10.271904071, 105.050733ms after stability testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:06.005Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: updating configuration: command=AddStaging server-id=server-178aa703-72b6-fd56-e030-6e845a8d3e97 server-addr=178aa703-72b6-fd56-e030-6e845a8d3e97 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:06.005Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: updating configuration: command=AddStaging server-id=server-178aa703-72b6-fd56-e030-6e845a8d3e97 server-addr=178aa703-72b6-fd56-e030-6e845a8d3e97 servers="[{Suffrage:Voter ID:server-178aa703-72b6-fd56-e030-6e845a8d3e97 Address:178aa703-72b6-fd56-e030-6e845a8d3e97} {Suffrage:Voter ID:server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 Address:931fe7dc-09f5-2a29-e0b1-3e9ac5532433} {Suffrage:Voter ID:server-204c7154-9551-7527-7b60-f402666ad003 Address:204c7154-9551-7527-7b60-f402666ad003}]" testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:06.006Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: aborting pipeline replication: peer="{Voter server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 931fe7dc-09f5-2a29-e0b1-3e9ac5532433}" testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:06.006Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: aborting pipeline replication: peer="{Voter server-931fe7dc-09f5-2a29-e0b1-3e9ac5532433 931fe7dc-09f5-2a29-e0b1-3e9ac5532433}" testLoggerAdapter verbose: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:06.006Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: aborting pipeline replication: peer="{Voter server-178aa703-72b6-fd56-e030-6e845a8d3e97 178aa703-72b6-fd56-e030-6e845a8d3e97}" testing.go:149: server-204c7154-9551-7527-7b60-f402666ad003: 2021-08-25T09:31:06.006Z [INFO] server-204c7154-9551-7527-7b60-f402666ad003: aborting pipeline replication: peer="{Voter server-178aa703-72b6-fd56-e030-6e845a8d3e97 178aa703-72b6-fd56-e030-6e845a8d3e97}" --- PASS: TestRaft_AddKnownPeer (0.27s) === RUN TestRaft_RemoveUnknownPeer testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.010Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.010Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.010Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: entering follower state: follower="Node at 07abca76-14fd-008b-dd28-a98ae32c361b [Follower]" leader= testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.010Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: entering follower state: follower="Node at 07abca76-14fd-008b-dd28-a98ae32c361b [Follower]" leader= testLoggerAdapter verbose: server-6d78bd77-f691-39d2-7688-e936b6b28d72: 2021-08-25T09:31:06.010Z [INFO] server-6d78bd77-f691-39d2-7688-e936b6b28d72: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testing.go:149: server-6d78bd77-f691-39d2-7688-e936b6b28d72: 2021-08-25T09:31:06.010Z [INFO] server-6d78bd77-f691-39d2-7688-e936b6b28d72: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testLoggerAdapter verbose: server-6d78bd77-f691-39d2-7688-e936b6b28d72: 2021-08-25T09:31:06.010Z [INFO] server-6d78bd77-f691-39d2-7688-e936b6b28d72: entering follower state: follower="Node at 6d78bd77-f691-39d2-7688-e936b6b28d72 [Follower]" leader= testing.go:149: server-6d78bd77-f691-39d2-7688-e936b6b28d72: 2021-08-25T09:31:06.010Z [INFO] server-6d78bd77-f691-39d2-7688-e936b6b28d72: entering follower state: follower="Node at 6d78bd77-f691-39d2-7688-e936b6b28d72 [Follower]" leader= testLoggerAdapter verbose: server-fcacfdc3-c29b-a79d-d601-3553c17c5863: 2021-08-25T09:31:06.010Z [INFO] server-fcacfdc3-c29b-a79d-d601-3553c17c5863: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testing.go:149: server-fcacfdc3-c29b-a79d-d601-3553c17c5863: 2021-08-25T09:31:06.010Z [INFO] server-fcacfdc3-c29b-a79d-d601-3553c17c5863: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.011Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:06.011Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-fcacfdc3-c29b-a79d-d601-3553c17c5863: 2021-08-25T09:31:06.011Z [INFO] server-fcacfdc3-c29b-a79d-d601-3553c17c5863: entering follower state: follower="Node at fcacfdc3-c29b-a79d-d601-3553c17c5863 [Follower]" leader= testing.go:149: server-fcacfdc3-c29b-a79d-d601-3553c17c5863: 2021-08-25T09:31:06.011Z [INFO] server-fcacfdc3-c29b-a79d-d601-3553c17c5863: entering follower state: follower="Node at fcacfdc3-c29b-a79d-d601-3553c17c5863 [Follower]" leader= testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [WARN] server-07abca76-14fd-008b-dd28-a98ae32c361b: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [WARN] server-07abca76-14fd-008b-dd28-a98ae32c361b: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: entering candidate state: node="Node at 07abca76-14fd-008b-dd28-a98ae32c361b [Candidate]" term=2 testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: entering candidate state: node="Node at 07abca76-14fd-008b-dd28-a98ae32c361b [Candidate]" term=2 testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: election won: tally=2 testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: election won: tally=2 testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: entering leader state: leader="Node at 07abca76-14fd-008b-dd28-a98ae32c361b [Leader]" testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.075Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: entering leader state: leader="Node at 07abca76-14fd-008b-dd28-a98ae32c361b [Leader]" testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: added peer, starting replication: peer=server-6d78bd77-f691-39d2-7688-e936b6b28d72 testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: added peer, starting replication: peer=server-6d78bd77-f691-39d2-7688-e936b6b28d72 testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: added peer, starting replication: peer=server-fcacfdc3-c29b-a79d-d601-3553c17c5863 testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: added peer, starting replication: peer=server-fcacfdc3-c29b-a79d-d601-3553c17c5863 testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: pipelining replication: peer="{Voter server-6d78bd77-f691-39d2-7688-e936b6b28d72 6d78bd77-f691-39d2-7688-e936b6b28d72}" testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: pipelining replication: peer="{Voter server-6d78bd77-f691-39d2-7688-e936b6b28d72 6d78bd77-f691-39d2-7688-e936b6b28d72}" testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: pipelining replication: peer="{Voter server-fcacfdc3-c29b-a79d-d601-3553c17c5863 fcacfdc3-c29b-a79d-d601-3553c17c5863}" testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.076Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: pipelining replication: peer="{Voter server-fcacfdc3-c29b-a79d-d601-3553c17c5863 fcacfdc3-c29b-a79d-d601-3553c17c5863}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.181Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.076126157 +0000 UTC m=+10.342342561 (1 nodes), 65.042078ms from start of poll, 66.117857ms from cluster start. Timeout at 2021-08-25 09:31:06.181216026 +0000 UTC m=+10.447432430, 105.089869ms after stability testing.go:149: cluster: 2021-08-25T09:31:06.181Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.076126157 +0000 UTC m=+10.342342561 (1 nodes), 65.042078ms from start of poll, 66.117857ms from cluster start. Timeout at 2021-08-25 09:31:06.181216026 +0000 UTC m=+10.447432430, 105.089869ms after stability testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.181Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: updating configuration: command=RemoveServer server-id=e0344d06-3ea4-1a5d-8d2d-1dcc88b0af24 server-addr= servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.181Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: updating configuration: command=RemoveServer server-id=e0344d06-3ea4-1a5d-8d2d-1dcc88b0af24 server-addr= servers="[{Suffrage:Voter ID:server-07abca76-14fd-008b-dd28-a98ae32c361b Address:07abca76-14fd-008b-dd28-a98ae32c361b} {Suffrage:Voter ID:server-6d78bd77-f691-39d2-7688-e936b6b28d72 Address:6d78bd77-f691-39d2-7688-e936b6b28d72} {Suffrage:Voter ID:server-fcacfdc3-c29b-a79d-d601-3553c17c5863 Address:fcacfdc3-c29b-a79d-d601-3553c17c5863}]" testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.182Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: aborting pipeline replication: peer="{Voter server-fcacfdc3-c29b-a79d-d601-3553c17c5863 fcacfdc3-c29b-a79d-d601-3553c17c5863}" testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.182Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: aborting pipeline replication: peer="{Voter server-fcacfdc3-c29b-a79d-d601-3553c17c5863 fcacfdc3-c29b-a79d-d601-3553c17c5863}" testLoggerAdapter verbose: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.182Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: aborting pipeline replication: peer="{Voter server-6d78bd77-f691-39d2-7688-e936b6b28d72 6d78bd77-f691-39d2-7688-e936b6b28d72}" testing.go:149: server-07abca76-14fd-008b-dd28-a98ae32c361b: 2021-08-25T09:31:06.182Z [INFO] server-07abca76-14fd-008b-dd28-a98ae32c361b: aborting pipeline replication: peer="{Voter server-6d78bd77-f691-39d2-7688-e936b6b28d72 6d78bd77-f691-39d2-7688-e936b6b28d72}" --- PASS: TestRaft_RemoveUnknownPeer (0.18s) === RUN TestRaft_SnapshotRestore testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.186Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-eee6c360-137e-37a5-657f-f0032536f79e Address:eee6c360-137e-37a5-657f-f0032536f79e}]" testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.186Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-eee6c360-137e-37a5-657f-f0032536f79e Address:eee6c360-137e-37a5-657f-f0032536f79e}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.186Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:06.186Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.186Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: entering follower state: follower="Node at eee6c360-137e-37a5-657f-f0032536f79e [Follower]" leader= testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.186Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: entering follower state: follower="Node at eee6c360-137e-37a5-657f-f0032536f79e [Follower]" leader= testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.267Z [WARN] server-eee6c360-137e-37a5-657f-f0032536f79e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.267Z [WARN] server-eee6c360-137e-37a5-657f-f0032536f79e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.267Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: entering candidate state: node="Node at eee6c360-137e-37a5-657f-f0032536f79e [Candidate]" term=2 testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.267Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: entering candidate state: node="Node at eee6c360-137e-37a5-657f-f0032536f79e [Candidate]" term=2 testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.268Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: election won: tally=1 testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.268Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: election won: tally=1 testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.268Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: entering leader state: leader="Node at eee6c360-137e-37a5-657f-f0032536f79e [Leader]" testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.268Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: entering leader state: leader="Node at eee6c360-137e-37a5-657f-f0032536f79e [Leader]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.373Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.268324468 +0000 UTC m=+10.534540912 (1 nodes), 82.02212ms from start of poll, 82.517134ms from cluster start. Timeout at 2021-08-25 09:31:06.373460193 +0000 UTC m=+10.639676697, 105.135785ms after stability testing.go:149: cluster: 2021-08-25T09:31:06.373Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.268324468 +0000 UTC m=+10.534540912 (1 nodes), 82.02212ms from start of poll, 82.517134ms from cluster start. Timeout at 2021-08-25 09:31:06.373460193 +0000 UTC m=+10.639676697, 105.135785ms after stability testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.376Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: starting snapshot up to: index=102 testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.376Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: starting snapshot up to: index=102 testing.go:153: 2021-08-25T09:31:06.376Z [INFO] creating new snapshot: path=/tmp/raft490931991/snapshots/2-102-1629883866376.tmp testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.401Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: compacting logs: from=1 to=92 testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.401Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.401Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: snapshot complete up to: index=102 testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.401Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: snapshot complete up to: index=102 testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.402Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: restored from snapshot: id=2-102-1629883866376 testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.402Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: restored from snapshot: id=2-102-1629883866376 testLoggerAdapter verbose: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.402Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-eee6c360-137e-37a5-657f-f0032536f79e Address:eee6c360-137e-37a5-657f-f0032536f79e}]" testing.go:149: server-eee6c360-137e-37a5-657f-f0032536f79e: 2021-08-25T09:31:06.402Z [INFO] server-eee6c360-137e-37a5-657f-f0032536f79e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-eee6c360-137e-37a5-657f-f0032536f79e Address:eee6c360-137e-37a5-657f-f0032536f79e}]" --- PASS: TestRaft_SnapshotRestore (0.22s) === RUN TestRaft_NoRestoreOnStart testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.404Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-243ae5f3-caef-0397-a1cb-b04b55875af8 Address:243ae5f3-caef-0397-a1cb-b04b55875af8}]" testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.404Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-243ae5f3-caef-0397-a1cb-b04b55875af8 Address:243ae5f3-caef-0397-a1cb-b04b55875af8}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.404Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:06.404Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.404Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering follower state: follower="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Follower]" leader= testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.404Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering follower state: follower="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Follower]" leader= testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.462Z [WARN] server-243ae5f3-caef-0397-a1cb-b04b55875af8: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.462Z [WARN] server-243ae5f3-caef-0397-a1cb-b04b55875af8: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.462Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering candidate state: node="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Candidate]" term=2 testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.462Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering candidate state: node="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Candidate]" term=2 testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.463Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: election won: tally=1 testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.463Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: election won: tally=1 testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.463Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering leader state: leader="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Leader]" testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.463Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering leader state: leader="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Leader]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.568Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.463134373 +0000 UTC m=+10.729350797 (1 nodes), 58.430541ms from start of poll, 58.86158ms from cluster start. Timeout at 2021-08-25 09:31:06.568286196 +0000 UTC m=+10.834502620, 105.151823ms after stability testing.go:149: cluster: 2021-08-25T09:31:06.568Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.463134373 +0000 UTC m=+10.729350797 (1 nodes), 58.430541ms from start of poll, 58.86158ms from cluster start. Timeout at 2021-08-25 09:31:06.568286196 +0000 UTC m=+10.834502620, 105.151823ms after stability testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.570Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: starting snapshot up to: index=102 testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.570Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: starting snapshot up to: index=102 testing.go:153: 2021-08-25T09:31:06.570Z [INFO] creating new snapshot: path=/tmp/raft294211937/snapshots/2-102-1629883866570.tmp testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.592Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: compacting logs: from=1 to=92 testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.592Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.592Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: snapshot complete up to: index=102 testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.592Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: snapshot complete up to: index=102 testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.592Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-243ae5f3-caef-0397-a1cb-b04b55875af8 Address:243ae5f3-caef-0397-a1cb-b04b55875af8}]" testing.go:149: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.592Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-243ae5f3-caef-0397-a1cb-b04b55875af8 Address:243ae5f3-caef-0397-a1cb-b04b55875af8}]" --- PASS: TestRaft_NoRestoreOnStart (0.19s) === RUN TestRaft_SnapshotRestore_PeerChange testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.592Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering follower state: follower="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Follower]" leader= raft_test.go:1026: Disabled unreliable test --- SKIP: TestRaft_SnapshotRestore_PeerChange (0.00s) === RUN TestRaft_AutoSnapshot raft_test.go:1132: Disabled unreliable test --- SKIP: TestRaft_AutoSnapshot (0.00s) === RUN TestRaft_UserSnapshot testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.594Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 Address:8cae2db2-e399-b4b5-a1db-4e5a7ce86f73}]" testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.594Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 Address:8cae2db2-e399-b4b5-a1db-4e5a7ce86f73}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.594Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:06.594Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.594Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: entering follower state: follower="Node at 8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 [Follower]" leader= testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.594Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: entering follower state: follower="Node at 8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 [Follower]" leader= testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [WARN] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [WARN] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: entering candidate state: node="Node at 8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 [Candidate]" term=2 testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: entering candidate state: node="Node at 8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 [Candidate]" term=2 testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: election won: tally=1 testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: election won: tally=1 testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: entering leader state: leader="Node at 8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 [Leader]" testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.687Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: entering leader state: leader="Node at 8cae2db2-e399-b4b5-a1db-4e5a7ce86f73 [Leader]" testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.691Z [WARN] server-243ae5f3-caef-0397-a1cb-b04b55875af8: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.691Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering candidate state: node="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Candidate]" term=3 testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.691Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: election won: tally=1 testLoggerAdapter verbose: server-243ae5f3-caef-0397-a1cb-b04b55875af8: 2021-08-25T09:31:06.691Z [INFO] server-243ae5f3-caef-0397-a1cb-b04b55875af8: entering leader state: leader="Node at 243ae5f3-caef-0397-a1cb-b04b55875af8 [Leader]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.792Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.687588727 +0000 UTC m=+10.953805151 (1 nodes), 93.380711ms from start of poll, 93.81589ms from cluster start. Timeout at 2021-08-25 09:31:06.792759409 +0000 UTC m=+11.058975913, 105.170762ms after stability testing.go:149: cluster: 2021-08-25T09:31:06.792Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.687588727 +0000 UTC m=+10.953805151 (1 nodes), 93.380711ms from start of poll, 93.81589ms from cluster start. Timeout at 2021-08-25 09:31:06.792759409 +0000 UTC m=+11.058975913, 105.170762ms after stability testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.793Z [ERROR] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.793Z [ERROR] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.793Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: starting snapshot up to: index=12 testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.793Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: starting snapshot up to: index=12 testing.go:153: 2021-08-25T09:31:06.793Z [INFO] creating new snapshot: path=/tmp/raft795130427/snapshots/2-12-1629883866793.tmp testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.818Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: compacting logs: from=1 to=2 testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.818Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: compacting logs: from=1 to=2 testLoggerAdapter verbose: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.818Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: snapshot complete up to: index=12 testing.go:149: server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: 2021-08-25T09:31:06.818Z [INFO] server-8cae2db2-e399-b4b5-a1db-4e5a7ce86f73: snapshot complete up to: index=12 --- PASS: TestRaft_UserSnapshot (0.23s) === RUN TestRaft_UserRestore testLoggerAdapter verbose: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:06.821Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f Address:d3508785-46c1-1a48-0b5b-cc5b0c34e53f} {Suffrage:Voter ID:server-aae7d584-79fe-9436-96ff-a3ba5b503a79 Address:aae7d584-79fe-9436-96ff-a3ba5b503a79} {Suffrage:Voter ID:server-55987921-1594-d891-edaa-d257d630bb5e Address:55987921-1594-d891-edaa-d257d630bb5e}]" testing.go:149: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:06.821Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f Address:d3508785-46c1-1a48-0b5b-cc5b0c34e53f} {Suffrage:Voter ID:server-aae7d584-79fe-9436-96ff-a3ba5b503a79 Address:aae7d584-79fe-9436-96ff-a3ba5b503a79} {Suffrage:Voter ID:server-55987921-1594-d891-edaa-d257d630bb5e Address:55987921-1594-d891-edaa-d257d630bb5e}]" testLoggerAdapter verbose: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:06.822Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: entering follower state: follower="Node at d3508785-46c1-1a48-0b5b-cc5b0c34e53f [Follower]" leader= testLoggerAdapter verbose: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:06.822Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f Address:d3508785-46c1-1a48-0b5b-cc5b0c34e53f} {Suffrage:Voter ID:server-aae7d584-79fe-9436-96ff-a3ba5b503a79 Address:aae7d584-79fe-9436-96ff-a3ba5b503a79} {Suffrage:Voter ID:server-55987921-1594-d891-edaa-d257d630bb5e Address:55987921-1594-d891-edaa-d257d630bb5e}]" testing.go:149: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:06.822Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: entering follower state: follower="Node at d3508785-46c1-1a48-0b5b-cc5b0c34e53f [Follower]" leader= testing.go:149: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:06.822Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f Address:d3508785-46c1-1a48-0b5b-cc5b0c34e53f} {Suffrage:Voter ID:server-aae7d584-79fe-9436-96ff-a3ba5b503a79 Address:aae7d584-79fe-9436-96ff-a3ba5b503a79} {Suffrage:Voter ID:server-55987921-1594-d891-edaa-d257d630bb5e Address:55987921-1594-d891-edaa-d257d630bb5e}]" testLoggerAdapter verbose: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:06.822Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: entering follower state: follower="Node at aae7d584-79fe-9436-96ff-a3ba5b503a79 [Follower]" leader= testing.go:149: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:06.822Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: entering follower state: follower="Node at aae7d584-79fe-9436-96ff-a3ba5b503a79 [Follower]" leader= testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.822Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f Address:d3508785-46c1-1a48-0b5b-cc5b0c34e53f} {Suffrage:Voter ID:server-aae7d584-79fe-9436-96ff-a3ba5b503a79 Address:aae7d584-79fe-9436-96ff-a3ba5b503a79} {Suffrage:Voter ID:server-55987921-1594-d891-edaa-d257d630bb5e Address:55987921-1594-d891-edaa-d257d630bb5e}]" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.822Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f Address:d3508785-46c1-1a48-0b5b-cc5b0c34e53f} {Suffrage:Voter ID:server-aae7d584-79fe-9436-96ff-a3ba5b503a79 Address:aae7d584-79fe-9436-96ff-a3ba5b503a79} {Suffrage:Voter ID:server-55987921-1594-d891-edaa-d257d630bb5e Address:55987921-1594-d891-edaa-d257d630bb5e}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.822Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:06.822Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.822Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: entering follower state: follower="Node at 55987921-1594-d891-edaa-d257d630bb5e [Follower]" leader= testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.822Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: entering follower state: follower="Node at 55987921-1594-d891-edaa-d257d630bb5e [Follower]" leader= testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [WARN] server-55987921-1594-d891-edaa-d257d630bb5e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [WARN] server-55987921-1594-d891-edaa-d257d630bb5e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: entering candidate state: node="Node at 55987921-1594-d891-edaa-d257d630bb5e [Candidate]" term=2 testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: entering candidate state: node="Node at 55987921-1594-d891-edaa-d257d630bb5e [Candidate]" term=2 testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: election won: tally=2 testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: election won: tally=2 testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: entering leader state: leader="Node at 55987921-1594-d891-edaa-d257d630bb5e [Leader]" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: entering leader state: leader="Node at 55987921-1594-d891-edaa-d257d630bb5e [Leader]" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: added peer, starting replication: peer=server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: added peer, starting replication: peer=server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: added peer, starting replication: peer=server-aae7d584-79fe-9436-96ff-a3ba5b503a79 testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: added peer, starting replication: peer=server-aae7d584-79fe-9436-96ff-a3ba5b503a79 testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.877Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.878Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.878Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:06.983Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.877802025 +0000 UTC m=+11.144018449 (1 nodes), 54.92649ms from start of poll, 56.173073ms from cluster start. Timeout at 2021-08-25 09:31:06.982952128 +0000 UTC m=+11.249168572, 105.150123ms after stability testing.go:149: cluster: 2021-08-25T09:31:06.983Z [INFO] stable state for Leader reached at 2021-08-25 09:31:06.877802025 +0000 UTC m=+11.144018449 (1 nodes), 54.92649ms from start of poll, 56.173073ms from cluster start. Timeout at 2021-08-25 09:31:06.982952128 +0000 UTC m=+11.249168572, 105.150123ms after stability testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.983Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: starting snapshot up to: index=12 testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:06.983Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: starting snapshot up to: index=12 testing.go:153: 2021-08-25T09:31:06.983Z [INFO] creating new snapshot: path=/tmp/raft865700137/snapshots/2-12-1629883866983.tmp testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.011Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: snapshot complete up to: index=12 testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.011Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: snapshot complete up to: index=12 testing.go:153: 2021-08-25T09:31:07.012Z [INFO] creating new snapshot: path=/tmp/raft865700137/snapshots/2-23-1629883867012.tmp testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.034Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: copied to local snapshot: bytes=71 testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.034Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.034Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: restored user snapshot: index=1 testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.034Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: restored user snapshot: index=1 testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.035Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.042Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.042Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:07.042Z [INFO] creating new snapshot: path=/tmp/raft257921573/snapshots/2-23-1629883867042.tmp testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.044Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.044Z [ERROR] server-55987921-1594-d891-edaa-d257d630bb5e: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:07.045Z [INFO] creating new snapshot: path=/tmp/raft986466719/snapshots/2-23-1629883867045.tmp testLoggerAdapter verbose: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:07.078Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: copied to local snapshot: bytes=71 testing.go:149: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:07.078Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:07.078Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: Installed remote snapshot testing.go:149: server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: 2021-08-25T09:31:07.078Z [INFO] server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f: Installed remote snapshot testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.079Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.079Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testLoggerAdapter verbose: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:07.084Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: copied to local snapshot: bytes=71 testing.go:149: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:07.084Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:07.084Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: Installed remote snapshot testing.go:149: server-aae7d584-79fe-9436-96ff-a3ba5b503a79: 2021-08-25T09:31:07.084Z [INFO] server-aae7d584-79fe-9436-96ff-a3ba5b503a79: Installed remote snapshot testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.084Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.084Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: pipelining replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.100Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.100Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-aae7d584-79fe-9436-96ff-a3ba5b503a79 aae7d584-79fe-9436-96ff-a3ba5b503a79}" testLoggerAdapter verbose: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.100Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testing.go:149: server-55987921-1594-d891-edaa-d257d630bb5e: 2021-08-25T09:31:07.100Z [INFO] server-55987921-1594-d891-edaa-d257d630bb5e: aborting pipeline replication: peer="{Voter server-d3508785-46c1-1a48-0b5b-cc5b0c34e53f d3508785-46c1-1a48-0b5b-cc5b0c34e53f}" testLoggerAdapter verbose: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.106Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 Address:99e8c11c-5748-dc3d-1ec8-5917b84ca858} {Suffrage:Voter ID:server-d716f667-3c96-a766-bed3-a5ff266b731c Address:d716f667-3c96-a766-bed3-a5ff266b731c} {Suffrage:Voter ID:server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6 Address:8adc0d77-e42d-0b5a-d11a-6d08377a70d6}]" testing.go:149: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.106Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 Address:99e8c11c-5748-dc3d-1ec8-5917b84ca858} {Suffrage:Voter ID:server-d716f667-3c96-a766-bed3-a5ff266b731c Address:d716f667-3c96-a766-bed3-a5ff266b731c} {Suffrage:Voter ID:server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6 Address:8adc0d77-e42d-0b5a-d11a-6d08377a70d6}]" testLoggerAdapter verbose: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.106Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: entering follower state: follower="Node at 99e8c11c-5748-dc3d-1ec8-5917b84ca858 [Follower]" leader= testing.go:149: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.106Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: entering follower state: follower="Node at 99e8c11c-5748-dc3d-1ec8-5917b84ca858 [Follower]" leader= testLoggerAdapter verbose: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.106Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 Address:99e8c11c-5748-dc3d-1ec8-5917b84ca858} {Suffrage:Voter ID:server-d716f667-3c96-a766-bed3-a5ff266b731c Address:d716f667-3c96-a766-bed3-a5ff266b731c} {Suffrage:Voter ID:server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6 Address:8adc0d77-e42d-0b5a-d11a-6d08377a70d6}]" testing.go:149: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.106Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 Address:99e8c11c-5748-dc3d-1ec8-5917b84ca858} {Suffrage:Voter ID:server-d716f667-3c96-a766-bed3-a5ff266b731c Address:d716f667-3c96-a766-bed3-a5ff266b731c} {Suffrage:Voter ID:server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6 Address:8adc0d77-e42d-0b5a-d11a-6d08377a70d6}]" testLoggerAdapter verbose: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.107Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: entering follower state: follower="Node at d716f667-3c96-a766-bed3-a5ff266b731c [Follower]" leader= testing.go:149: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.107Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: entering follower state: follower="Node at d716f667-3c96-a766-bed3-a5ff266b731c [Follower]" leader= testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.107Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 Address:99e8c11c-5748-dc3d-1ec8-5917b84ca858} {Suffrage:Voter ID:server-d716f667-3c96-a766-bed3-a5ff266b731c Address:d716f667-3c96-a766-bed3-a5ff266b731c} {Suffrage:Voter ID:server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6 Address:8adc0d77-e42d-0b5a-d11a-6d08377a70d6}]" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.107Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 Address:99e8c11c-5748-dc3d-1ec8-5917b84ca858} {Suffrage:Voter ID:server-d716f667-3c96-a766-bed3-a5ff266b731c Address:d716f667-3c96-a766-bed3-a5ff266b731c} {Suffrage:Voter ID:server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6 Address:8adc0d77-e42d-0b5a-d11a-6d08377a70d6}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:07.107Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:07.107Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.107Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: entering follower state: follower="Node at 8adc0d77-e42d-0b5a-d11a-6d08377a70d6 [Follower]" leader= testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.107Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: entering follower state: follower="Node at 8adc0d77-e42d-0b5a-d11a-6d08377a70d6 [Follower]" leader= testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [WARN] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [WARN] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: entering candidate state: node="Node at 8adc0d77-e42d-0b5a-d11a-6d08377a70d6 [Candidate]" term=2 testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: entering candidate state: node="Node at 8adc0d77-e42d-0b5a-d11a-6d08377a70d6 [Candidate]" term=2 testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: election won: tally=2 testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: election won: tally=2 testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: entering leader state: leader="Node at 8adc0d77-e42d-0b5a-d11a-6d08377a70d6 [Leader]" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: entering leader state: leader="Node at 8adc0d77-e42d-0b5a-d11a-6d08377a70d6 [Leader]" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: added peer, starting replication: peer=server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: added peer, starting replication: peer=server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: added peer, starting replication: peer=server-d716f667-3c96-a766-bed3-a5ff266b731c testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.159Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: added peer, starting replication: peer=server-d716f667-3c96-a766-bed3-a5ff266b731c testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.160Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.160Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.160Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.160Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:07.265Z [INFO] stable state for Leader reached at 2021-08-25 09:31:07.160116074 +0000 UTC m=+11.426332478 (1 nodes), 52.621567ms from start of poll, 53.809515ms from cluster start. Timeout at 2021-08-25 09:31:07.265210603 +0000 UTC m=+11.531427027, 105.094549ms after stability testing.go:149: cluster: 2021-08-25T09:31:07.265Z [INFO] stable state for Leader reached at 2021-08-25 09:31:07.160116074 +0000 UTC m=+11.426332478 (1 nodes), 52.621567ms from start of poll, 53.809515ms from cluster start. Timeout at 2021-08-25 09:31:07.265210603 +0000 UTC m=+11.531427027, 105.094549ms after stability testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.265Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: starting snapshot up to: index=12 testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.265Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: starting snapshot up to: index=12 testing.go:153: 2021-08-25T09:31:07.266Z [INFO] creating new snapshot: path=/tmp/raft098219303/snapshots/2-12-1629883867266.tmp testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.293Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: snapshot complete up to: index=12 testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.293Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: snapshot complete up to: index=12 testing.go:153: 2021-08-25T09:31:07.294Z [INFO] creating new snapshot: path=/tmp/raft098219303/snapshots/2-23-1629883867294.tmp testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: copied to local snapshot: bytes=71 testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: restored user snapshot: index=1 testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: restored user snapshot: index=1 testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.316Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.317Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.317Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.323Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.323Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:07.324Z [INFO] creating new snapshot: path=/tmp/raft877753965/snapshots/2-23-1629883867324.tmp testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.325Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.325Z [ERROR] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:07.326Z [INFO] creating new snapshot: path=/tmp/raft990833475/snapshots/2-23-1629883867326.tmp testLoggerAdapter verbose: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.360Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: copied to local snapshot: bytes=71 testing.go:149: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.360Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.360Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: Installed remote snapshot testing.go:149: server-d716f667-3c96-a766-bed3-a5ff266b731c: 2021-08-25T09:31:07.360Z [INFO] server-d716f667-3c96-a766-bed3-a5ff266b731c: Installed remote snapshot testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.361Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.361Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testLoggerAdapter verbose: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.365Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: copied to local snapshot: bytes=71 testing.go:149: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.365Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.367Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: Installed remote snapshot testing.go:149: server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: 2021-08-25T09:31:07.367Z [INFO] server-99e8c11c-5748-dc3d-1ec8-5917b84ca858: Installed remote snapshot testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.367Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.367Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: pipelining replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.383Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.383Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-d716f667-3c96-a766-bed3-a5ff266b731c d716f667-3c96-a766-bed3-a5ff266b731c}" testLoggerAdapter verbose: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.383Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testing.go:149: server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: 2021-08-25T09:31:07.383Z [INFO] server-8adc0d77-e42d-0b5a-d11a-6d08377a70d6: aborting pipeline replication: peer="{Voter server-99e8c11c-5748-dc3d-1ec8-5917b84ca858 99e8c11c-5748-dc3d-1ec8-5917b84ca858}" testLoggerAdapter verbose: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.388Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 Address:f2bf917a-575d-0ae4-37bb-36e38d7840a1} {Suffrage:Voter ID:server-b2dbee26-bcdb-f817-96ea-99c621047f4a Address:b2dbee26-bcdb-f817-96ea-99c621047f4a} {Suffrage:Voter ID:server-6d1d8058-5590-ab98-830f-28dd8ccc8100 Address:6d1d8058-5590-ab98-830f-28dd8ccc8100}]" testing.go:149: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.388Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 Address:f2bf917a-575d-0ae4-37bb-36e38d7840a1} {Suffrage:Voter ID:server-b2dbee26-bcdb-f817-96ea-99c621047f4a Address:b2dbee26-bcdb-f817-96ea-99c621047f4a} {Suffrage:Voter ID:server-6d1d8058-5590-ab98-830f-28dd8ccc8100 Address:6d1d8058-5590-ab98-830f-28dd8ccc8100}]" testLoggerAdapter verbose: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.388Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: entering follower state: follower="Node at f2bf917a-575d-0ae4-37bb-36e38d7840a1 [Follower]" leader= testing.go:149: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.388Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: entering follower state: follower="Node at f2bf917a-575d-0ae4-37bb-36e38d7840a1 [Follower]" leader= testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.388Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 Address:f2bf917a-575d-0ae4-37bb-36e38d7840a1} {Suffrage:Voter ID:server-b2dbee26-bcdb-f817-96ea-99c621047f4a Address:b2dbee26-bcdb-f817-96ea-99c621047f4a} {Suffrage:Voter ID:server-6d1d8058-5590-ab98-830f-28dd8ccc8100 Address:6d1d8058-5590-ab98-830f-28dd8ccc8100}]" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.388Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 Address:f2bf917a-575d-0ae4-37bb-36e38d7840a1} {Suffrage:Voter ID:server-b2dbee26-bcdb-f817-96ea-99c621047f4a Address:b2dbee26-bcdb-f817-96ea-99c621047f4a} {Suffrage:Voter ID:server-6d1d8058-5590-ab98-830f-28dd8ccc8100 Address:6d1d8058-5590-ab98-830f-28dd8ccc8100}]" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.389Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: entering follower state: follower="Node at b2dbee26-bcdb-f817-96ea-99c621047f4a [Follower]" leader= testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.389Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: entering follower state: follower="Node at b2dbee26-bcdb-f817-96ea-99c621047f4a [Follower]" leader= testLoggerAdapter verbose: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.389Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 Address:f2bf917a-575d-0ae4-37bb-36e38d7840a1} {Suffrage:Voter ID:server-b2dbee26-bcdb-f817-96ea-99c621047f4a Address:b2dbee26-bcdb-f817-96ea-99c621047f4a} {Suffrage:Voter ID:server-6d1d8058-5590-ab98-830f-28dd8ccc8100 Address:6d1d8058-5590-ab98-830f-28dd8ccc8100}]" testing.go:149: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.389Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 Address:f2bf917a-575d-0ae4-37bb-36e38d7840a1} {Suffrage:Voter ID:server-b2dbee26-bcdb-f817-96ea-99c621047f4a Address:b2dbee26-bcdb-f817-96ea-99c621047f4a} {Suffrage:Voter ID:server-6d1d8058-5590-ab98-830f-28dd8ccc8100 Address:6d1d8058-5590-ab98-830f-28dd8ccc8100}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:07.389Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:07.389Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.389Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: entering follower state: follower="Node at 6d1d8058-5590-ab98-830f-28dd8ccc8100 [Follower]" leader= testing.go:149: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.389Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: entering follower state: follower="Node at 6d1d8058-5590-ab98-830f-28dd8ccc8100 [Follower]" leader= testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [WARN] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [WARN] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: entering candidate state: node="Node at b2dbee26-bcdb-f817-96ea-99c621047f4a [Candidate]" term=2 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: entering candidate state: node="Node at b2dbee26-bcdb-f817-96ea-99c621047f4a [Candidate]" term=2 testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: election won: tally=2 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: election won: tally=2 testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: entering leader state: leader="Node at b2dbee26-bcdb-f817-96ea-99c621047f4a [Leader]" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: entering leader state: leader="Node at b2dbee26-bcdb-f817-96ea-99c621047f4a [Leader]" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: added peer, starting replication: peer=server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: added peer, starting replication: peer=server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: added peer, starting replication: peer=server-6d1d8058-5590-ab98-830f-28dd8ccc8100 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.443Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: added peer, starting replication: peer=server-6d1d8058-5590-ab98-830f-28dd8ccc8100 testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.444Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.444Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.444Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.444Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:07.548Z [INFO] stable state for Leader reached at 2021-08-25 09:31:07.443841091 +0000 UTC m=+11.710057495 (1 nodes), 54.468514ms from start of poll, 55.541093ms from cluster start. Timeout at 2021-08-25 09:31:07.548848148 +0000 UTC m=+11.815064632, 105.007137ms after stability testing.go:149: cluster: 2021-08-25T09:31:07.548Z [INFO] stable state for Leader reached at 2021-08-25 09:31:07.443841091 +0000 UTC m=+11.710057495 (1 nodes), 54.468514ms from start of poll, 55.541093ms from cluster start. Timeout at 2021-08-25 09:31:07.548848148 +0000 UTC m=+11.815064632, 105.007137ms after stability testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.549Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: starting snapshot up to: index=12 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.549Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: starting snapshot up to: index=12 testing.go:153: 2021-08-25T09:31:07.549Z [INFO] creating new snapshot: path=/tmp/raft152936267/snapshots/2-12-1629883867549.tmp testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.577Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: snapshot complete up to: index=12 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.577Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: snapshot complete up to: index=12 testing.go:153: 2021-08-25T09:31:07.578Z [INFO] creating new snapshot: path=/tmp/raft152936267/snapshots/2-23-1629883867578.tmp testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.598Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: copied to local snapshot: bytes=71 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.598Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: restored user snapshot: index=1 testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: restored user snapshot: index=1 testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.599Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.606Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.606Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.606Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.606Z [ERROR] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:07.606Z [INFO] creating new snapshot: path=/tmp/raft255563761/snapshots/2-23-1629883867606.tmp testing.go:153: 2021-08-25T09:31:07.607Z [INFO] creating new snapshot: path=/tmp/raft765636021/snapshots/2-23-1629883867607.tmp testLoggerAdapter verbose: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.643Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: copied to local snapshot: bytes=71 testing.go:149: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.643Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.643Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: Installed remote snapshot testing.go:149: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.643Z [INFO] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: Installed remote snapshot testLoggerAdapter verbose: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.644Z [WARN] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: skipping application of old log: index=23 testing.go:149: server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: 2021-08-25T09:31:07.644Z [WARN] server-f2bf917a-575d-0ae4-37bb-36e38d7840a1: skipping application of old log: index=23 testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.644Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.644Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testLoggerAdapter verbose: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.648Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: copied to local snapshot: bytes=71 testing.go:149: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.648Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.649Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: Installed remote snapshot testing.go:149: server-6d1d8058-5590-ab98-830f-28dd8ccc8100: 2021-08-25T09:31:07.649Z [INFO] server-6d1d8058-5590-ab98-830f-28dd8ccc8100: Installed remote snapshot testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.649Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.649Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: pipelining replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.655Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.655Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-6d1d8058-5590-ab98-830f-28dd8ccc8100 6d1d8058-5590-ab98-830f-28dd8ccc8100}" testLoggerAdapter verbose: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.655Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testing.go:149: server-b2dbee26-bcdb-f817-96ea-99c621047f4a: 2021-08-25T09:31:07.655Z [INFO] server-b2dbee26-bcdb-f817-96ea-99c621047f4a: aborting pipeline replication: peer="{Voter server-f2bf917a-575d-0ae4-37bb-36e38d7840a1 f2bf917a-575d-0ae4-37bb-36e38d7840a1}" testLoggerAdapter verbose: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.659Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 Address:5eeecd91-d2d0-75a6-6b5e-0eb50cc34593} {Suffrage:Voter ID:server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 Address:04723eb6-dbe6-d7cb-b272-204e75aebdd9} {Suffrage:Voter ID:server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 Address:05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12}]" testing.go:149: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.659Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 Address:5eeecd91-d2d0-75a6-6b5e-0eb50cc34593} {Suffrage:Voter ID:server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 Address:04723eb6-dbe6-d7cb-b272-204e75aebdd9} {Suffrage:Voter ID:server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 Address:05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12}]" testLoggerAdapter verbose: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.660Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: entering follower state: follower="Node at 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 [Follower]" leader= testing.go:149: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.660Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: entering follower state: follower="Node at 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 [Follower]" leader= testLoggerAdapter verbose: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.660Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 Address:5eeecd91-d2d0-75a6-6b5e-0eb50cc34593} {Suffrage:Voter ID:server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 Address:04723eb6-dbe6-d7cb-b272-204e75aebdd9} {Suffrage:Voter ID:server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 Address:05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12}]" testing.go:149: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.660Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 Address:5eeecd91-d2d0-75a6-6b5e-0eb50cc34593} {Suffrage:Voter ID:server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 Address:04723eb6-dbe6-d7cb-b272-204e75aebdd9} {Suffrage:Voter ID:server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 Address:05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12}]" testLoggerAdapter verbose: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.660Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: entering follower state: follower="Node at 04723eb6-dbe6-d7cb-b272-204e75aebdd9 [Follower]" leader= testing.go:149: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.660Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: entering follower state: follower="Node at 04723eb6-dbe6-d7cb-b272-204e75aebdd9 [Follower]" leader= testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.660Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 Address:5eeecd91-d2d0-75a6-6b5e-0eb50cc34593} {Suffrage:Voter ID:server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 Address:04723eb6-dbe6-d7cb-b272-204e75aebdd9} {Suffrage:Voter ID:server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 Address:05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12}]" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.660Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 Address:5eeecd91-d2d0-75a6-6b5e-0eb50cc34593} {Suffrage:Voter ID:server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 Address:04723eb6-dbe6-d7cb-b272-204e75aebdd9} {Suffrage:Voter ID:server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 Address:05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:07.660Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:07.660Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.660Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: entering follower state: follower="Node at 05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 [Follower]" leader= testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.660Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: entering follower state: follower="Node at 05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 [Follower]" leader= testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [WARN] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [WARN] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: entering candidate state: node="Node at 05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 [Candidate]" term=2 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: entering candidate state: node="Node at 05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 [Candidate]" term=2 testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: election won: tally=2 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: election won: tally=2 testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: entering leader state: leader="Node at 05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 [Leader]" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: entering leader state: leader="Node at 05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12 [Leader]" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: added peer, starting replication: peer=server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: added peer, starting replication: peer=server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: added peer, starting replication: peer=server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: added peer, starting replication: peer=server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.718Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:07.823Z [INFO] stable state for Leader reached at 2021-08-25 09:31:07.718675144 +0000 UTC m=+11.984891568 (1 nodes), 58.006241ms from start of poll, 58.986308ms from cluster start. Timeout at 2021-08-25 09:31:07.823791891 +0000 UTC m=+12.090008295, 105.116727ms after stability testing.go:149: cluster: 2021-08-25T09:31:07.823Z [INFO] stable state for Leader reached at 2021-08-25 09:31:07.718675144 +0000 UTC m=+11.984891568 (1 nodes), 58.006241ms from start of poll, 58.986308ms from cluster start. Timeout at 2021-08-25 09:31:07.823791891 +0000 UTC m=+12.090008295, 105.116727ms after stability testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.824Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: starting snapshot up to: index=12 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.824Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: starting snapshot up to: index=12 testing.go:153: 2021-08-25T09:31:07.824Z [INFO] creating new snapshot: path=/tmp/raft248361555/snapshots/2-12-1629883867824.tmp testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.851Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: snapshot complete up to: index=12 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.851Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: snapshot complete up to: index=12 testing.go:153: 2021-08-25T09:31:07.852Z [INFO] creating new snapshot: path=/tmp/raft248361555/snapshots/2-113-1629883867851.tmp testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.872Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: copied to local snapshot: bytes=71 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.872Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: restored user snapshot: index=1 testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: restored user snapshot: index=1 testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.873Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.879Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.879Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:07.880Z [INFO] creating new snapshot: path=/tmp/raft549052847/snapshots/2-113-1629883867880.tmp testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.881Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.881Z [ERROR] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:07.882Z [INFO] creating new snapshot: path=/tmp/raft328772537/snapshots/2-113-1629883867882.tmp testLoggerAdapter verbose: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.918Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: copied to local snapshot: bytes=71 testing.go:149: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.918Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.918Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: Installed remote snapshot testing.go:149: server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: 2021-08-25T09:31:07.918Z [INFO] server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593: Installed remote snapshot testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.919Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.919Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testLoggerAdapter verbose: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.924Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: copied to local snapshot: bytes=71 testing.go:149: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.924Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.924Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: Installed remote snapshot testing.go:149: server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: 2021-08-25T09:31:07.924Z [INFO] server-04723eb6-dbe6-d7cb-b272-204e75aebdd9: Installed remote snapshot testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.924Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.924Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: pipelining replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.940Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.940Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-04723eb6-dbe6-d7cb-b272-204e75aebdd9 04723eb6-dbe6-d7cb-b272-204e75aebdd9}" testLoggerAdapter verbose: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.940Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testing.go:149: server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: 2021-08-25T09:31:07.940Z [INFO] server-05a2a5ac-a42a-e5b1-b0c0-aec28ec89e12: aborting pipeline replication: peer="{Voter server-5eeecd91-d2d0-75a6-6b5e-0eb50cc34593 5eeecd91-d2d0-75a6-6b5e-0eb50cc34593}" testLoggerAdapter verbose: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:07.945Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20b8d2ed-ee98-699a-0849-2e2d2e99593c Address:20b8d2ed-ee98-699a-0849-2e2d2e99593c} {Suffrage:Voter ID:server-d6a10a2d-8aca-f825-8c38-b610fca47393 Address:d6a10a2d-8aca-f825-8c38-b610fca47393} {Suffrage:Voter ID:server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640 Address:cd97b6fe-27e4-7d70-faf2-cfaa191ae640}]" testing.go:149: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:07.945Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20b8d2ed-ee98-699a-0849-2e2d2e99593c Address:20b8d2ed-ee98-699a-0849-2e2d2e99593c} {Suffrage:Voter ID:server-d6a10a2d-8aca-f825-8c38-b610fca47393 Address:d6a10a2d-8aca-f825-8c38-b610fca47393} {Suffrage:Voter ID:server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640 Address:cd97b6fe-27e4-7d70-faf2-cfaa191ae640}]" testLoggerAdapter verbose: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:07.946Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: entering follower state: follower="Node at 20b8d2ed-ee98-699a-0849-2e2d2e99593c [Follower]" leader= testing.go:149: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:07.946Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: entering follower state: follower="Node at 20b8d2ed-ee98-699a-0849-2e2d2e99593c [Follower]" leader= testLoggerAdapter verbose: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:07.946Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20b8d2ed-ee98-699a-0849-2e2d2e99593c Address:20b8d2ed-ee98-699a-0849-2e2d2e99593c} {Suffrage:Voter ID:server-d6a10a2d-8aca-f825-8c38-b610fca47393 Address:d6a10a2d-8aca-f825-8c38-b610fca47393} {Suffrage:Voter ID:server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640 Address:cd97b6fe-27e4-7d70-faf2-cfaa191ae640}]" testing.go:149: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:07.946Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20b8d2ed-ee98-699a-0849-2e2d2e99593c Address:20b8d2ed-ee98-699a-0849-2e2d2e99593c} {Suffrage:Voter ID:server-d6a10a2d-8aca-f825-8c38-b610fca47393 Address:d6a10a2d-8aca-f825-8c38-b610fca47393} {Suffrage:Voter ID:server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640 Address:cd97b6fe-27e4-7d70-faf2-cfaa191ae640}]" testLoggerAdapter verbose: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:07.946Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: entering follower state: follower="Node at d6a10a2d-8aca-f825-8c38-b610fca47393 [Follower]" leader= testing.go:149: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:07.946Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: entering follower state: follower="Node at d6a10a2d-8aca-f825-8c38-b610fca47393 [Follower]" leader= testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:07.946Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20b8d2ed-ee98-699a-0849-2e2d2e99593c Address:20b8d2ed-ee98-699a-0849-2e2d2e99593c} {Suffrage:Voter ID:server-d6a10a2d-8aca-f825-8c38-b610fca47393 Address:d6a10a2d-8aca-f825-8c38-b610fca47393} {Suffrage:Voter ID:server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640 Address:cd97b6fe-27e4-7d70-faf2-cfaa191ae640}]" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:07.946Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20b8d2ed-ee98-699a-0849-2e2d2e99593c Address:20b8d2ed-ee98-699a-0849-2e2d2e99593c} {Suffrage:Voter ID:server-d6a10a2d-8aca-f825-8c38-b610fca47393 Address:d6a10a2d-8aca-f825-8c38-b610fca47393} {Suffrage:Voter ID:server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640 Address:cd97b6fe-27e4-7d70-faf2-cfaa191ae640}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:07.946Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:07.946Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:07.946Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: entering follower state: follower="Node at cd97b6fe-27e4-7d70-faf2-cfaa191ae640 [Follower]" leader= testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:07.946Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: entering follower state: follower="Node at cd97b6fe-27e4-7d70-faf2-cfaa191ae640 [Follower]" leader= testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [WARN] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [WARN] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: entering candidate state: node="Node at cd97b6fe-27e4-7d70-faf2-cfaa191ae640 [Candidate]" term=2 testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: entering candidate state: node="Node at cd97b6fe-27e4-7d70-faf2-cfaa191ae640 [Candidate]" term=2 testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: election won: tally=2 testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: election won: tally=2 testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: entering leader state: leader="Node at cd97b6fe-27e4-7d70-faf2-cfaa191ae640 [Leader]" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.036Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: entering leader state: leader="Node at cd97b6fe-27e4-7d70-faf2-cfaa191ae640 [Leader]" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: added peer, starting replication: peer=server-20b8d2ed-ee98-699a-0849-2e2d2e99593c testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: added peer, starting replication: peer=server-20b8d2ed-ee98-699a-0849-2e2d2e99593c testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: added peer, starting replication: peer=server-d6a10a2d-8aca-f825-8c38-b610fca47393 testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: added peer, starting replication: peer=server-d6a10a2d-8aca-f825-8c38-b610fca47393 testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.037Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.142Z [INFO] stable state for Leader reached at 2021-08-25 09:31:08.037062418 +0000 UTC m=+12.303278822 (1 nodes), 90.334438ms from start of poll, 91.532505ms from cluster start. Timeout at 2021-08-25 09:31:08.142202743 +0000 UTC m=+12.408419267, 105.140445ms after stability testing.go:149: cluster: 2021-08-25T09:31:08.142Z [INFO] stable state for Leader reached at 2021-08-25 09:31:08.037062418 +0000 UTC m=+12.303278822 (1 nodes), 90.334438ms from start of poll, 91.532505ms from cluster start. Timeout at 2021-08-25 09:31:08.142202743 +0000 UTC m=+12.408419267, 105.140445ms after stability testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.142Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: starting snapshot up to: index=12 testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.142Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: starting snapshot up to: index=12 testing.go:153: 2021-08-25T09:31:08.143Z [INFO] creating new snapshot: path=/tmp/raft237964417/snapshots/2-12-1629883868143.tmp testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.170Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: snapshot complete up to: index=12 testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.170Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: snapshot complete up to: index=12 testing.go:153: 2021-08-25T09:31:08.171Z [INFO] creating new snapshot: path=/tmp/raft237964417/snapshots/2-1013-1629883868171.tmp testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.199Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: copied to local snapshot: bytes=71 testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.199Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.199Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: restored user snapshot: index=1 testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.199Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: restored user snapshot: index=1 testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.200Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.208Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.208Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.209Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.209Z [ERROR] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:08.211Z [INFO] creating new snapshot: path=/tmp/raft987653629/snapshots/2-1013-1629883868211.tmp testing.go:153: 2021-08-25T09:31:08.211Z [INFO] creating new snapshot: path=/tmp/raft557900599/snapshots/2-1013-1629883868211.tmp testLoggerAdapter verbose: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:08.239Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:08.239Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: copied to local snapshot: bytes=71 testing.go:149: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:08.239Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: copied to local snapshot: bytes=71 testing.go:149: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:08.239Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:08.239Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: Installed remote snapshot testing.go:149: server-d6a10a2d-8aca-f825-8c38-b610fca47393: 2021-08-25T09:31:08.239Z [INFO] server-d6a10a2d-8aca-f825-8c38-b610fca47393: Installed remote snapshot testLoggerAdapter verbose: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:08.239Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: Installed remote snapshot testing.go:149: server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: 2021-08-25T09:31:08.239Z [INFO] server-20b8d2ed-ee98-699a-0849-2e2d2e99593c: Installed remote snapshot testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.239Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.239Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.239Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.239Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: pipelining replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.250Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.250Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-d6a10a2d-8aca-f825-8c38-b610fca47393 d6a10a2d-8aca-f825-8c38-b610fca47393}" testLoggerAdapter verbose: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.250Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testing.go:149: server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: 2021-08-25T09:31:08.250Z [INFO] server-cd97b6fe-27e4-7d70-faf2-cfaa191ae640: aborting pipeline replication: peer="{Voter server-20b8d2ed-ee98-699a-0849-2e2d2e99593c 20b8d2ed-ee98-699a-0849-2e2d2e99593c}" testLoggerAdapter verbose: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.255Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5622461-0627-5f6c-fe99-a4ba5a632a91 Address:d5622461-0627-5f6c-fe99-a4ba5a632a91} {Suffrage:Voter ID:server-d586e47d-e0a6-65fe-9392-2370d6d507e2 Address:d586e47d-e0a6-65fe-9392-2370d6d507e2} {Suffrage:Voter ID:server-c817f651-f295-15fe-623e-659e32c18c2c Address:c817f651-f295-15fe-623e-659e32c18c2c}]" testing.go:149: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.255Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5622461-0627-5f6c-fe99-a4ba5a632a91 Address:d5622461-0627-5f6c-fe99-a4ba5a632a91} {Suffrage:Voter ID:server-d586e47d-e0a6-65fe-9392-2370d6d507e2 Address:d586e47d-e0a6-65fe-9392-2370d6d507e2} {Suffrage:Voter ID:server-c817f651-f295-15fe-623e-659e32c18c2c Address:c817f651-f295-15fe-623e-659e32c18c2c}]" testLoggerAdapter verbose: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.255Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: entering follower state: follower="Node at d5622461-0627-5f6c-fe99-a4ba5a632a91 [Follower]" leader= testing.go:149: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.255Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: entering follower state: follower="Node at d5622461-0627-5f6c-fe99-a4ba5a632a91 [Follower]" leader= testLoggerAdapter verbose: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.256Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5622461-0627-5f6c-fe99-a4ba5a632a91 Address:d5622461-0627-5f6c-fe99-a4ba5a632a91} {Suffrage:Voter ID:server-d586e47d-e0a6-65fe-9392-2370d6d507e2 Address:d586e47d-e0a6-65fe-9392-2370d6d507e2} {Suffrage:Voter ID:server-c817f651-f295-15fe-623e-659e32c18c2c Address:c817f651-f295-15fe-623e-659e32c18c2c}]" testing.go:149: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.256Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5622461-0627-5f6c-fe99-a4ba5a632a91 Address:d5622461-0627-5f6c-fe99-a4ba5a632a91} {Suffrage:Voter ID:server-d586e47d-e0a6-65fe-9392-2370d6d507e2 Address:d586e47d-e0a6-65fe-9392-2370d6d507e2} {Suffrage:Voter ID:server-c817f651-f295-15fe-623e-659e32c18c2c Address:c817f651-f295-15fe-623e-659e32c18c2c}]" testLoggerAdapter verbose: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.256Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: entering follower state: follower="Node at d586e47d-e0a6-65fe-9392-2370d6d507e2 [Follower]" leader= testing.go:149: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.256Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: entering follower state: follower="Node at d586e47d-e0a6-65fe-9392-2370d6d507e2 [Follower]" leader= testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.256Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5622461-0627-5f6c-fe99-a4ba5a632a91 Address:d5622461-0627-5f6c-fe99-a4ba5a632a91} {Suffrage:Voter ID:server-d586e47d-e0a6-65fe-9392-2370d6d507e2 Address:d586e47d-e0a6-65fe-9392-2370d6d507e2} {Suffrage:Voter ID:server-c817f651-f295-15fe-623e-659e32c18c2c Address:c817f651-f295-15fe-623e-659e32c18c2c}]" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.256Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5622461-0627-5f6c-fe99-a4ba5a632a91 Address:d5622461-0627-5f6c-fe99-a4ba5a632a91} {Suffrage:Voter ID:server-d586e47d-e0a6-65fe-9392-2370d6d507e2 Address:d586e47d-e0a6-65fe-9392-2370d6d507e2} {Suffrage:Voter ID:server-c817f651-f295-15fe-623e-659e32c18c2c Address:c817f651-f295-15fe-623e-659e32c18c2c}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.256Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:08.256Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.256Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: entering follower state: follower="Node at c817f651-f295-15fe-623e-659e32c18c2c [Follower]" leader= testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.256Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: entering follower state: follower="Node at c817f651-f295-15fe-623e-659e32c18c2c [Follower]" leader= testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [WARN] server-c817f651-f295-15fe-623e-659e32c18c2c: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [WARN] server-c817f651-f295-15fe-623e-659e32c18c2c: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: entering candidate state: node="Node at c817f651-f295-15fe-623e-659e32c18c2c [Candidate]" term=2 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: entering candidate state: node="Node at c817f651-f295-15fe-623e-659e32c18c2c [Candidate]" term=2 testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: election won: tally=2 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: election won: tally=2 testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: entering leader state: leader="Node at c817f651-f295-15fe-623e-659e32c18c2c [Leader]" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: entering leader state: leader="Node at c817f651-f295-15fe-623e-659e32c18c2c [Leader]" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: added peer, starting replication: peer=server-d5622461-0627-5f6c-fe99-a4ba5a632a91 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: added peer, starting replication: peer=server-d5622461-0627-5f6c-fe99-a4ba5a632a91 testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: added peer, starting replication: peer=server-d586e47d-e0a6-65fe-9392-2370d6d507e2 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: added peer, starting replication: peer=server-d586e47d-e0a6-65fe-9392-2370d6d507e2 testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.320Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.425Z [INFO] stable state for Leader reached at 2021-08-25 09:31:08.320608152 +0000 UTC m=+12.586824536 (1 nodes), 64.116326ms from start of poll, 65.208043ms from cluster start. Timeout at 2021-08-25 09:31:08.425724898 +0000 UTC m=+12.691941282, 105.116746ms after stability testing.go:149: cluster: 2021-08-25T09:31:08.425Z [INFO] stable state for Leader reached at 2021-08-25 09:31:08.320608152 +0000 UTC m=+12.586824536 (1 nodes), 64.116326ms from start of poll, 65.208043ms from cluster start. Timeout at 2021-08-25 09:31:08.425724898 +0000 UTC m=+12.691941282, 105.116746ms after stability testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.426Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: starting snapshot up to: index=12 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.426Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: starting snapshot up to: index=12 testing.go:153: 2021-08-25T09:31:08.426Z [INFO] creating new snapshot: path=/tmp/raft569570239/snapshots/2-12-1629883868426.tmp testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.454Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: snapshot complete up to: index=12 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.454Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: snapshot complete up to: index=12 testing.go:153: 2021-08-25T09:31:08.455Z [INFO] creating new snapshot: path=/tmp/raft569570239/snapshots/2-10013-1629883868455.tmp testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.481Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: copied to local snapshot: bytes=71 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.481Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: restored user snapshot: index=1 testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: restored user snapshot: index=1 testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.482Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.488Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.488Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:08.489Z [INFO] creating new snapshot: path=/tmp/raft808114501/snapshots/2-10013-1629883868489.tmp testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.490Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.490Z [ERROR] server-c817f651-f295-15fe-623e-659e32c18c2c: failed to get log: index=23 error="log not found" testing.go:153: 2021-08-25T09:31:08.491Z [INFO] creating new snapshot: path=/tmp/raft866417243/snapshots/2-10013-1629883868491.tmp testLoggerAdapter verbose: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.532Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: copied to local snapshot: bytes=71 testing.go:149: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.532Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.532Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: Installed remote snapshot testing.go:149: server-d586e47d-e0a6-65fe-9392-2370d6d507e2: 2021-08-25T09:31:08.532Z [INFO] server-d586e47d-e0a6-65fe-9392-2370d6d507e2: Installed remote snapshot testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.532Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.532Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testLoggerAdapter verbose: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.537Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: copied to local snapshot: bytes=71 testing.go:149: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.537Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.538Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: Installed remote snapshot testing.go:149: server-d5622461-0627-5f6c-fe99-a4ba5a632a91: 2021-08-25T09:31:08.538Z [INFO] server-d5622461-0627-5f6c-fe99-a4ba5a632a91: Installed remote snapshot testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.538Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.538Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: pipelining replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.554Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.554Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d586e47d-e0a6-65fe-9392-2370d6d507e2 d586e47d-e0a6-65fe-9392-2370d6d507e2}" testLoggerAdapter verbose: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.554Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" testing.go:149: server-c817f651-f295-15fe-623e-659e32c18c2c: 2021-08-25T09:31:08.554Z [INFO] server-c817f651-f295-15fe-623e-659e32c18c2c: aborting pipeline replication: peer="{Voter server-d5622461-0627-5f6c-fe99-a4ba5a632a91 d5622461-0627-5f6c-fe99-a4ba5a632a91}" --- PASS: TestRaft_UserRestore (1.74s) === RUN TestRaft_SendSnapshotFollower testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.558Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-37119388-362e-973d-92b2-51d33844c523 Address:37119388-362e-973d-92b2-51d33844c523} {Suffrage:Voter ID:server-43fb39b5-4607-03b8-990a-01483cd126f6 Address:43fb39b5-4607-03b8-990a-01483cd126f6} {Suffrage:Voter ID:server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 Address:2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}]" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.558Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-37119388-362e-973d-92b2-51d33844c523 Address:37119388-362e-973d-92b2-51d33844c523} {Suffrage:Voter ID:server-43fb39b5-4607-03b8-990a-01483cd126f6 Address:43fb39b5-4607-03b8-990a-01483cd126f6} {Suffrage:Voter ID:server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 Address:2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}]" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.559Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: entering follower state: follower="Node at 37119388-362e-973d-92b2-51d33844c523 [Follower]" leader= testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.559Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: entering follower state: follower="Node at 37119388-362e-973d-92b2-51d33844c523 [Follower]" leader= testLoggerAdapter verbose: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.559Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-37119388-362e-973d-92b2-51d33844c523 Address:37119388-362e-973d-92b2-51d33844c523} {Suffrage:Voter ID:server-43fb39b5-4607-03b8-990a-01483cd126f6 Address:43fb39b5-4607-03b8-990a-01483cd126f6} {Suffrage:Voter ID:server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 Address:2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}]" testing.go:149: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.559Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-37119388-362e-973d-92b2-51d33844c523 Address:37119388-362e-973d-92b2-51d33844c523} {Suffrage:Voter ID:server-43fb39b5-4607-03b8-990a-01483cd126f6 Address:43fb39b5-4607-03b8-990a-01483cd126f6} {Suffrage:Voter ID:server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 Address:2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}]" testLoggerAdapter verbose: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.559Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: entering follower state: follower="Node at 43fb39b5-4607-03b8-990a-01483cd126f6 [Follower]" leader= testing.go:149: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.559Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: entering follower state: follower="Node at 43fb39b5-4607-03b8-990a-01483cd126f6 [Follower]" leader= testLoggerAdapter verbose: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.559Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-37119388-362e-973d-92b2-51d33844c523 Address:37119388-362e-973d-92b2-51d33844c523} {Suffrage:Voter ID:server-43fb39b5-4607-03b8-990a-01483cd126f6 Address:43fb39b5-4607-03b8-990a-01483cd126f6} {Suffrage:Voter ID:server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 Address:2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}]" testing.go:149: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.559Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-37119388-362e-973d-92b2-51d33844c523 Address:37119388-362e-973d-92b2-51d33844c523} {Suffrage:Voter ID:server-43fb39b5-4607-03b8-990a-01483cd126f6 Address:43fb39b5-4607-03b8-990a-01483cd126f6} {Suffrage:Voter ID:server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 Address:2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.559Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:08.559Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.559Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: entering follower state: follower="Node at 2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 [Follower]" leader= testing.go:149: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.559Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: entering follower state: follower="Node at 2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 [Follower]" leader= testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.611Z [WARN] server-37119388-362e-973d-92b2-51d33844c523: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.611Z [WARN] server-37119388-362e-973d-92b2-51d33844c523: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: entering candidate state: node="Node at 37119388-362e-973d-92b2-51d33844c523 [Candidate]" term=2 testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: entering candidate state: node="Node at 37119388-362e-973d-92b2-51d33844c523 [Candidate]" term=2 testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: election won: tally=2 testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: election won: tally=2 testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: entering leader state: leader="Node at 37119388-362e-973d-92b2-51d33844c523 [Leader]" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: entering leader state: leader="Node at 37119388-362e-973d-92b2-51d33844c523 [Leader]" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: added peer, starting replication: peer=server-43fb39b5-4607-03b8-990a-01483cd126f6 testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: added peer, starting replication: peer=server-43fb39b5-4607-03b8-990a-01483cd126f6 testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: added peer, starting replication: peer=server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: added peer, starting replication: peer=server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: pipelining replication: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: pipelining replication: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: pipelining replication: peer="{Voter server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.612Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: pipelining replication: peer="{Voter server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.717Z [INFO] stable state for Follower reached at 2021-08-25 09:31:08.612529877 +0000 UTC m=+12.878746301 (2 nodes), 52.804091ms from start of poll, 53.86739ms from cluster start. Timeout at 2021-08-25 09:31:08.717688099 +0000 UTC m=+12.983904543, 105.158242ms after stability testing.go:149: cluster: 2021-08-25T09:31:08.717Z [INFO] stable state for Follower reached at 2021-08-25 09:31:08.612529877 +0000 UTC m=+12.878746301 (2 nodes), 52.804091ms from start of poll, 53.86739ms from cluster start. Timeout at 2021-08-25 09:31:08.717688099 +0000 UTC m=+12.983904543, 105.158242ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.717Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:08.717Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.823Z [INFO] stable state for Leader reached at 2021-08-25 09:31:08.717968653 +0000 UTC m=+12.984185057 (1 nodes), 2.6µs from start of poll, 159.306146ms from cluster start. Timeout at 2021-08-25 09:31:08.823140535 +0000 UTC m=+13.089357019, 105.171962ms after stability testing.go:149: cluster: 2021-08-25T09:31:08.823Z [INFO] stable state for Leader reached at 2021-08-25 09:31:08.717968653 +0000 UTC m=+12.984185057 (1 nodes), 2.6µs from start of poll, 159.306146ms from cluster start. Timeout at 2021-08-25 09:31:08.823140535 +0000 UTC m=+13.089357019, 105.171962ms after stability testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.823Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to pipeline appendEntries: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="append pipeline closed" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.823Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to pipeline appendEntries: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="append pipeline closed" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.823Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: aborting pipeline replication: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.823Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: aborting pipeline replication: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.823Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to appendEntries to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.823Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to appendEntries to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.826Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to heartbeat to: peer=43fb39b5-4607-03b8-990a-01483cd126f6 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.826Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to heartbeat to: peer=43fb39b5-4607-03b8-990a-01483cd126f6 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" raft_test.go:1320: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.826Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: starting snapshot up to: index=102 testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.826Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: starting snapshot up to: index=102 testing.go:153: 2021-08-25T09:31:08.827Z [INFO] creating new snapshot: path=/tmp/raft765819977/snapshots/2-102-1629883868827.tmp testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.834Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to appendEntries to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.834Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to appendEntries to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.844Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to heartbeat to: peer=43fb39b5-4607-03b8-990a-01483cd126f6 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.844Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to heartbeat to: peer=43fb39b5-4607-03b8-990a-01483cd126f6 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.844Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to appendEntries to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.844Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to appendEntries to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.855Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: compacting logs: from=1 to=92 testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.855Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.855Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: snapshot complete up to: index=102 testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.855Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: snapshot complete up to: index=102 testLoggerAdapter verbose: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.855Z [ERROR] server-43fb39b5-4607-03b8-990a-01483cd126f6: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.855Z [ERROR] server-43fb39b5-4607-03b8-990a-01483cd126f6: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.855Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: starting snapshot up to: index=102 testing.go:149: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.855Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: starting snapshot up to: index=102 testing.go:153: 2021-08-25T09:31:08.855Z [INFO] creating new snapshot: path=/tmp/raft198525837/snapshots/2-102-1629883868855.tmp testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.861Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to heartbeat to: peer=43fb39b5-4607-03b8-990a-01483cd126f6 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.861Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to heartbeat to: peer=43fb39b5-4607-03b8-990a-01483cd126f6 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.870Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to get log: index=2 error="log not found" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.870Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to get log: index=2 error="log not found" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.871Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to install snapshot: id=2-102-1629883868827 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.871Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to install snapshot: id=2-102-1629883868827 error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.871Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to send snapshot to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.871Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to send snapshot to: peer="{Voter server-43fb39b5-4607-03b8-990a-01483cd126f6 43fb39b5-4607-03b8-990a-01483cd126f6}" error="failed to connect to peer: 43fb39b5-4607-03b8-990a-01483cd126f6" testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.872Z [WARN] server-37119388-362e-973d-92b2-51d33844c523: failed to contact: server-id=server-43fb39b5-4607-03b8-990a-01483cd126f6 time=50.136182ms testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.872Z [WARN] server-37119388-362e-973d-92b2-51d33844c523: failed to contact: server-id=server-43fb39b5-4607-03b8-990a-01483cd126f6 time=50.136182ms testLoggerAdapter verbose: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.882Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: compacting logs: from=1 to=92 testing.go:149: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.882Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.882Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: snapshot complete up to: index=102 testing.go:149: server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: 2021-08-25T09:31:08.882Z [INFO] server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6: snapshot complete up to: index=102 testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.920Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to get log: index=2 error="log not found" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.920Z [ERROR] server-37119388-362e-973d-92b2-51d33844c523: failed to get log: index=2 error="log not found" testing.go:153: 2021-08-25T09:31:08.920Z [INFO] creating new snapshot: path=/tmp/raft106233187/snapshots/2-102-1629883868920.tmp testLoggerAdapter verbose: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.948Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: copied to local snapshot: bytes=693 testing.go:149: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.948Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: copied to local snapshot: bytes=693 testLoggerAdapter verbose: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.948Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: Installed remote snapshot testing.go:149: server-43fb39b5-4607-03b8-990a-01483cd126f6: 2021-08-25T09:31:08.948Z [INFO] server-43fb39b5-4607-03b8-990a-01483cd126f6: Installed remote snapshot testLoggerAdapter verbose: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.950Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: aborting pipeline replication: peer="{Voter server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}" testing.go:149: server-37119388-362e-973d-92b2-51d33844c523: 2021-08-25T09:31:08.950Z [INFO] server-37119388-362e-973d-92b2-51d33844c523: aborting pipeline replication: peer="{Voter server-2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6 2d1ecf8b-587d-b06c-8ad7-83c880ebd5c6}" --- PASS: TestRaft_SendSnapshotFollower (0.40s) === RUN TestRaft_SendSnapshotAndLogsFollower testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:08.955Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e Address:dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e} {Suffrage:Voter ID:server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 Address:e677fbf9-c92d-40a1-2d87-ba39ab43b829} {Suffrage:Voter ID:server-540075b3-bb51-75bb-89b6-177b8d6d1b82 Address:540075b3-bb51-75bb-89b6-177b8d6d1b82}]" testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:08.955Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e Address:dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e} {Suffrage:Voter ID:server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 Address:e677fbf9-c92d-40a1-2d87-ba39ab43b829} {Suffrage:Voter ID:server-540075b3-bb51-75bb-89b6-177b8d6d1b82 Address:540075b3-bb51-75bb-89b6-177b8d6d1b82}]" testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:08.955Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering follower state: follower="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Follower]" leader= testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:08.955Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering follower state: follower="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Follower]" leader= testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:08.955Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e Address:dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e} {Suffrage:Voter ID:server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 Address:e677fbf9-c92d-40a1-2d87-ba39ab43b829} {Suffrage:Voter ID:server-540075b3-bb51-75bb-89b6-177b8d6d1b82 Address:540075b3-bb51-75bb-89b6-177b8d6d1b82}]" testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:08.955Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e Address:dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e} {Suffrage:Voter ID:server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 Address:e677fbf9-c92d-40a1-2d87-ba39ab43b829} {Suffrage:Voter ID:server-540075b3-bb51-75bb-89b6-177b8d6d1b82 Address:540075b3-bb51-75bb-89b6-177b8d6d1b82}]" testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:08.956Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: entering follower state: follower="Node at e677fbf9-c92d-40a1-2d87-ba39ab43b829 [Follower]" leader= testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:08.956Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: entering follower state: follower="Node at e677fbf9-c92d-40a1-2d87-ba39ab43b829 [Follower]" leader= testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:08.956Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e Address:dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e} {Suffrage:Voter ID:server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 Address:e677fbf9-c92d-40a1-2d87-ba39ab43b829} {Suffrage:Voter ID:server-540075b3-bb51-75bb-89b6-177b8d6d1b82 Address:540075b3-bb51-75bb-89b6-177b8d6d1b82}]" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:08.956Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e Address:dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e} {Suffrage:Voter ID:server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 Address:e677fbf9-c92d-40a1-2d87-ba39ab43b829} {Suffrage:Voter ID:server-540075b3-bb51-75bb-89b6-177b8d6d1b82 Address:540075b3-bb51-75bb-89b6-177b8d6d1b82}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:08.956Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:08.956Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:08.956Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering follower state: follower="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Follower]" leader= testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:08.956Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering follower state: follower="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Follower]" leader= testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.009Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.009Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering candidate state: node="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Candidate]" term=2 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering candidate state: node="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Candidate]" term=2 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: election won: tally=2 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: election won: tally=2 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering leader state: leader="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Leader]" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering leader state: leader="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Leader]" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.010Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.115Z [INFO] stable state for Follower reached at 2021-08-25 09:31:09.010321778 +0000 UTC m=+13.276538182 (2 nodes), 53.907907ms from start of poll, 55.070697ms from cluster start. Timeout at 2021-08-25 09:31:09.115422526 +0000 UTC m=+13.381638950, 105.100768ms after stability testing.go:149: cluster: 2021-08-25T09:31:09.115Z [INFO] stable state for Follower reached at 2021-08-25 09:31:09.010321778 +0000 UTC m=+13.276538182 (2 nodes), 53.907907ms from start of poll, 55.070697ms from cluster start. Timeout at 2021-08-25 09:31:09.115422526 +0000 UTC m=+13.381638950, 105.100768ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.115Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:09.115Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.220Z [INFO] stable state for Leader reached at 2021-08-25 09:31:09.115687281 +0000 UTC m=+13.381903665 (1 nodes), 2.22µs from start of poll, 160.43618ms from cluster start. Timeout at 2021-08-25 09:31:09.220813007 +0000 UTC m=+13.487029491, 105.125826ms after stability testing.go:149: cluster: 2021-08-25T09:31:09.220Z [INFO] stable state for Leader reached at 2021-08-25 09:31:09.115687281 +0000 UTC m=+13.381903665 (1 nodes), 2.22µs from start of poll, 160.43618ms from cluster start. Timeout at 2021-08-25 09:31:09.220813007 +0000 UTC m=+13.487029491, 105.125826ms after stability testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.221Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to pipeline appendEntries: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="append pipeline closed" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.221Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to pipeline appendEntries: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="append pipeline closed" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.221Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.221Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.221Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.221Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.224Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to heartbeat to: peer=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.224Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to heartbeat to: peer=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" raft_test.go:1362: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.224Z [ERROR] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.224Z [ERROR] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.224Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: starting snapshot up to: index=101 testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.224Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: starting snapshot up to: index=101 testing.go:153: 2021-08-25T09:31:09.224Z [INFO] creating new snapshot: path=/tmp/raft428650257/snapshots/2-101-1629883869224.tmp testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.231Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.231Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.240Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to heartbeat to: peer=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.240Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to heartbeat to: peer=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.242Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.242Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.252Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: compacting logs: from=1 to=92 testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.252Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.253Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: snapshot complete up to: index=101 testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.253Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: snapshot complete up to: index=101 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.253Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: starting snapshot up to: index=102 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.253Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: starting snapshot up to: index=102 testing.go:153: 2021-08-25T09:31:09.253Z [INFO] creating new snapshot: path=/tmp/raft962829675/snapshots/2-102-1629883869253.tmp testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.260Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to heartbeat to: peer=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.260Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to heartbeat to: peer=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.271Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to contact: server-id=server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e time=50.116143ms testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.271Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to contact: server-id=server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e time=50.116143ms testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.271Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.271Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to appendEntries to: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" error="failed to connect to peer: dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e" testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.275Z [WARN] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: heartbeat timeout reached, starting election: last-leader=540075b3-bb51-75bb-89b6-177b8d6d1b82 testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.275Z [WARN] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: heartbeat timeout reached, starting election: last-leader=540075b3-bb51-75bb-89b6-177b8d6d1b82 testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.275Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering candidate state: node="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Candidate]" term=3 testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.275Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering candidate state: node="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Candidate]" term=3 testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.276Z [ERROR] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to make requestVote RPC: target="{Voter server-540075b3-bb51-75bb-89b6-177b8d6d1b82 540075b3-bb51-75bb-89b6-177b8d6d1b82}" error="failed to connect to peer: 540075b3-bb51-75bb-89b6-177b8d6d1b82" testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.276Z [ERROR] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to make requestVote RPC: target="{Voter server-540075b3-bb51-75bb-89b6-177b8d6d1b82 540075b3-bb51-75bb-89b6-177b8d6d1b82}" error="failed to connect to peer: 540075b3-bb51-75bb-89b6-177b8d6d1b82" testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.276Z [ERROR] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to make requestVote RPC: target="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" error="failed to connect to peer: e677fbf9-c92d-40a1-2d87-ba39ab43b829" testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.276Z [ERROR] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to make requestVote RPC: target="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" error="failed to connect to peer: e677fbf9-c92d-40a1-2d87-ba39ab43b829" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.280Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: compacting logs: from=1 to=92 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.280Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.280Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: snapshot complete up to: index=102 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.280Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: snapshot complete up to: index=102 raft_test.go:1383: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.320Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to get log: index=2 error="log not found" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.320Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to get log: index=2 error="log not found" testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.321Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: ignoring installSnapshot request with older term than current term: request-term=2 current-term=3 testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.321Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: ignoring installSnapshot request with older term than current term: request-term=2 current-term=3 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.321Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: peer has newer term, stopping replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.321Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: peer has newer term, stopping replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.321Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering follower state: follower="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Follower]" leader= testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.321Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering follower state: follower="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Follower]" leader= testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.321Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.321Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.374Z [WARN] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: Election timeout reached, restarting election testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.374Z [WARN] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: Election timeout reached, restarting election testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.374Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering candidate state: node="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Candidate]" term=4 testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.374Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering candidate state: node="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Candidate]" term=4 testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.374Z [WARN] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: rejecting vote request since we have a leader: from=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e leader=540075b3-bb51-75bb-89b6-177b8d6d1b82 testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.374Z [WARN] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: rejecting vote request since we have a leader: from=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e leader=540075b3-bb51-75bb-89b6-177b8d6d1b82 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.374Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: rejecting vote request since our last index is greater: candidate=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e last-index=202 last-candidate-index=2 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.374Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: rejecting vote request since our last index is greater: candidate=dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e last-index=202 last-candidate-index=2 testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.382Z [WARN] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: heartbeat timeout reached, starting election: last-leader=540075b3-bb51-75bb-89b6-177b8d6d1b82 testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.382Z [WARN] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: heartbeat timeout reached, starting election: last-leader=540075b3-bb51-75bb-89b6-177b8d6d1b82 testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.382Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: entering candidate state: node="Node at e677fbf9-c92d-40a1-2d87-ba39ab43b829 [Candidate]" term=3 testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.382Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: entering candidate state: node="Node at e677fbf9-c92d-40a1-2d87-ba39ab43b829 [Candidate]" term=3 testLoggerAdapter verbose: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.382Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: entering follower state: follower="Node at e677fbf9-c92d-40a1-2d87-ba39ab43b829 [Follower]" leader= testing.go:149: server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: 2021-08-25T09:31:09.382Z [INFO] server-e677fbf9-c92d-40a1-2d87-ba39ab43b829: entering follower state: follower="Node at e677fbf9-c92d-40a1-2d87-ba39ab43b829 [Follower]" leader= testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering candidate state: node="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Candidate]" term=5 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering candidate state: node="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Candidate]" term=5 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: election won: tally=2 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: election won: tally=2 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering leader state: leader="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Leader]" testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.399Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering follower state: follower="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Follower]" leader= testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: entering leader state: leader="Node at 540075b3-bb51-75bb-89b6-177b8d6d1b82 [Leader]" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: added peer, starting replication: peer=server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.399Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: entering follower state: follower="Node at dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e [Follower]" leader= testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.399Z [WARN] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to get previous log: previous-index=202 last-index=2 error="log not found" testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.399Z [WARN] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: failed to get previous log: previous-index=202 last-index=2 error="log not found" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: appendEntries rejected, sending older logs: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" next=3 testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.399Z [WARN] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: appendEntries rejected, sending older logs: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" next=3 testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.400Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to get log: index=2 error="log not found" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.400Z [ERROR] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: failed to get log: index=2 error="log not found" testing.go:153: 2021-08-25T09:31:09.400Z [INFO] creating new snapshot: path=/tmp/raft562394951/snapshots/2-102-1629883869400.tmp testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.428Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: copied to local snapshot: bytes=693 testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.428Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: copied to local snapshot: bytes=693 testLoggerAdapter verbose: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.429Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: Installed remote snapshot testing.go:149: server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: 2021-08-25T09:31:09.429Z [INFO] server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e: Installed remote snapshot testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.429Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.429Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: pipelining replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.432Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.432Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-e677fbf9-c92d-40a1-2d87-ba39ab43b829 e677fbf9-c92d-40a1-2d87-ba39ab43b829}" testLoggerAdapter verbose: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.432Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" testing.go:149: server-540075b3-bb51-75bb-89b6-177b8d6d1b82: 2021-08-25T09:31:09.432Z [INFO] server-540075b3-bb51-75bb-89b6-177b8d6d1b82: aborting pipeline replication: peer="{Voter server-dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e dbc5d3cd-12df-fe6d-a292-6a7bb5590a2e}" --- PASS: TestRaft_SendSnapshotAndLogsFollower (0.48s) === RUN TestRaft_ReJoinFollower testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.438Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3} {Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.438Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3} {Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.438Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering follower state: follower="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Follower]" leader= testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.438Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering follower state: follower="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Follower]" leader= testLoggerAdapter verbose: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.438Z [INFO] server-081c4e6a-550d-289e-5f10-021ed3b72622: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3} {Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testing.go:149: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.438Z [INFO] server-081c4e6a-550d-289e-5f10-021ed3b72622: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3} {Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testLoggerAdapter verbose: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.438Z [INFO] server-081c4e6a-550d-289e-5f10-021ed3b72622: entering follower state: follower="Node at 081c4e6a-550d-289e-5f10-021ed3b72622 [Follower]" leader= testing.go:149: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.438Z [INFO] server-081c4e6a-550d-289e-5f10-021ed3b72622: entering follower state: follower="Node at 081c4e6a-550d-289e-5f10-021ed3b72622 [Follower]" leader= testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.438Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3} {Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.438Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3} {Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.439Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:09.439Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.439Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering follower state: follower="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Follower]" leader= testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.439Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering follower state: follower="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Follower]" leader= testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering candidate state: node="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Candidate]" term=2 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering candidate state: node="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Candidate]" term=2 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: election won: tally=2 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: election won: tally=2 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering leader state: leader="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Leader]" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering leader state: leader="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Leader]" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-081c4e6a-550d-289e-5f10-021ed3b72622 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-081c4e6a-550d-289e-5f10-021ed3b72622 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.507Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.508Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.508Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.613Z [INFO] stable state for Leader reached at 2021-08-25 09:31:09.507786178 +0000 UTC m=+13.774002582 (1 nodes), 68.635161ms from start of poll, 69.834288ms from cluster start. Timeout at 2021-08-25 09:31:09.612911584 +0000 UTC m=+13.879128048, 105.125466ms after stability testing.go:149: cluster: 2021-08-25T09:31:09.613Z [INFO] stable state for Leader reached at 2021-08-25 09:31:09.507786178 +0000 UTC m=+13.774002582 (1 nodes), 68.635161ms from start of poll, 69.834288ms from cluster start. Timeout at 2021-08-25 09:31:09.612911584 +0000 UTC m=+13.879128048, 105.125466ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.618Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:09.618Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.723Z [INFO] stable state for Follower reached at 2021-08-25 09:31:09.618437124 +0000 UTC m=+13.884653508 (2 nodes), 3.06µs from start of poll, 180.485214ms from cluster start. Timeout at 2021-08-25 09:31:09.723511754 +0000 UTC m=+13.989728238, 105.07473ms after stability testing.go:149: cluster: 2021-08-25T09:31:09.723Z [INFO] stable state for Follower reached at 2021-08-25 09:31:09.618437124 +0000 UTC m=+13.884653508 (2 nodes), 3.06µs from start of poll, 180.485214ms from cluster start. Timeout at 2021-08-25 09:31:09.723511754 +0000 UTC m=+13.989728238, 105.07473ms after stability testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.723Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: updating configuration: command=RemoveServer server-id=server-ded9c8bd-6279-0276-15e8-13f587309de3 server-addr= servers="[{Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.723Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: updating configuration: command=RemoveServer server-id=server-ded9c8bd-6279-0276-15e8-13f587309de3 server-addr= servers="[{Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881}]" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.724Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: removed peer, stopping replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 last-index=3 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.724Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: removed peer, stopping replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 last-index=3 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.724Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.724Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.786Z [WARN] server-ded9c8bd-6279-0276-15e8-13f587309de3: heartbeat timeout reached, starting election: last-leader=bc1116fe-ae2a-50da-2733-45068030f881 testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.786Z [WARN] server-ded9c8bd-6279-0276-15e8-13f587309de3: heartbeat timeout reached, starting election: last-leader=bc1116fe-ae2a-50da-2733-45068030f881 testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.787Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering candidate state: node="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Candidate]" term=3 testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.787Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering candidate state: node="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Candidate]" term=3 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.787Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: rejecting vote request since we have a leader: from=ded9c8bd-6279-0276-15e8-13f587309de3 leader=bc1116fe-ae2a-50da-2733-45068030f881 testLoggerAdapter verbose: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.787Z [WARN] server-081c4e6a-550d-289e-5f10-021ed3b72622: rejecting vote request since we have a leader: from=ded9c8bd-6279-0276-15e8-13f587309de3 leader=bc1116fe-ae2a-50da-2733-45068030f881 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.787Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: rejecting vote request since we have a leader: from=ded9c8bd-6279-0276-15e8-13f587309de3 leader=bc1116fe-ae2a-50da-2733-45068030f881 testing.go:149: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.787Z [WARN] server-081c4e6a-550d-289e-5f10-021ed3b72622: rejecting vote request since we have a leader: from=ded9c8bd-6279-0276-15e8-13f587309de3 leader=bc1116fe-ae2a-50da-2733-45068030f881 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.829Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: updating configuration: command=AddStaging server-id=server-ded9c8bd-6279-0276-15e8-13f587309de3 server-addr=ded9c8bd-6279-0276-15e8-13f587309de3 servers="[{Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881} {Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3}]" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.829Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: updating configuration: command=AddStaging server-id=server-ded9c8bd-6279-0276-15e8-13f587309de3 server-addr=ded9c8bd-6279-0276-15e8-13f587309de3 servers="[{Suffrage:Voter ID:server-081c4e6a-550d-289e-5f10-021ed3b72622 Address:081c4e6a-550d-289e-5f10-021ed3b72622} {Suffrage:Voter ID:server-bc1116fe-ae2a-50da-2733-45068030f881 Address:bc1116fe-ae2a-50da-2733-45068030f881} {Suffrage:Voter ID:server-ded9c8bd-6279-0276-15e8-13f587309de3 Address:ded9c8bd-6279-0276-15e8-13f587309de3}]" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.829Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.829Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.829Z [ERROR] server-bc1116fe-ae2a-50da-2733-45068030f881: peer has newer term, stopping replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.829Z [ERROR] server-bc1116fe-ae2a-50da-2733-45068030f881: peer has newer term, stopping replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.830Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering follower state: follower="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Follower]" leader= testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.830Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering follower state: follower="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Follower]" leader= testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.830Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.830Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.830Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" testing.go:149: cluster: 2021-08-25T09:31:09.830Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.842Z [WARN] server-ded9c8bd-6279-0276-15e8-13f587309de3: Election timeout reached, restarting election testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.842Z [WARN] server-ded9c8bd-6279-0276-15e8-13f587309de3: Election timeout reached, restarting election testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.842Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering candidate state: node="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Candidate]" term=4 testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.842Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering candidate state: node="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Candidate]" term=4 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.842Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: rejecting vote request since our last index is greater: candidate=ded9c8bd-6279-0276-15e8-13f587309de3 last-index=4 last-candidate-index=3 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.842Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: rejecting vote request since our last index is greater: candidate=ded9c8bd-6279-0276-15e8-13f587309de3 last-index=4 last-candidate-index=3 testLoggerAdapter verbose: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.842Z [WARN] server-081c4e6a-550d-289e-5f10-021ed3b72622: rejecting vote request since we have a leader: from=ded9c8bd-6279-0276-15e8-13f587309de3 leader=bc1116fe-ae2a-50da-2733-45068030f881 testing.go:149: server-081c4e6a-550d-289e-5f10-021ed3b72622: 2021-08-25T09:31:09.842Z [WARN] server-081c4e6a-550d-289e-5f10-021ed3b72622: rejecting vote request since we have a leader: from=ded9c8bd-6279-0276-15e8-13f587309de3 leader=bc1116fe-ae2a-50da-2733-45068030f881 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering candidate state: node="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Candidate]" term=5 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering candidate state: node="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Candidate]" term=5 testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.889Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering follower state: follower="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Follower]" leader= testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.889Z [INFO] server-ded9c8bd-6279-0276-15e8-13f587309de3: entering follower state: follower="Node at ded9c8bd-6279-0276-15e8-13f587309de3 [Follower]" leader= testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: election won: tally=2 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: election won: tally=2 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering leader state: leader="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Leader]" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: entering leader state: leader="Node at bc1116fe-ae2a-50da-2733-45068030f881 [Leader]" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-081c4e6a-550d-289e-5f10-021ed3b72622 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-081c4e6a-550d-289e-5f10-021ed3b72622 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: added peer, starting replication: peer=server-ded9c8bd-6279-0276-15e8-13f587309de3 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.889Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" testLoggerAdapter verbose: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.889Z [WARN] server-ded9c8bd-6279-0276-15e8-13f587309de3: failed to get previous log: previous-index=4 last-index=3 error="log not found" testing.go:149: server-ded9c8bd-6279-0276-15e8-13f587309de3: 2021-08-25T09:31:09.889Z [WARN] server-ded9c8bd-6279-0276-15e8-13f587309de3: failed to get previous log: previous-index=4 last-index=3 error="log not found" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.890Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: appendEntries rejected, sending older logs: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" next=4 testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.890Z [WARN] server-bc1116fe-ae2a-50da-2733-45068030f881: appendEntries rejected, sending older logs: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" next=4 testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.890Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.890Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: pipelining replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.994Z [INFO] stable state for Leader reached at 2021-08-25 09:31:09.88973413 +0000 UTC m=+14.155950554 (1 nodes), 59.518618ms from start of poll, 451.78226ms from cluster start. Timeout at 2021-08-25 09:31:09.994760565 +0000 UTC m=+14.260976969, 105.026415ms after stability testing.go:149: cluster: 2021-08-25T09:31:09.994Z [INFO] stable state for Leader reached at 2021-08-25 09:31:09.88973413 +0000 UTC m=+14.155950554 (1 nodes), 59.518618ms from start of poll, 451.78226ms from cluster start. Timeout at 2021-08-25 09:31:09.994760565 +0000 UTC m=+14.260976969, 105.026415ms after stability testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.995Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.995Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-ded9c8bd-6279-0276-15e8-13f587309de3 ded9c8bd-6279-0276-15e8-13f587309de3}" testLoggerAdapter verbose: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.995Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" testing.go:149: server-bc1116fe-ae2a-50da-2733-45068030f881: 2021-08-25T09:31:09.995Z [INFO] server-bc1116fe-ae2a-50da-2733-45068030f881: aborting pipeline replication: peer="{Voter server-081c4e6a-550d-289e-5f10-021ed3b72622 081c4e6a-550d-289e-5f10-021ed3b72622}" --- PASS: TestRaft_ReJoinFollower (0.56s) === RUN TestRaft_LeaderLeaseExpire testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:09.998Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-88830c53-3729-34c0-46cc-7675dcddee4a Address:88830c53-3729-34c0-46cc-7675dcddee4a} {Suffrage:Voter ID:server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a Address:aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}]" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:09.998Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-88830c53-3729-34c0-46cc-7675dcddee4a Address:88830c53-3729-34c0-46cc-7675dcddee4a} {Suffrage:Voter ID:server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a Address:aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}]" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:09.998Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering follower state: follower="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Follower]" leader= testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:09.998Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering follower state: follower="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Follower]" leader= testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:09.998Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-88830c53-3729-34c0-46cc-7675dcddee4a Address:88830c53-3729-34c0-46cc-7675dcddee4a} {Suffrage:Voter ID:server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a Address:aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}]" testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:09.998Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-88830c53-3729-34c0-46cc-7675dcddee4a Address:88830c53-3729-34c0-46cc-7675dcddee4a} {Suffrage:Voter ID:server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a Address:aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:09.998Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:09.998Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:09.998Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering follower state: follower="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Follower]" leader= testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:09.998Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering follower state: follower="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Follower]" leader= testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.064Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.064Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering candidate state: node="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Candidate]" term=2 testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering candidate state: node="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Candidate]" term=2 testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: election won: tally=2 testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: election won: tally=2 testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering leader state: leader="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Leader]" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering leader state: leader="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Leader]" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: added peer, starting replication: peer=server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: added peer, starting replication: peer=server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: pipelining replication: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.065Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: pipelining replication: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.170Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.06557974 +0000 UTC m=+14.331796164 (1 nodes), 66.7231ms from start of poll, 67.446332ms from cluster start. Timeout at 2021-08-25 09:31:10.170704706 +0000 UTC m=+14.436921190, 105.125026ms after stability testing.go:149: cluster: 2021-08-25T09:31:10.170Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.06557974 +0000 UTC m=+14.331796164 (1 nodes), 66.7231ms from start of poll, 67.446332ms from cluster start. Timeout at 2021-08-25 09:31:10.170704706 +0000 UTC m=+14.436921190, 105.125026ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.176Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:10.176Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.281Z [INFO] stable state for Follower reached at 2021-08-25 09:31:10.176285781 +0000 UTC m=+14.442502165 (1 nodes), 2.04µs from start of poll, 178.152333ms from cluster start. Timeout at 2021-08-25 09:31:10.281413786 +0000 UTC m=+14.547630270, 105.128105ms after stability testing.go:149: cluster: 2021-08-25T09:31:10.281Z [INFO] stable state for Follower reached at 2021-08-25 09:31:10.176285781 +0000 UTC m=+14.442502165 (1 nodes), 2.04µs from start of poll, 178.152333ms from cluster start. Timeout at 2021-08-25 09:31:10.281413786 +0000 UTC m=+14.547630270, 105.128105ms after stability raft_test.go:1492: [INFO] Disconnecting Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Follower] testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.287Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to pipeline appendEntries: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="append pipeline closed" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.287Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to pipeline appendEntries: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="append pipeline closed" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.288Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: aborting pipeline replication: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.288Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: aborting pipeline replication: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.291Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to heartbeat to: peer=aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.291Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to heartbeat to: peer=aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.295Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to appendEntries to: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.295Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to appendEntries to: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.310Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to heartbeat to: peer=aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.310Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to heartbeat to: peer=aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.312Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to appendEntries to: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.312Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to appendEntries to: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.327Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to heartbeat to: peer=aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.327Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to heartbeat to: peer=aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.328Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to appendEntries to: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.328Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to appendEntries to: peer="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.331Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to contact: server-id=server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a time=50.056129ms testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.331Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to contact: server-id=server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a time=50.056129ms testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.331Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to contact quorum of nodes, stepping down testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.331Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.331Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering follower state: follower="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Follower]" leader= testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.331Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering follower state: follower="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Follower]" leader= testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.331Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:10.331Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.378Z [WARN] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: heartbeat timeout reached, starting election: last-leader=88830c53-3729-34c0-46cc-7675dcddee4a testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.378Z [WARN] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: heartbeat timeout reached, starting election: last-leader=88830c53-3729-34c0-46cc-7675dcddee4a testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.378Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering candidate state: node="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Candidate]" term=3 testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.378Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering candidate state: node="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Candidate]" term=3 testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.378Z [ERROR] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: failed to make requestVote RPC: target="{Voter server-88830c53-3729-34c0-46cc-7675dcddee4a 88830c53-3729-34c0-46cc-7675dcddee4a}" error="failed to connect to peer: 88830c53-3729-34c0-46cc-7675dcddee4a" testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.378Z [ERROR] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: failed to make requestVote RPC: target="{Voter server-88830c53-3729-34c0-46cc-7675dcddee4a 88830c53-3729-34c0-46cc-7675dcddee4a}" error="failed to connect to peer: 88830c53-3729-34c0-46cc-7675dcddee4a" testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.416Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.416Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.416Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering candidate state: node="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Candidate]" term=3 testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.416Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering candidate state: node="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Candidate]" term=3 testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.416Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to make requestVote RPC: target="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.416Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to make requestVote RPC: target="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.468Z [WARN] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: Election timeout reached, restarting election testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.468Z [WARN] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: Election timeout reached, restarting election testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.468Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering candidate state: node="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Candidate]" term=4 testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.468Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering candidate state: node="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Candidate]" term=4 testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.468Z [ERROR] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: failed to make requestVote RPC: target="{Voter server-88830c53-3729-34c0-46cc-7675dcddee4a 88830c53-3729-34c0-46cc-7675dcddee4a}" error="failed to connect to peer: 88830c53-3729-34c0-46cc-7675dcddee4a" testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.468Z [ERROR] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: failed to make requestVote RPC: target="{Voter server-88830c53-3729-34c0-46cc-7675dcddee4a 88830c53-3729-34c0-46cc-7675dcddee4a}" error="failed to connect to peer: 88830c53-3729-34c0-46cc-7675dcddee4a" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.483Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.37870431 +0000 UTC m=+14.644920714 (0 nodes), 46.739141ms from start of poll, 380.570882ms from cluster start. Timeout at 2021-08-25 09:31:10.483868512 +0000 UTC m=+14.750084996, 105.164282ms after stability testing.go:149: cluster: 2021-08-25T09:31:10.483Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.37870431 +0000 UTC m=+14.644920714 (0 nodes), 46.739141ms from start of poll, 380.570882ms from cluster start. Timeout at 2021-08-25 09:31:10.483868512 +0000 UTC m=+14.750084996, 105.164282ms after stability testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.510Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: Election timeout reached, restarting election testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.510Z [WARN] server-88830c53-3729-34c0-46cc-7675dcddee4a: Election timeout reached, restarting election testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.510Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering candidate state: node="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Candidate]" term=4 testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.510Z [INFO] server-88830c53-3729-34c0-46cc-7675dcddee4a: entering candidate state: node="Node at 88830c53-3729-34c0-46cc-7675dcddee4a [Candidate]" term=4 testLoggerAdapter verbose: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.511Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to make requestVote RPC: target="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testing.go:149: server-88830c53-3729-34c0-46cc-7675dcddee4a: 2021-08-25T09:31:10.511Z [ERROR] server-88830c53-3729-34c0-46cc-7675dcddee4a: failed to make requestVote RPC: target="{Voter server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a}" error="failed to connect to peer: aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a" testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.550Z [WARN] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: Election timeout reached, restarting election testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.550Z [WARN] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: Election timeout reached, restarting election testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.550Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering candidate state: node="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Candidate]" term=5 testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.550Z [INFO] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: entering candidate state: node="Node at aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a [Candidate]" term=5 testLoggerAdapter verbose: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.551Z [ERROR] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: failed to make requestVote RPC: target="{Voter server-88830c53-3729-34c0-46cc-7675dcddee4a 88830c53-3729-34c0-46cc-7675dcddee4a}" error="failed to connect to peer: 88830c53-3729-34c0-46cc-7675dcddee4a" testing.go:149: server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: 2021-08-25T09:31:10.551Z [ERROR] server-aa99f1cd-8408-ba94-fb30-1eb87a4f3c2a: failed to make requestVote RPC: target="{Voter server-88830c53-3729-34c0-46cc-7675dcddee4a 88830c53-3729-34c0-46cc-7675dcddee4a}" error="failed to connect to peer: 88830c53-3729-34c0-46cc-7675dcddee4a" --- PASS: TestRaft_LeaderLeaseExpire (0.59s) === RUN TestRaft_Barrier testLoggerAdapter verbose: server-154e89a4-3aab-2684-8a41-72b8848dc531: 2021-08-25T09:31:10.592Z [INFO] server-154e89a4-3aab-2684-8a41-72b8848dc531: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-154e89a4-3aab-2684-8a41-72b8848dc531 Address:154e89a4-3aab-2684-8a41-72b8848dc531} {Suffrage:Voter ID:server-e878387b-410a-e289-4e3a-06aac34f1966 Address:e878387b-410a-e289-4e3a-06aac34f1966} {Suffrage:Voter ID:server-82e396a7-fc2a-4c62-9a44-6e2b16874476 Address:82e396a7-fc2a-4c62-9a44-6e2b16874476}]" testing.go:149: server-154e89a4-3aab-2684-8a41-72b8848dc531: 2021-08-25T09:31:10.592Z [INFO] server-154e89a4-3aab-2684-8a41-72b8848dc531: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-154e89a4-3aab-2684-8a41-72b8848dc531 Address:154e89a4-3aab-2684-8a41-72b8848dc531} {Suffrage:Voter ID:server-e878387b-410a-e289-4e3a-06aac34f1966 Address:e878387b-410a-e289-4e3a-06aac34f1966} {Suffrage:Voter ID:server-82e396a7-fc2a-4c62-9a44-6e2b16874476 Address:82e396a7-fc2a-4c62-9a44-6e2b16874476}]" testLoggerAdapter verbose: server-154e89a4-3aab-2684-8a41-72b8848dc531: 2021-08-25T09:31:10.592Z [INFO] server-154e89a4-3aab-2684-8a41-72b8848dc531: entering follower state: follower="Node at 154e89a4-3aab-2684-8a41-72b8848dc531 [Follower]" leader= testing.go:149: server-154e89a4-3aab-2684-8a41-72b8848dc531: 2021-08-25T09:31:10.592Z [INFO] server-154e89a4-3aab-2684-8a41-72b8848dc531: entering follower state: follower="Node at 154e89a4-3aab-2684-8a41-72b8848dc531 [Follower]" leader= testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.593Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-154e89a4-3aab-2684-8a41-72b8848dc531 Address:154e89a4-3aab-2684-8a41-72b8848dc531} {Suffrage:Voter ID:server-e878387b-410a-e289-4e3a-06aac34f1966 Address:e878387b-410a-e289-4e3a-06aac34f1966} {Suffrage:Voter ID:server-82e396a7-fc2a-4c62-9a44-6e2b16874476 Address:82e396a7-fc2a-4c62-9a44-6e2b16874476}]" testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.593Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-154e89a4-3aab-2684-8a41-72b8848dc531 Address:154e89a4-3aab-2684-8a41-72b8848dc531} {Suffrage:Voter ID:server-e878387b-410a-e289-4e3a-06aac34f1966 Address:e878387b-410a-e289-4e3a-06aac34f1966} {Suffrage:Voter ID:server-82e396a7-fc2a-4c62-9a44-6e2b16874476 Address:82e396a7-fc2a-4c62-9a44-6e2b16874476}]" testLoggerAdapter verbose: server-82e396a7-fc2a-4c62-9a44-6e2b16874476: 2021-08-25T09:31:10.593Z [INFO] server-82e396a7-fc2a-4c62-9a44-6e2b16874476: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-154e89a4-3aab-2684-8a41-72b8848dc531 Address:154e89a4-3aab-2684-8a41-72b8848dc531} {Suffrage:Voter ID:server-e878387b-410a-e289-4e3a-06aac34f1966 Address:e878387b-410a-e289-4e3a-06aac34f1966} {Suffrage:Voter ID:server-82e396a7-fc2a-4c62-9a44-6e2b16874476 Address:82e396a7-fc2a-4c62-9a44-6e2b16874476}]" testing.go:149: server-82e396a7-fc2a-4c62-9a44-6e2b16874476: 2021-08-25T09:31:10.593Z [INFO] server-82e396a7-fc2a-4c62-9a44-6e2b16874476: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-154e89a4-3aab-2684-8a41-72b8848dc531 Address:154e89a4-3aab-2684-8a41-72b8848dc531} {Suffrage:Voter ID:server-e878387b-410a-e289-4e3a-06aac34f1966 Address:e878387b-410a-e289-4e3a-06aac34f1966} {Suffrage:Voter ID:server-82e396a7-fc2a-4c62-9a44-6e2b16874476 Address:82e396a7-fc2a-4c62-9a44-6e2b16874476}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.593Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:10.593Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.593Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: entering follower state: follower="Node at e878387b-410a-e289-4e3a-06aac34f1966 [Follower]" leader= testLoggerAdapter verbose: server-82e396a7-fc2a-4c62-9a44-6e2b16874476: 2021-08-25T09:31:10.593Z [INFO] server-82e396a7-fc2a-4c62-9a44-6e2b16874476: entering follower state: follower="Node at 82e396a7-fc2a-4c62-9a44-6e2b16874476 [Follower]" leader= testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.593Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: entering follower state: follower="Node at e878387b-410a-e289-4e3a-06aac34f1966 [Follower]" leader= testing.go:149: server-82e396a7-fc2a-4c62-9a44-6e2b16874476: 2021-08-25T09:31:10.593Z [INFO] server-82e396a7-fc2a-4c62-9a44-6e2b16874476: entering follower state: follower="Node at 82e396a7-fc2a-4c62-9a44-6e2b16874476 [Follower]" leader= testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [WARN] server-e878387b-410a-e289-4e3a-06aac34f1966: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [WARN] server-e878387b-410a-e289-4e3a-06aac34f1966: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: entering candidate state: node="Node at e878387b-410a-e289-4e3a-06aac34f1966 [Candidate]" term=2 testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: entering candidate state: node="Node at e878387b-410a-e289-4e3a-06aac34f1966 [Candidate]" term=2 testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: election won: tally=2 testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: election won: tally=2 testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: entering leader state: leader="Node at e878387b-410a-e289-4e3a-06aac34f1966 [Leader]" testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: entering leader state: leader="Node at e878387b-410a-e289-4e3a-06aac34f1966 [Leader]" testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: added peer, starting replication: peer=server-154e89a4-3aab-2684-8a41-72b8848dc531 testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.659Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: added peer, starting replication: peer=server-154e89a4-3aab-2684-8a41-72b8848dc531 testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.660Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: added peer, starting replication: peer=server-82e396a7-fc2a-4c62-9a44-6e2b16874476 testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.660Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: added peer, starting replication: peer=server-82e396a7-fc2a-4c62-9a44-6e2b16874476 testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.660Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: pipelining replication: peer="{Voter server-82e396a7-fc2a-4c62-9a44-6e2b16874476 82e396a7-fc2a-4c62-9a44-6e2b16874476}" testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.660Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: pipelining replication: peer="{Voter server-82e396a7-fc2a-4c62-9a44-6e2b16874476 82e396a7-fc2a-4c62-9a44-6e2b16874476}" testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.660Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: pipelining replication: peer="{Voter server-154e89a4-3aab-2684-8a41-72b8848dc531 154e89a4-3aab-2684-8a41-72b8848dc531}" testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.660Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: pipelining replication: peer="{Voter server-154e89a4-3aab-2684-8a41-72b8848dc531 154e89a4-3aab-2684-8a41-72b8848dc531}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.765Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.660110845 +0000 UTC m=+14.926327269 (1 nodes), 66.588093ms from start of poll, 67.743005ms from cluster start. Timeout at 2021-08-25 09:31:10.765230411 +0000 UTC m=+15.031446895, 105.119626ms after stability testing.go:149: cluster: 2021-08-25T09:31:10.765Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.660110845 +0000 UTC m=+14.926327269 (1 nodes), 66.588093ms from start of poll, 67.743005ms from cluster start. Timeout at 2021-08-25 09:31:10.765230411 +0000 UTC m=+15.031446895, 105.119626ms after stability testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.778Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: aborting pipeline replication: peer="{Voter server-82e396a7-fc2a-4c62-9a44-6e2b16874476 82e396a7-fc2a-4c62-9a44-6e2b16874476}" testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.778Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: aborting pipeline replication: peer="{Voter server-82e396a7-fc2a-4c62-9a44-6e2b16874476 82e396a7-fc2a-4c62-9a44-6e2b16874476}" testLoggerAdapter verbose: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.778Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: aborting pipeline replication: peer="{Voter server-154e89a4-3aab-2684-8a41-72b8848dc531 154e89a4-3aab-2684-8a41-72b8848dc531}" testing.go:149: server-e878387b-410a-e289-4e3a-06aac34f1966: 2021-08-25T09:31:10.778Z [INFO] server-e878387b-410a-e289-4e3a-06aac34f1966: aborting pipeline replication: peer="{Voter server-154e89a4-3aab-2684-8a41-72b8848dc531 154e89a4-3aab-2684-8a41-72b8848dc531}" --- PASS: TestRaft_Barrier (0.19s) === RUN TestRaft_VerifyLeader testLoggerAdapter verbose: server-a608c8f4-ef70-7540-bd39-ebb870cca270: 2021-08-25T09:31:10.782Z [INFO] server-a608c8f4-ef70-7540-bd39-ebb870cca270: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a608c8f4-ef70-7540-bd39-ebb870cca270 Address:a608c8f4-ef70-7540-bd39-ebb870cca270} {Suffrage:Voter ID:server-6b40de83-1845-9d67-1b6c-7d3ce573a402 Address:6b40de83-1845-9d67-1b6c-7d3ce573a402} {Suffrage:Voter ID:server-78c42101-5b4f-cfb9-118b-cf5560acad66 Address:78c42101-5b4f-cfb9-118b-cf5560acad66}]" testing.go:149: server-a608c8f4-ef70-7540-bd39-ebb870cca270: 2021-08-25T09:31:10.782Z [INFO] server-a608c8f4-ef70-7540-bd39-ebb870cca270: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a608c8f4-ef70-7540-bd39-ebb870cca270 Address:a608c8f4-ef70-7540-bd39-ebb870cca270} {Suffrage:Voter ID:server-6b40de83-1845-9d67-1b6c-7d3ce573a402 Address:6b40de83-1845-9d67-1b6c-7d3ce573a402} {Suffrage:Voter ID:server-78c42101-5b4f-cfb9-118b-cf5560acad66 Address:78c42101-5b4f-cfb9-118b-cf5560acad66}]" testLoggerAdapter verbose: server-a608c8f4-ef70-7540-bd39-ebb870cca270: 2021-08-25T09:31:10.782Z [INFO] server-a608c8f4-ef70-7540-bd39-ebb870cca270: entering follower state: follower="Node at a608c8f4-ef70-7540-bd39-ebb870cca270 [Follower]" leader= testing.go:149: server-a608c8f4-ef70-7540-bd39-ebb870cca270: 2021-08-25T09:31:10.782Z [INFO] server-a608c8f4-ef70-7540-bd39-ebb870cca270: entering follower state: follower="Node at a608c8f4-ef70-7540-bd39-ebb870cca270 [Follower]" leader= testLoggerAdapter verbose: server-6b40de83-1845-9d67-1b6c-7d3ce573a402: 2021-08-25T09:31:10.782Z [INFO] server-6b40de83-1845-9d67-1b6c-7d3ce573a402: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a608c8f4-ef70-7540-bd39-ebb870cca270 Address:a608c8f4-ef70-7540-bd39-ebb870cca270} {Suffrage:Voter ID:server-6b40de83-1845-9d67-1b6c-7d3ce573a402 Address:6b40de83-1845-9d67-1b6c-7d3ce573a402} {Suffrage:Voter ID:server-78c42101-5b4f-cfb9-118b-cf5560acad66 Address:78c42101-5b4f-cfb9-118b-cf5560acad66}]" testing.go:149: server-6b40de83-1845-9d67-1b6c-7d3ce573a402: 2021-08-25T09:31:10.782Z [INFO] server-6b40de83-1845-9d67-1b6c-7d3ce573a402: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a608c8f4-ef70-7540-bd39-ebb870cca270 Address:a608c8f4-ef70-7540-bd39-ebb870cca270} {Suffrage:Voter ID:server-6b40de83-1845-9d67-1b6c-7d3ce573a402 Address:6b40de83-1845-9d67-1b6c-7d3ce573a402} {Suffrage:Voter ID:server-78c42101-5b4f-cfb9-118b-cf5560acad66 Address:78c42101-5b4f-cfb9-118b-cf5560acad66}]" testLoggerAdapter verbose: server-6b40de83-1845-9d67-1b6c-7d3ce573a402: 2021-08-25T09:31:10.782Z [INFO] server-6b40de83-1845-9d67-1b6c-7d3ce573a402: entering follower state: follower="Node at 6b40de83-1845-9d67-1b6c-7d3ce573a402 [Follower]" leader= testing.go:149: server-6b40de83-1845-9d67-1b6c-7d3ce573a402: 2021-08-25T09:31:10.782Z [INFO] server-6b40de83-1845-9d67-1b6c-7d3ce573a402: entering follower state: follower="Node at 6b40de83-1845-9d67-1b6c-7d3ce573a402 [Follower]" leader= testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.783Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a608c8f4-ef70-7540-bd39-ebb870cca270 Address:a608c8f4-ef70-7540-bd39-ebb870cca270} {Suffrage:Voter ID:server-6b40de83-1845-9d67-1b6c-7d3ce573a402 Address:6b40de83-1845-9d67-1b6c-7d3ce573a402} {Suffrage:Voter ID:server-78c42101-5b4f-cfb9-118b-cf5560acad66 Address:78c42101-5b4f-cfb9-118b-cf5560acad66}]" testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.783Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a608c8f4-ef70-7540-bd39-ebb870cca270 Address:a608c8f4-ef70-7540-bd39-ebb870cca270} {Suffrage:Voter ID:server-6b40de83-1845-9d67-1b6c-7d3ce573a402 Address:6b40de83-1845-9d67-1b6c-7d3ce573a402} {Suffrage:Voter ID:server-78c42101-5b4f-cfb9-118b-cf5560acad66 Address:78c42101-5b4f-cfb9-118b-cf5560acad66}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.783Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:10.783Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.783Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: entering follower state: follower="Node at 78c42101-5b4f-cfb9-118b-cf5560acad66 [Follower]" leader= testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.783Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: entering follower state: follower="Node at 78c42101-5b4f-cfb9-118b-cf5560acad66 [Follower]" leader= testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [WARN] server-78c42101-5b4f-cfb9-118b-cf5560acad66: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [WARN] server-78c42101-5b4f-cfb9-118b-cf5560acad66: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: entering candidate state: node="Node at 78c42101-5b4f-cfb9-118b-cf5560acad66 [Candidate]" term=2 testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: entering candidate state: node="Node at 78c42101-5b4f-cfb9-118b-cf5560acad66 [Candidate]" term=2 testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: election won: tally=2 testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: election won: tally=2 testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: entering leader state: leader="Node at 78c42101-5b4f-cfb9-118b-cf5560acad66 [Leader]" testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: entering leader state: leader="Node at 78c42101-5b4f-cfb9-118b-cf5560acad66 [Leader]" testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: added peer, starting replication: peer=server-a608c8f4-ef70-7540-bd39-ebb870cca270 testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.860Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: added peer, starting replication: peer=server-a608c8f4-ef70-7540-bd39-ebb870cca270 testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.861Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: added peer, starting replication: peer=server-6b40de83-1845-9d67-1b6c-7d3ce573a402 testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.861Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: added peer, starting replication: peer=server-6b40de83-1845-9d67-1b6c-7d3ce573a402 testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.861Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: pipelining replication: peer="{Voter server-6b40de83-1845-9d67-1b6c-7d3ce573a402 6b40de83-1845-9d67-1b6c-7d3ce573a402}" testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.861Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: pipelining replication: peer="{Voter server-6b40de83-1845-9d67-1b6c-7d3ce573a402 6b40de83-1845-9d67-1b6c-7d3ce573a402}" testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.861Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: pipelining replication: peer="{Voter server-a608c8f4-ef70-7540-bd39-ebb870cca270 a608c8f4-ef70-7540-bd39-ebb870cca270}" testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.861Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: pipelining replication: peer="{Voter server-a608c8f4-ef70-7540-bd39-ebb870cca270 a608c8f4-ef70-7540-bd39-ebb870cca270}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.966Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.86107815 +0000 UTC m=+15.127294594 (1 nodes), 77.842993ms from start of poll, 78.890255ms from cluster start. Timeout at 2021-08-25 09:31:10.966182378 +0000 UTC m=+15.232398822, 105.104228ms after stability testing.go:149: cluster: 2021-08-25T09:31:10.966Z [INFO] stable state for Leader reached at 2021-08-25 09:31:10.86107815 +0000 UTC m=+15.127294594 (1 nodes), 77.842993ms from start of poll, 78.890255ms from cluster start. Timeout at 2021-08-25 09:31:10.966182378 +0000 UTC m=+15.232398822, 105.104228ms after stability testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.966Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: aborting pipeline replication: peer="{Voter server-6b40de83-1845-9d67-1b6c-7d3ce573a402 6b40de83-1845-9d67-1b6c-7d3ce573a402}" testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.966Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: aborting pipeline replication: peer="{Voter server-6b40de83-1845-9d67-1b6c-7d3ce573a402 6b40de83-1845-9d67-1b6c-7d3ce573a402}" testLoggerAdapter verbose: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.966Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: aborting pipeline replication: peer="{Voter server-a608c8f4-ef70-7540-bd39-ebb870cca270 a608c8f4-ef70-7540-bd39-ebb870cca270}" testing.go:149: server-78c42101-5b4f-cfb9-118b-cf5560acad66: 2021-08-25T09:31:10.966Z [INFO] server-78c42101-5b4f-cfb9-118b-cf5560acad66: aborting pipeline replication: peer="{Voter server-a608c8f4-ef70-7540-bd39-ebb870cca270 a608c8f4-ef70-7540-bd39-ebb870cca270}" --- PASS: TestRaft_VerifyLeader (0.19s) === RUN TestRaft_VerifyLeader_Single testLoggerAdapter verbose: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:10.969Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f60b581-6195-885e-c190-3adde8d05dfe Address:9f60b581-6195-885e-c190-3adde8d05dfe}]" testing.go:149: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:10.969Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f60b581-6195-885e-c190-3adde8d05dfe Address:9f60b581-6195-885e-c190-3adde8d05dfe}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:10.969Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:10.969Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:10.969Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: entering follower state: follower="Node at 9f60b581-6195-885e-c190-3adde8d05dfe [Follower]" leader= testing.go:149: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:10.969Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: entering follower state: follower="Node at 9f60b581-6195-885e-c190-3adde8d05dfe [Follower]" leader= testLoggerAdapter verbose: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.046Z [WARN] server-9f60b581-6195-885e-c190-3adde8d05dfe: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.046Z [WARN] server-9f60b581-6195-885e-c190-3adde8d05dfe: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.046Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: entering candidate state: node="Node at 9f60b581-6195-885e-c190-3adde8d05dfe [Candidate]" term=2 testing.go:149: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.046Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: entering candidate state: node="Node at 9f60b581-6195-885e-c190-3adde8d05dfe [Candidate]" term=2 testLoggerAdapter verbose: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.047Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: election won: tally=1 testing.go:149: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.047Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: election won: tally=1 testLoggerAdapter verbose: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.047Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: entering leader state: leader="Node at 9f60b581-6195-885e-c190-3adde8d05dfe [Leader]" testing.go:149: server-9f60b581-6195-885e-c190-3adde8d05dfe: 2021-08-25T09:31:11.047Z [INFO] server-9f60b581-6195-885e-c190-3adde8d05dfe: entering leader state: leader="Node at 9f60b581-6195-885e-c190-3adde8d05dfe [Leader]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.152Z [INFO] stable state for Leader reached at 2021-08-25 09:31:11.04723531 +0000 UTC m=+15.313451714 (1 nodes), 77.616135ms from start of poll, 78.089351ms from cluster start. Timeout at 2021-08-25 09:31:11.152368415 +0000 UTC m=+15.418584899, 105.133185ms after stability testing.go:149: cluster: 2021-08-25T09:31:11.152Z [INFO] stable state for Leader reached at 2021-08-25 09:31:11.04723531 +0000 UTC m=+15.313451714 (1 nodes), 77.616135ms from start of poll, 78.089351ms from cluster start. Timeout at 2021-08-25 09:31:11.152368415 +0000 UTC m=+15.418584899, 105.133185ms after stability --- PASS: TestRaft_VerifyLeader_Single (0.19s) === RUN TestRaft_VerifyLeader_Fail testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.156Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-069231f4-d283-a901-d9f9-5a87c397b0be Address:069231f4-d283-a901-d9f9-5a87c397b0be} {Suffrage:Voter ID:server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 Address:4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}]" testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.156Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-069231f4-d283-a901-d9f9-5a87c397b0be Address:069231f4-d283-a901-d9f9-5a87c397b0be} {Suffrage:Voter ID:server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 Address:4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}]" testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.156Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering follower state: follower="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Follower]" leader= testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.156Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering follower state: follower="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Follower]" leader= testLoggerAdapter verbose: server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: 2021-08-25T09:31:11.156Z [INFO] server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-069231f4-d283-a901-d9f9-5a87c397b0be Address:069231f4-d283-a901-d9f9-5a87c397b0be} {Suffrage:Voter ID:server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 Address:4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}]" testing.go:149: server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: 2021-08-25T09:31:11.156Z [INFO] server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-069231f4-d283-a901-d9f9-5a87c397b0be Address:069231f4-d283-a901-d9f9-5a87c397b0be} {Suffrage:Voter ID:server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 Address:4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.156Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:11.156Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: 2021-08-25T09:31:11.156Z [INFO] server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: entering follower state: follower="Node at 4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 [Follower]" leader= testing.go:149: server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: 2021-08-25T09:31:11.156Z [INFO] server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9: entering follower state: follower="Node at 4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 [Follower]" leader= testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [WARN] server-069231f4-d283-a901-d9f9-5a87c397b0be: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [WARN] server-069231f4-d283-a901-d9f9-5a87c397b0be: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering candidate state: node="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Candidate]" term=2 testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering candidate state: node="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Candidate]" term=2 testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: election won: tally=2 testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: election won: tally=2 testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering leader state: leader="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Leader]" testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.228Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering leader state: leader="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Leader]" testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.229Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: added peer, starting replication: peer=server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.229Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: added peer, starting replication: peer=server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.229Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: pipelining replication: peer="{Voter server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}" testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.229Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: pipelining replication: peer="{Voter server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.334Z [INFO] stable state for Leader reached at 2021-08-25 09:31:11.229073616 +0000 UTC m=+15.495290040 (1 nodes), 72.441942ms from start of poll, 73.239367ms from cluster start. Timeout at 2021-08-25 09:31:11.334252137 +0000 UTC m=+15.600468601, 105.178561ms after stability testing.go:149: cluster: 2021-08-25T09:31:11.334Z [INFO] stable state for Leader reached at 2021-08-25 09:31:11.229073616 +0000 UTC m=+15.495290040 (1 nodes), 72.441942ms from start of poll, 73.239367ms from cluster start. Timeout at 2021-08-25 09:31:11.334252137 +0000 UTC m=+15.600468601, 105.178561ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.334Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:11.334Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.439Z [INFO] stable state for Follower reached at 2021-08-25 09:31:11.334634061 +0000 UTC m=+15.600850505 (1 nodes), 2.44µs from start of poll, 178.799832ms from cluster start. Timeout at 2021-08-25 09:31:11.439746928 +0000 UTC m=+15.705963352, 105.112847ms after stability testing.go:149: cluster: 2021-08-25T09:31:11.439Z [INFO] stable state for Follower reached at 2021-08-25 09:31:11.334634061 +0000 UTC m=+15.600850505 (1 nodes), 2.44µs from start of poll, 178.799832ms from cluster start. Timeout at 2021-08-25 09:31:11.439746928 +0000 UTC m=+15.705963352, 105.112847ms after stability testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.440Z [WARN] server-069231f4-d283-a901-d9f9-5a87c397b0be: new leader elected, stepping down testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.440Z [WARN] server-069231f4-d283-a901-d9f9-5a87c397b0be: new leader elected, stepping down testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.440Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering follower state: follower="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Follower]" leader= testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.440Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: entering follower state: follower="Node at 069231f4-d283-a901-d9f9-5a87c397b0be [Follower]" leader= testLoggerAdapter verbose: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.440Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: aborting pipeline replication: peer="{Voter server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}" testing.go:149: server-069231f4-d283-a901-d9f9-5a87c397b0be: 2021-08-25T09:31:11.440Z [INFO] server-069231f4-d283-a901-d9f9-5a87c397b0be: aborting pipeline replication: peer="{Voter server-4c8c4a3b-e938-1e01-5360-9e6bdbc562e9 4c8c4a3b-e938-1e01-5360-9e6bdbc562e9}" --- PASS: TestRaft_VerifyLeader_Fail (0.29s) === RUN TestRaft_VerifyLeader_PartialConnect testLoggerAdapter verbose: server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: 2021-08-25T09:31:11.443Z [INFO] server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 Address:ce5fdc15-67fd-fed9-48c0-67c7f16df928} {Suffrage:Voter ID:server-44e3b96b-e662-8caf-28f6-a369ff3ef97b Address:44e3b96b-e662-8caf-28f6-a369ff3ef97b} {Suffrage:Voter ID:server-185c6ebc-ee71-0646-9494-9c3280d22dff Address:185c6ebc-ee71-0646-9494-9c3280d22dff}]" testing.go:149: server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: 2021-08-25T09:31:11.443Z [INFO] server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 Address:ce5fdc15-67fd-fed9-48c0-67c7f16df928} {Suffrage:Voter ID:server-44e3b96b-e662-8caf-28f6-a369ff3ef97b Address:44e3b96b-e662-8caf-28f6-a369ff3ef97b} {Suffrage:Voter ID:server-185c6ebc-ee71-0646-9494-9c3280d22dff Address:185c6ebc-ee71-0646-9494-9c3280d22dff}]" testLoggerAdapter verbose: server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: 2021-08-25T09:31:11.443Z [INFO] server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: entering follower state: follower="Node at ce5fdc15-67fd-fed9-48c0-67c7f16df928 [Follower]" leader= testing.go:149: server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: 2021-08-25T09:31:11.443Z [INFO] server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: entering follower state: follower="Node at ce5fdc15-67fd-fed9-48c0-67c7f16df928 [Follower]" leader= testLoggerAdapter verbose: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.443Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 Address:ce5fdc15-67fd-fed9-48c0-67c7f16df928} {Suffrage:Voter ID:server-44e3b96b-e662-8caf-28f6-a369ff3ef97b Address:44e3b96b-e662-8caf-28f6-a369ff3ef97b} {Suffrage:Voter ID:server-185c6ebc-ee71-0646-9494-9c3280d22dff Address:185c6ebc-ee71-0646-9494-9c3280d22dff}]" testing.go:149: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.443Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 Address:ce5fdc15-67fd-fed9-48c0-67c7f16df928} {Suffrage:Voter ID:server-44e3b96b-e662-8caf-28f6-a369ff3ef97b Address:44e3b96b-e662-8caf-28f6-a369ff3ef97b} {Suffrage:Voter ID:server-185c6ebc-ee71-0646-9494-9c3280d22dff Address:185c6ebc-ee71-0646-9494-9c3280d22dff}]" testLoggerAdapter verbose: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.443Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: entering follower state: follower="Node at 44e3b96b-e662-8caf-28f6-a369ff3ef97b [Follower]" leader= testing.go:149: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.443Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: entering follower state: follower="Node at 44e3b96b-e662-8caf-28f6-a369ff3ef97b [Follower]" leader= testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.444Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 Address:ce5fdc15-67fd-fed9-48c0-67c7f16df928} {Suffrage:Voter ID:server-44e3b96b-e662-8caf-28f6-a369ff3ef97b Address:44e3b96b-e662-8caf-28f6-a369ff3ef97b} {Suffrage:Voter ID:server-185c6ebc-ee71-0646-9494-9c3280d22dff Address:185c6ebc-ee71-0646-9494-9c3280d22dff}]" testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.444Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 Address:ce5fdc15-67fd-fed9-48c0-67c7f16df928} {Suffrage:Voter ID:server-44e3b96b-e662-8caf-28f6-a369ff3ef97b Address:44e3b96b-e662-8caf-28f6-a369ff3ef97b} {Suffrage:Voter ID:server-185c6ebc-ee71-0646-9494-9c3280d22dff Address:185c6ebc-ee71-0646-9494-9c3280d22dff}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.444Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:11.444Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.444Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: entering follower state: follower="Node at 185c6ebc-ee71-0646-9494-9c3280d22dff [Follower]" leader= testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.444Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: entering follower state: follower="Node at 185c6ebc-ee71-0646-9494-9c3280d22dff [Follower]" leader= testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.538Z [WARN] server-185c6ebc-ee71-0646-9494-9c3280d22dff: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.538Z [WARN] server-185c6ebc-ee71-0646-9494-9c3280d22dff: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: entering candidate state: node="Node at 185c6ebc-ee71-0646-9494-9c3280d22dff [Candidate]" term=2 testLoggerAdapter verbose: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.539Z [WARN] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: entering candidate state: node="Node at 185c6ebc-ee71-0646-9494-9c3280d22dff [Candidate]" term=2 testing.go:149: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.539Z [WARN] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.539Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: entering candidate state: node="Node at 44e3b96b-e662-8caf-28f6-a369ff3ef97b [Candidate]" term=2 testing.go:149: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.539Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: entering candidate state: node="Node at 44e3b96b-e662-8caf-28f6-a369ff3ef97b [Candidate]" term=2 testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: election won: tally=2 testLoggerAdapter verbose: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.539Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: duplicate requestVote for same term: term=2 testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: election won: tally=2 testLoggerAdapter verbose: server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: 2021-08-25T09:31:11.539Z [INFO] server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: duplicate requestVote for same term: term=2 testing.go:149: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.539Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: duplicate requestVote for same term: term=2 testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: entering leader state: leader="Node at 185c6ebc-ee71-0646-9494-9c3280d22dff [Leader]" testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: entering leader state: leader="Node at 185c6ebc-ee71-0646-9494-9c3280d22dff [Leader]" testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: added peer, starting replication: peer=server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: added peer, starting replication: peer=server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: added peer, starting replication: peer=server-44e3b96b-e662-8caf-28f6-a369ff3ef97b testing.go:149: server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: 2021-08-25T09:31:11.539Z [INFO] server-ce5fdc15-67fd-fed9-48c0-67c7f16df928: duplicate requestVote for same term: term=2 testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: added peer, starting replication: peer=server-44e3b96b-e662-8caf-28f6-a369ff3ef97b testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: pipelining replication: peer="{Voter server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 ce5fdc15-67fd-fed9-48c0-67c7f16df928}" testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.539Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: pipelining replication: peer="{Voter server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 ce5fdc15-67fd-fed9-48c0-67c7f16df928}" testLoggerAdapter verbose: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.540Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: entering follower state: follower="Node at 44e3b96b-e662-8caf-28f6-a369ff3ef97b [Follower]" leader=185c6ebc-ee71-0646-9494-9c3280d22dff testing.go:149: server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: 2021-08-25T09:31:11.540Z [INFO] server-44e3b96b-e662-8caf-28f6-a369ff3ef97b: entering follower state: follower="Node at 44e3b96b-e662-8caf-28f6-a369ff3ef97b [Follower]" leader=185c6ebc-ee71-0646-9494-9c3280d22dff testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.540Z [WARN] server-185c6ebc-ee71-0646-9494-9c3280d22dff: rejecting vote request since we have a leader: from=44e3b96b-e662-8caf-28f6-a369ff3ef97b leader=185c6ebc-ee71-0646-9494-9c3280d22dff testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.540Z [WARN] server-185c6ebc-ee71-0646-9494-9c3280d22dff: rejecting vote request since we have a leader: from=44e3b96b-e662-8caf-28f6-a369ff3ef97b leader=185c6ebc-ee71-0646-9494-9c3280d22dff testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.540Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: pipelining replication: peer="{Voter server-44e3b96b-e662-8caf-28f6-a369ff3ef97b 44e3b96b-e662-8caf-28f6-a369ff3ef97b}" testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.540Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: pipelining replication: peer="{Voter server-44e3b96b-e662-8caf-28f6-a369ff3ef97b 44e3b96b-e662-8caf-28f6-a369ff3ef97b}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.645Z [INFO] stable state for Leader reached at 2021-08-25 09:31:11.540143219 +0000 UTC m=+15.806359603 (1 nodes), 95.887675ms from start of poll, 96.993671ms from cluster start. Timeout at 2021-08-25 09:31:11.645270205 +0000 UTC m=+15.911486629, 105.127026ms after stability testing.go:149: cluster: 2021-08-25T09:31:11.645Z [INFO] stable state for Leader reached at 2021-08-25 09:31:11.540143219 +0000 UTC m=+15.806359603 (1 nodes), 95.887675ms from start of poll, 96.993671ms from cluster start. Timeout at 2021-08-25 09:31:11.645270205 +0000 UTC m=+15.911486629, 105.127026ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.650Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:11.650Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.756Z [INFO] stable state for Follower reached at 2021-08-25 09:31:11.650882297 +0000 UTC m=+15.917098701 (2 nodes), 3.16µs from start of poll, 207.732769ms from cluster start. Timeout at 2021-08-25 09:31:11.756050859 +0000 UTC m=+16.022267343, 105.168642ms after stability testing.go:149: cluster: 2021-08-25T09:31:11.756Z [INFO] stable state for Follower reached at 2021-08-25 09:31:11.650882297 +0000 UTC m=+15.917098701 (2 nodes), 3.16µs from start of poll, 207.732769ms from cluster start. Timeout at 2021-08-25 09:31:11.756050859 +0000 UTC m=+16.022267343, 105.168642ms after stability raft_test.go:1647: [INFO] Disconnecting Node at ce5fdc15-67fd-fed9-48c0-67c7f16df928 [Follower] testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.756Z [ERROR] server-185c6ebc-ee71-0646-9494-9c3280d22dff: failed to heartbeat to: peer=ce5fdc15-67fd-fed9-48c0-67c7f16df928 error="failed to connect to peer: ce5fdc15-67fd-fed9-48c0-67c7f16df928" testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.756Z [ERROR] server-185c6ebc-ee71-0646-9494-9c3280d22dff: failed to heartbeat to: peer=ce5fdc15-67fd-fed9-48c0-67c7f16df928 error="failed to connect to peer: ce5fdc15-67fd-fed9-48c0-67c7f16df928" testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.756Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: aborting pipeline replication: peer="{Voter server-44e3b96b-e662-8caf-28f6-a369ff3ef97b 44e3b96b-e662-8caf-28f6-a369ff3ef97b}" testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.756Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: aborting pipeline replication: peer="{Voter server-44e3b96b-e662-8caf-28f6-a369ff3ef97b 44e3b96b-e662-8caf-28f6-a369ff3ef97b}" testLoggerAdapter verbose: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.756Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: aborting pipeline replication: peer="{Voter server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 ce5fdc15-67fd-fed9-48c0-67c7f16df928}" testing.go:149: server-185c6ebc-ee71-0646-9494-9c3280d22dff: 2021-08-25T09:31:11.756Z [INFO] server-185c6ebc-ee71-0646-9494-9c3280d22dff: aborting pipeline replication: peer="{Voter server-ce5fdc15-67fd-fed9-48c0-67c7f16df928 ce5fdc15-67fd-fed9-48c0-67c7f16df928}" --- PASS: TestRaft_VerifyLeader_PartialConnect (0.32s) === RUN TestRaft_StartAsLeader testLoggerAdapter verbose: server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: 2021-08-25T09:31:11.759Z [INFO] server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2 Address:3d7137dd-6195-1bb9-9bca-8359c07eb2a2}]" testing.go:149: server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: 2021-08-25T09:31:11.759Z [INFO] server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2 Address:3d7137dd-6195-1bb9-9bca-8359c07eb2a2}]" testLoggerAdapter verbose: server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: 2021-08-25T09:31:11.759Z [INFO] server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: entering leader state: leader="Node at 3d7137dd-6195-1bb9-9bca-8359c07eb2a2 [Leader]" testing.go:149: server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: 2021-08-25T09:31:11.759Z [INFO] server-3d7137dd-6195-1bb9-9bca-8359c07eb2a2: entering leader state: leader="Node at 3d7137dd-6195-1bb9-9bca-8359c07eb2a2 [Leader]" --- PASS: TestRaft_StartAsLeader (0.00s) === RUN TestRaft_NotifyCh testLoggerAdapter verbose: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.761Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7744c0db-b751-2cf5-cf8e-674ca958b595 Address:7744c0db-b751-2cf5-cf8e-674ca958b595}]" testing.go:149: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.761Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7744c0db-b751-2cf5-cf8e-674ca958b595 Address:7744c0db-b751-2cf5-cf8e-674ca958b595}]" testLoggerAdapter verbose: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.761Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: entering follower state: follower="Node at 7744c0db-b751-2cf5-cf8e-674ca958b595 [Follower]" leader= testing.go:149: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.761Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: entering follower state: follower="Node at 7744c0db-b751-2cf5-cf8e-674ca958b595 [Follower]" leader= testLoggerAdapter verbose: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [WARN] server-7744c0db-b751-2cf5-cf8e-674ca958b595: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [WARN] server-7744c0db-b751-2cf5-cf8e-674ca958b595: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: entering candidate state: node="Node at 7744c0db-b751-2cf5-cf8e-674ca958b595 [Candidate]" term=2 testing.go:149: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: entering candidate state: node="Node at 7744c0db-b751-2cf5-cf8e-674ca958b595 [Candidate]" term=2 testLoggerAdapter verbose: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: election won: tally=1 testing.go:149: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: election won: tally=1 testLoggerAdapter verbose: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: entering leader state: leader="Node at 7744c0db-b751-2cf5-cf8e-674ca958b595 [Leader]" testing.go:149: server-7744c0db-b751-2cf5-cf8e-674ca958b595: 2021-08-25T09:31:11.844Z [INFO] server-7744c0db-b751-2cf5-cf8e-674ca958b595: entering leader state: leader="Node at 7744c0db-b751-2cf5-cf8e-674ca958b595 [Leader]" --- PASS: TestRaft_NotifyCh (0.08s) === RUN TestRaft_Voting testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.847Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 Address:ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12} {Suffrage:Voter ID:server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 Address:cfd7fc9e-5cf7-1217-f77d-dcfcad192f50} {Suffrage:Voter ID:server-f9eeae9a-4039-312d-7455-5b19e6edd20c Address:f9eeae9a-4039-312d-7455-5b19e6edd20c}]" testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.847Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 Address:ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12} {Suffrage:Voter ID:server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 Address:cfd7fc9e-5cf7-1217-f77d-dcfcad192f50} {Suffrage:Voter ID:server-f9eeae9a-4039-312d-7455-5b19e6edd20c Address:f9eeae9a-4039-312d-7455-5b19e6edd20c}]" testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.847Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: entering follower state: follower="Node at ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 [Follower]" leader= testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.847Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: entering follower state: follower="Node at ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 [Follower]" leader= testLoggerAdapter verbose: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.847Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 Address:ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12} {Suffrage:Voter ID:server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 Address:cfd7fc9e-5cf7-1217-f77d-dcfcad192f50} {Suffrage:Voter ID:server-f9eeae9a-4039-312d-7455-5b19e6edd20c Address:f9eeae9a-4039-312d-7455-5b19e6edd20c}]" testing.go:149: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.847Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 Address:ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12} {Suffrage:Voter ID:server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 Address:cfd7fc9e-5cf7-1217-f77d-dcfcad192f50} {Suffrage:Voter ID:server-f9eeae9a-4039-312d-7455-5b19e6edd20c Address:f9eeae9a-4039-312d-7455-5b19e6edd20c}]" testLoggerAdapter verbose: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.847Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: entering follower state: follower="Node at cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 [Follower]" leader= testing.go:149: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.847Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: entering follower state: follower="Node at cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 [Follower]" leader= testLoggerAdapter verbose: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:11.848Z [INFO] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 Address:ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12} {Suffrage:Voter ID:server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 Address:cfd7fc9e-5cf7-1217-f77d-dcfcad192f50} {Suffrage:Voter ID:server-f9eeae9a-4039-312d-7455-5b19e6edd20c Address:f9eeae9a-4039-312d-7455-5b19e6edd20c}]" testing.go:149: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:11.848Z [INFO] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 Address:ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12} {Suffrage:Voter ID:server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 Address:cfd7fc9e-5cf7-1217-f77d-dcfcad192f50} {Suffrage:Voter ID:server-f9eeae9a-4039-312d-7455-5b19e6edd20c Address:f9eeae9a-4039-312d-7455-5b19e6edd20c}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:11.848Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:11.848Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:11.848Z [INFO] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: entering follower state: follower="Node at f9eeae9a-4039-312d-7455-5b19e6edd20c [Follower]" leader= testing.go:149: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:11.848Z [INFO] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: entering follower state: follower="Node at f9eeae9a-4039-312d-7455-5b19e6edd20c [Follower]" leader= testLoggerAdapter verbose: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.903Z [WARN] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.903Z [WARN] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.903Z [WARN] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.903Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: entering candidate state: node="Node at cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 [Candidate]" term=2 testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.903Z [WARN] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.903Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: entering candidate state: node="Node at ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 [Candidate]" term=2 testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.903Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: entering candidate state: node="Node at ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 [Candidate]" term=2 testing.go:149: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.903Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: entering candidate state: node="Node at cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 [Candidate]" term=2 testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.903Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: election won: tally=2 testLoggerAdapter verbose: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.903Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: duplicate requestVote for same term: term=2 testLoggerAdapter verbose: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:11.903Z [INFO] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: duplicate requestVote for same term: term=2 testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.903Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: election won: tally=2 testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: entering leader state: leader="Node at ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 [Leader]" testing.go:149: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.903Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: duplicate requestVote for same term: term=2 testing.go:149: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:11.903Z [INFO] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: duplicate requestVote for same term: term=2 testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: entering leader state: leader="Node at ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 [Leader]" testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: added peer, starting replication: peer=server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: added peer, starting replication: peer=server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: added peer, starting replication: peer=server-f9eeae9a-4039-312d-7455-5b19e6edd20c testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: added peer, starting replication: peer=server-f9eeae9a-4039-312d-7455-5b19e6edd20c testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [WARN] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: rejecting vote request since we have a leader: from=cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 leader=ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [WARN] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: rejecting vote request since we have a leader: from=cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 leader=ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 testLoggerAdapter verbose: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.904Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: entering follower state: follower="Node at cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 [Follower]" leader=ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: pipelining replication: peer="{Voter server-f9eeae9a-4039-312d-7455-5b19e6edd20c f9eeae9a-4039-312d-7455-5b19e6edd20c}" testing.go:149: server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: 2021-08-25T09:31:11.904Z [INFO] server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50: entering follower state: follower="Node at cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 [Follower]" leader=ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: pipelining replication: peer="{Voter server-f9eeae9a-4039-312d-7455-5b19e6edd20c f9eeae9a-4039-312d-7455-5b19e6edd20c}" testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: pipelining replication: peer="{Voter server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 cfd7fc9e-5cf7-1217-f77d-dcfcad192f50}" testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:11.904Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: pipelining replication: peer="{Voter server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 cfd7fc9e-5cf7-1217-f77d-dcfcad192f50}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.009Z [INFO] stable state for Follower reached at 2021-08-25 09:31:11.904619316 +0000 UTC m=+16.170835740 (2 nodes), 56.347337ms from start of poll, 57.424855ms from cluster start. Timeout at 2021-08-25 09:31:12.009738003 +0000 UTC m=+16.275954487, 105.118747ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.009Z [INFO] stable state for Follower reached at 2021-08-25 09:31:11.904619316 +0000 UTC m=+16.170835740 (2 nodes), 56.347337ms from start of poll, 57.424855ms from cluster start. Timeout at 2021-08-25 09:31:12.009738003 +0000 UTC m=+16.275954487, 105.118747ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.009Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:12.009Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.115Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.010038834 +0000 UTC m=+16.276255218 (1 nodes), 2.779µs from start of poll, 162.844333ms from cluster start. Timeout at 2021-08-25 09:31:12.115205356 +0000 UTC m=+16.381421820, 105.166602ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.115Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.010038834 +0000 UTC m=+16.276255218 (1 nodes), 2.779µs from start of poll, 162.844333ms from cluster start. Timeout at 2021-08-25 09:31:12.115205356 +0000 UTC m=+16.381421820, 105.166602ms after stability testLoggerAdapter verbose: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:12.115Z [WARN] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: rejecting vote request since we have a leader: from=cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 leader=ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 testing.go:149: server-f9eeae9a-4039-312d-7455-5b19e6edd20c: 2021-08-25T09:31:12.115Z [WARN] server-f9eeae9a-4039-312d-7455-5b19e6edd20c: rejecting vote request since we have a leader: from=cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 leader=ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12 testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:12.115Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: aborting pipeline replication: peer="{Voter server-f9eeae9a-4039-312d-7455-5b19e6edd20c f9eeae9a-4039-312d-7455-5b19e6edd20c}" testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:12.115Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: aborting pipeline replication: peer="{Voter server-f9eeae9a-4039-312d-7455-5b19e6edd20c f9eeae9a-4039-312d-7455-5b19e6edd20c}" testLoggerAdapter verbose: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:12.115Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: aborting pipeline replication: peer="{Voter server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 cfd7fc9e-5cf7-1217-f77d-dcfcad192f50}" testing.go:149: server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: 2021-08-25T09:31:12.115Z [INFO] server-ca1b40fe-fd32-2b2e-1f3e-0a8506fd8f12: aborting pipeline replication: peer="{Voter server-cfd7fc9e-5cf7-1217-f77d-dcfcad192f50 cfd7fc9e-5cf7-1217-f77d-dcfcad192f50}" --- PASS: TestRaft_Voting (0.27s) === RUN TestRaft_ProtocolVersion_RejectRPC testLoggerAdapter verbose: server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: 2021-08-25T09:31:12.119Z [INFO] server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 Address:32bff70f-0d8e-2a90-18bd-68e65c07a9a7} {Suffrage:Voter ID:server-8eb64893-06c0-8572-5ae3-e1cb27037597 Address:8eb64893-06c0-8572-5ae3-e1cb27037597} {Suffrage:Voter ID:server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 Address:2bbabbba-66b2-4e00-b66a-0d3ca2412fe9}]" testing.go:149: server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: 2021-08-25T09:31:12.119Z [INFO] server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 Address:32bff70f-0d8e-2a90-18bd-68e65c07a9a7} {Suffrage:Voter ID:server-8eb64893-06c0-8572-5ae3-e1cb27037597 Address:8eb64893-06c0-8572-5ae3-e1cb27037597} {Suffrage:Voter ID:server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 Address:2bbabbba-66b2-4e00-b66a-0d3ca2412fe9}]" testLoggerAdapter verbose: server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: 2021-08-25T09:31:12.120Z [INFO] server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: entering follower state: follower="Node at 32bff70f-0d8e-2a90-18bd-68e65c07a9a7 [Follower]" leader= testing.go:149: server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: 2021-08-25T09:31:12.120Z [INFO] server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7: entering follower state: follower="Node at 32bff70f-0d8e-2a90-18bd-68e65c07a9a7 [Follower]" leader= testLoggerAdapter verbose: server-8eb64893-06c0-8572-5ae3-e1cb27037597: 2021-08-25T09:31:12.120Z [INFO] server-8eb64893-06c0-8572-5ae3-e1cb27037597: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 Address:32bff70f-0d8e-2a90-18bd-68e65c07a9a7} {Suffrage:Voter ID:server-8eb64893-06c0-8572-5ae3-e1cb27037597 Address:8eb64893-06c0-8572-5ae3-e1cb27037597} {Suffrage:Voter ID:server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 Address:2bbabbba-66b2-4e00-b66a-0d3ca2412fe9}]" testing.go:149: server-8eb64893-06c0-8572-5ae3-e1cb27037597: 2021-08-25T09:31:12.120Z [INFO] server-8eb64893-06c0-8572-5ae3-e1cb27037597: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 Address:32bff70f-0d8e-2a90-18bd-68e65c07a9a7} {Suffrage:Voter ID:server-8eb64893-06c0-8572-5ae3-e1cb27037597 Address:8eb64893-06c0-8572-5ae3-e1cb27037597} {Suffrage:Voter ID:server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 Address:2bbabbba-66b2-4e00-b66a-0d3ca2412fe9}]" testLoggerAdapter verbose: server-8eb64893-06c0-8572-5ae3-e1cb27037597: 2021-08-25T09:31:12.120Z [INFO] server-8eb64893-06c0-8572-5ae3-e1cb27037597: entering follower state: follower="Node at 8eb64893-06c0-8572-5ae3-e1cb27037597 [Follower]" leader= testing.go:149: server-8eb64893-06c0-8572-5ae3-e1cb27037597: 2021-08-25T09:31:12.120Z [INFO] server-8eb64893-06c0-8572-5ae3-e1cb27037597: entering follower state: follower="Node at 8eb64893-06c0-8572-5ae3-e1cb27037597 [Follower]" leader= testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.120Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 Address:32bff70f-0d8e-2a90-18bd-68e65c07a9a7} {Suffrage:Voter ID:server-8eb64893-06c0-8572-5ae3-e1cb27037597 Address:8eb64893-06c0-8572-5ae3-e1cb27037597} {Suffrage:Voter ID:server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 Address:2bbabbba-66b2-4e00-b66a-0d3ca2412fe9}]" testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.120Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 Address:32bff70f-0d8e-2a90-18bd-68e65c07a9a7} {Suffrage:Voter ID:server-8eb64893-06c0-8572-5ae3-e1cb27037597 Address:8eb64893-06c0-8572-5ae3-e1cb27037597} {Suffrage:Voter ID:server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 Address:2bbabbba-66b2-4e00-b66a-0d3ca2412fe9}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.120Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:12.120Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.120Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: entering follower state: follower="Node at 2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 [Follower]" leader= testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.120Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: entering follower state: follower="Node at 2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 [Follower]" leader= testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.182Z [WARN] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.182Z [WARN] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.182Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: entering candidate state: node="Node at 2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 [Candidate]" term=2 testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.182Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: entering candidate state: node="Node at 2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 [Candidate]" term=2 testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: election won: tally=2 testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: election won: tally=2 testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: entering leader state: leader="Node at 2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 [Leader]" testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: entering leader state: leader="Node at 2bbabbba-66b2-4e00-b66a-0d3ca2412fe9 [Leader]" testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: added peer, starting replication: peer=server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: added peer, starting replication: peer=server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: added peer, starting replication: peer=server-8eb64893-06c0-8572-5ae3-e1cb27037597 testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: added peer, starting replication: peer=server-8eb64893-06c0-8572-5ae3-e1cb27037597 testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: pipelining replication: peer="{Voter server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 32bff70f-0d8e-2a90-18bd-68e65c07a9a7}" testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: pipelining replication: peer="{Voter server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 32bff70f-0d8e-2a90-18bd-68e65c07a9a7}" testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: pipelining replication: peer="{Voter server-8eb64893-06c0-8572-5ae3-e1cb27037597 8eb64893-06c0-8572-5ae3-e1cb27037597}" testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.183Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: pipelining replication: peer="{Voter server-8eb64893-06c0-8572-5ae3-e1cb27037597 8eb64893-06c0-8572-5ae3-e1cb27037597}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.288Z [INFO] stable state for Follower reached at 2021-08-25 09:31:12.183364902 +0000 UTC m=+16.449581345 (2 nodes), 62.641264ms from start of poll, 63.753099ms from cluster start. Timeout at 2021-08-25 09:31:12.288516505 +0000 UTC m=+16.554732909, 105.151564ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.288Z [INFO] stable state for Follower reached at 2021-08-25 09:31:12.183364902 +0000 UTC m=+16.449581345 (2 nodes), 62.641264ms from start of poll, 63.753099ms from cluster start. Timeout at 2021-08-25 09:31:12.288516505 +0000 UTC m=+16.554732909, 105.151564ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.288Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:12.288Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.394Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.288834695 +0000 UTC m=+16.555051099 (1 nodes), 2.28µs from start of poll, 169.222853ms from cluster start. Timeout at 2021-08-25 09:31:12.393888288 +0000 UTC m=+16.660104772, 105.053673ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.394Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.288834695 +0000 UTC m=+16.555051099 (1 nodes), 2.28µs from start of poll, 169.222853ms from cluster start. Timeout at 2021-08-25 09:31:12.393888288 +0000 UTC m=+16.660104772, 105.053673ms after stability testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.394Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: aborting pipeline replication: peer="{Voter server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 32bff70f-0d8e-2a90-18bd-68e65c07a9a7}" testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.394Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: aborting pipeline replication: peer="{Voter server-32bff70f-0d8e-2a90-18bd-68e65c07a9a7 32bff70f-0d8e-2a90-18bd-68e65c07a9a7}" testLoggerAdapter verbose: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.394Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: aborting pipeline replication: peer="{Voter server-8eb64893-06c0-8572-5ae3-e1cb27037597 8eb64893-06c0-8572-5ae3-e1cb27037597}" testing.go:149: server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: 2021-08-25T09:31:12.394Z [INFO] server-2bbabbba-66b2-4e00-b66a-0d3ca2412fe9: aborting pipeline replication: peer="{Voter server-8eb64893-06c0-8572-5ae3-e1cb27037597 8eb64893-06c0-8572-5ae3-e1cb27037597}" --- PASS: TestRaft_ProtocolVersion_RejectRPC (0.28s) === RUN TestRaft_ProtocolVersion_Upgrade_1_2 testLoggerAdapter verbose: 9bae2e09-24b9-2550-7023-7e8fcc774eff: 2021-08-25T09:31:12.397Z [INFO] 9bae2e09-24b9-2550-7023-7e8fcc774eff: initial configuration: index=1 servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5}]" testing.go:149: 9bae2e09-24b9-2550-7023-7e8fcc774eff: 2021-08-25T09:31:12.397Z [INFO] 9bae2e09-24b9-2550-7023-7e8fcc774eff: initial configuration: index=1 servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5}]" testLoggerAdapter verbose: 9bae2e09-24b9-2550-7023-7e8fcc774eff: 2021-08-25T09:31:12.397Z [INFO] 9bae2e09-24b9-2550-7023-7e8fcc774eff: entering follower state: follower="Node at 9bae2e09-24b9-2550-7023-7e8fcc774eff [Follower]" leader= testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.397Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: initial configuration: index=1 servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5}]" testing.go:149: 9bae2e09-24b9-2550-7023-7e8fcc774eff: 2021-08-25T09:31:12.397Z [INFO] 9bae2e09-24b9-2550-7023-7e8fcc774eff: entering follower state: follower="Node at 9bae2e09-24b9-2550-7023-7e8fcc774eff [Follower]" leader= testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.397Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: initial configuration: index=1 servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5}]" testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.398Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: entering follower state: follower="Node at ffd23400-f971-4802-10ed-1e16f93069d5 [Follower]" leader= testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.398Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: entering follower state: follower="Node at ffd23400-f971-4802-10ed-1e16f93069d5 [Follower]" leader= testLoggerAdapter verbose: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.398Z [INFO] 2263bd07-accf-402d-e2a8-6ed62da1cf25: initial configuration: index=0 servers=[] testing.go:149: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.398Z [INFO] 2263bd07-accf-402d-e2a8-6ed62da1cf25: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.398Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:12.398Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.398Z [INFO] 2263bd07-accf-402d-e2a8-6ed62da1cf25: entering follower state: follower="Node at 2263bd07-accf-402d-e2a8-6ed62da1cf25 [Follower]" leader= testing.go:149: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.398Z [INFO] 2263bd07-accf-402d-e2a8-6ed62da1cf25: entering follower state: follower="Node at 2263bd07-accf-402d-e2a8-6ed62da1cf25 [Follower]" leader= testLoggerAdapter verbose: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.459Z [WARN] 2263bd07-accf-402d-e2a8-6ed62da1cf25: no known peers, aborting election testing.go:149: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.459Z [WARN] 2263bd07-accf-402d-e2a8-6ed62da1cf25: no known peers, aborting election testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.470Z [WARN] ffd23400-f971-4802-10ed-1e16f93069d5: heartbeat timeout reached, starting election: last-leader= testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.470Z [WARN] ffd23400-f971-4802-10ed-1e16f93069d5: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.470Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: entering candidate state: node="Node at ffd23400-f971-4802-10ed-1e16f93069d5 [Candidate]" term=2 testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.470Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: entering candidate state: node="Node at ffd23400-f971-4802-10ed-1e16f93069d5 [Candidate]" term=2 testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.470Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: election won: tally=2 testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.470Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: election won: tally=2 testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.471Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: entering leader state: leader="Node at ffd23400-f971-4802-10ed-1e16f93069d5 [Leader]" testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.471Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: entering leader state: leader="Node at ffd23400-f971-4802-10ed-1e16f93069d5 [Leader]" testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.471Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: added peer, starting replication: peer=9bae2e09-24b9-2550-7023-7e8fcc774eff testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.471Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: added peer, starting replication: peer=9bae2e09-24b9-2550-7023-7e8fcc774eff testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.471Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: pipelining replication: peer="{Voter 9bae2e09-24b9-2550-7023-7e8fcc774eff 9bae2e09-24b9-2550-7023-7e8fcc774eff}" testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.471Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: pipelining replication: peer="{Voter 9bae2e09-24b9-2550-7023-7e8fcc774eff 9bae2e09-24b9-2550-7023-7e8fcc774eff}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.576Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.471196172 +0000 UTC m=+16.737412616 (1 nodes), 72.260859ms from start of poll, 73.818193ms from cluster start. Timeout at 2021-08-25 09:31:12.576347236 +0000 UTC m=+16.842563740, 105.151124ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.576Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.471196172 +0000 UTC m=+16.737412616 (1 nodes), 72.260859ms from start of poll, 73.818193ms from cluster start. Timeout at 2021-08-25 09:31:12.576347236 +0000 UTC m=+16.842563740, 105.151124ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.576Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:12.576Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.681Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.576652127 +0000 UTC m=+16.842868531 (1 nodes), 2.66µs from start of poll, 179.274108ms from cluster start. Timeout at 2021-08-25 09:31:12.681728798 +0000 UTC m=+16.947945362, 105.076831ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.681Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.576652127 +0000 UTC m=+16.842868531 (1 nodes), 2.66µs from start of poll, 179.274108ms from cluster start. Timeout at 2021-08-25 09:31:12.681728798 +0000 UTC m=+16.947945362, 105.076831ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.681Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:12.681Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.787Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.68201779 +0000 UTC m=+16.948234234 (1 nodes), 2.739µs from start of poll, 284.639811ms from cluster start. Timeout at 2021-08-25 09:31:12.787190412 +0000 UTC m=+17.053406916, 105.172682ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.787Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.68201779 +0000 UTC m=+16.948234234 (1 nodes), 2.739µs from start of poll, 284.639811ms from cluster start. Timeout at 2021-08-25 09:31:12.787190412 +0000 UTC m=+17.053406916, 105.172682ms after stability testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.787Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: updating configuration: command=AddStaging server-id=2263bd07-accf-402d-e2a8-6ed62da1cf25 server-addr=2263bd07-accf-402d-e2a8-6ed62da1cf25 servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5} {Suffrage:Voter ID:2263bd07-accf-402d-e2a8-6ed62da1cf25 Address:2263bd07-accf-402d-e2a8-6ed62da1cf25}]" testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.787Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: updating configuration: command=AddStaging server-id=2263bd07-accf-402d-e2a8-6ed62da1cf25 server-addr=2263bd07-accf-402d-e2a8-6ed62da1cf25 servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5} {Suffrage:Voter ID:2263bd07-accf-402d-e2a8-6ed62da1cf25 Address:2263bd07-accf-402d-e2a8-6ed62da1cf25}]" testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.787Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: added peer, starting replication: peer=2263bd07-accf-402d-e2a8-6ed62da1cf25 testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.787Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: added peer, starting replication: peer=2263bd07-accf-402d-e2a8-6ed62da1cf25 testLoggerAdapter verbose: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.787Z [WARN] 2263bd07-accf-402d-e2a8-6ed62da1cf25: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: 2263bd07-accf-402d-e2a8-6ed62da1cf25: 2021-08-25T09:31:12.787Z [WARN] 2263bd07-accf-402d-e2a8-6ed62da1cf25: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.788Z [WARN] ffd23400-f971-4802-10ed-1e16f93069d5: appendEntries rejected, sending older logs: peer="{Voter 2263bd07-accf-402d-e2a8-6ed62da1cf25 2263bd07-accf-402d-e2a8-6ed62da1cf25}" next=1 testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.788Z [WARN] ffd23400-f971-4802-10ed-1e16f93069d5: appendEntries rejected, sending older logs: peer="{Voter 2263bd07-accf-402d-e2a8-6ed62da1cf25 2263bd07-accf-402d-e2a8-6ed62da1cf25}" next=1 testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.788Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: pipelining replication: peer="{Voter 2263bd07-accf-402d-e2a8-6ed62da1cf25 2263bd07-accf-402d-e2a8-6ed62da1cf25}" testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:12.788Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: pipelining replication: peer="{Voter 2263bd07-accf-402d-e2a8-6ed62da1cf25 2263bd07-accf-402d-e2a8-6ed62da1cf25}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.793Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:12.793Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.898Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.793260501 +0000 UTC m=+17.059476905 (1 nodes), 2.3µs from start of poll, 395.882482ms from cluster start. Timeout at 2021-08-25 09:31:12.898397866 +0000 UTC m=+17.164614330, 105.137425ms after stability testing.go:149: cluster: 2021-08-25T09:31:12.898Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.793260501 +0000 UTC m=+17.059476905 (1 nodes), 2.3µs from start of poll, 395.882482ms from cluster start. Timeout at 2021-08-25 09:31:12.898397866 +0000 UTC m=+17.164614330, 105.137425ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:12.898Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:12.898Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.003Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.898679179 +0000 UTC m=+17.164895583 (1 nodes), 2.12µs from start of poll, 501.30116ms from cluster start. Timeout at 2021-08-25 09:31:13.003697815 +0000 UTC m=+17.269914259, 105.018676ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.003Z [INFO] stable state for Leader reached at 2021-08-25 09:31:12.898679179 +0000 UTC m=+17.164895583 (1 nodes), 2.12µs from start of poll, 501.30116ms from cluster start. Timeout at 2021-08-25 09:31:13.003697815 +0000 UTC m=+17.269914259, 105.018676ms after stability testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: updating configuration: command=RemoveServer server-id=2263bd07-accf-402d-e2a8-6ed62da1cf25 server-addr= servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5}]" testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: updating configuration: command=RemoveServer server-id=2263bd07-accf-402d-e2a8-6ed62da1cf25 server-addr= servers="[{Suffrage:Voter ID:9bae2e09-24b9-2550-7023-7e8fcc774eff Address:9bae2e09-24b9-2550-7023-7e8fcc774eff} {Suffrage:Voter ID:ffd23400-f971-4802-10ed-1e16f93069d5 Address:ffd23400-f971-4802-10ed-1e16f93069d5}]" testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: removed peer, stopping replication: peer=2263bd07-accf-402d-e2a8-6ed62da1cf25 last-index=4 testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: removed peer, stopping replication: peer=2263bd07-accf-402d-e2a8-6ed62da1cf25 last-index=4 testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: aborting pipeline replication: peer="{Voter 2263bd07-accf-402d-e2a8-6ed62da1cf25 2263bd07-accf-402d-e2a8-6ed62da1cf25}" testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: aborting pipeline replication: peer="{Voter 2263bd07-accf-402d-e2a8-6ed62da1cf25 2263bd07-accf-402d-e2a8-6ed62da1cf25}" testLoggerAdapter verbose: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: aborting pipeline replication: peer="{Voter 9bae2e09-24b9-2550-7023-7e8fcc774eff 9bae2e09-24b9-2550-7023-7e8fcc774eff}" testing.go:149: ffd23400-f971-4802-10ed-1e16f93069d5: 2021-08-25T09:31:13.004Z [INFO] ffd23400-f971-4802-10ed-1e16f93069d5: aborting pipeline replication: peer="{Voter 9bae2e09-24b9-2550-7023-7e8fcc774eff 9bae2e09-24b9-2550-7023-7e8fcc774eff}" --- PASS: TestRaft_ProtocolVersion_Upgrade_1_2 (0.61s) === RUN TestRaft_ProtocolVersion_Upgrade_2_3 testLoggerAdapter verbose: 4149dc47-5924-ffe4-78f2-b35df55172f5: 2021-08-25T09:31:13.008Z [INFO] 4149dc47-5924-ffe4-78f2-b35df55172f5: initial configuration: index=1 servers="[{Suffrage:Voter ID:4149dc47-5924-ffe4-78f2-b35df55172f5 Address:4149dc47-5924-ffe4-78f2-b35df55172f5} {Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de}]" testing.go:149: 4149dc47-5924-ffe4-78f2-b35df55172f5: 2021-08-25T09:31:13.008Z [INFO] 4149dc47-5924-ffe4-78f2-b35df55172f5: initial configuration: index=1 servers="[{Suffrage:Voter ID:4149dc47-5924-ffe4-78f2-b35df55172f5 Address:4149dc47-5924-ffe4-78f2-b35df55172f5} {Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de}]" testLoggerAdapter verbose: 4149dc47-5924-ffe4-78f2-b35df55172f5: 2021-08-25T09:31:13.008Z [INFO] 4149dc47-5924-ffe4-78f2-b35df55172f5: entering follower state: follower="Node at 4149dc47-5924-ffe4-78f2-b35df55172f5 [Follower]" leader= testing.go:149: 4149dc47-5924-ffe4-78f2-b35df55172f5: 2021-08-25T09:31:13.008Z [INFO] 4149dc47-5924-ffe4-78f2-b35df55172f5: entering follower state: follower="Node at 4149dc47-5924-ffe4-78f2-b35df55172f5 [Follower]" leader= testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.008Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: initial configuration: index=1 servers="[{Suffrage:Voter ID:4149dc47-5924-ffe4-78f2-b35df55172f5 Address:4149dc47-5924-ffe4-78f2-b35df55172f5} {Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de}]" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.008Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: initial configuration: index=1 servers="[{Suffrage:Voter ID:4149dc47-5924-ffe4-78f2-b35df55172f5 Address:4149dc47-5924-ffe4-78f2-b35df55172f5} {Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.008Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:13.008Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.008Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: entering follower state: follower="Node at b7e00268-87d5-10b0-607a-c947482187de [Follower]" leader= testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.008Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: entering follower state: follower="Node at b7e00268-87d5-10b0-607a-c947482187de [Follower]" leader= testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [WARN] b7e00268-87d5-10b0-607a-c947482187de: heartbeat timeout reached, starting election: last-leader= testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [WARN] b7e00268-87d5-10b0-607a-c947482187de: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: entering candidate state: node="Node at b7e00268-87d5-10b0-607a-c947482187de [Candidate]" term=2 testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: entering candidate state: node="Node at b7e00268-87d5-10b0-607a-c947482187de [Candidate]" term=2 testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: election won: tally=2 testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: election won: tally=2 testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: entering leader state: leader="Node at b7e00268-87d5-10b0-607a-c947482187de [Leader]" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: entering leader state: leader="Node at b7e00268-87d5-10b0-607a-c947482187de [Leader]" testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: added peer, starting replication: peer=4149dc47-5924-ffe4-78f2-b35df55172f5 testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: added peer, starting replication: peer=4149dc47-5924-ffe4-78f2-b35df55172f5 testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: pipelining replication: peer="{Voter 4149dc47-5924-ffe4-78f2-b35df55172f5 4149dc47-5924-ffe4-78f2-b35df55172f5}" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.100Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: pipelining replication: peer="{Voter 4149dc47-5924-ffe4-78f2-b35df55172f5 4149dc47-5924-ffe4-78f2-b35df55172f5}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.205Z [INFO] stable state for Follower reached at 2021-08-25 09:31:13.100624013 +0000 UTC m=+17.366840417 (1 nodes), 92.11875ms from start of poll, 92.781348ms from cluster start. Timeout at 2021-08-25 09:31:13.205756038 +0000 UTC m=+17.471972502, 105.132085ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.205Z [INFO] stable state for Follower reached at 2021-08-25 09:31:13.100624013 +0000 UTC m=+17.366840417 (1 nodes), 92.11875ms from start of poll, 92.781348ms from cluster start. Timeout at 2021-08-25 09:31:13.205756038 +0000 UTC m=+17.471972502, 105.132085ms after stability testLoggerAdapter verbose: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.206Z [INFO] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: initial configuration: index=0 servers=[] testing.go:149: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.206Z [INFO] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.207Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:13.207Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.207Z [INFO] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: entering follower state: follower="Node at 92450e97-8bc9-f45e-0f54-459f2edf8aa1 [Follower]" leader= testing.go:149: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.207Z [INFO] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: entering follower state: follower="Node at 92450e97-8bc9-f45e-0f54-459f2edf8aa1 [Follower]" leader= testLoggerAdapter verbose: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.265Z [WARN] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: no known peers, aborting election testing.go:149: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.265Z [WARN] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: no known peers, aborting election testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.312Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.207170205 +0000 UTC m=+17.473386709 (1 nodes), 33.637µs from start of poll, 199.32764ms from cluster start. Timeout at 2021-08-25 09:31:13.312302451 +0000 UTC m=+17.578518915, 105.132206ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.312Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.207170205 +0000 UTC m=+17.473386709 (1 nodes), 33.637µs from start of poll, 199.32764ms from cluster start. Timeout at 2021-08-25 09:31:13.312302451 +0000 UTC m=+17.578518915, 105.132206ms after stability testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.312Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: updating configuration: command=AddStaging server-id=server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 server-addr=92450e97-8bc9-f45e-0f54-459f2edf8aa1 servers="[{Suffrage:Voter ID:4149dc47-5924-ffe4-78f2-b35df55172f5 Address:4149dc47-5924-ffe4-78f2-b35df55172f5} {Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de} {Suffrage:Voter ID:server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 Address:92450e97-8bc9-f45e-0f54-459f2edf8aa1}]" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.312Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: updating configuration: command=AddStaging server-id=server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 server-addr=92450e97-8bc9-f45e-0f54-459f2edf8aa1 servers="[{Suffrage:Voter ID:4149dc47-5924-ffe4-78f2-b35df55172f5 Address:4149dc47-5924-ffe4-78f2-b35df55172f5} {Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de} {Suffrage:Voter ID:server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 Address:92450e97-8bc9-f45e-0f54-459f2edf8aa1}]" testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.312Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: added peer, starting replication: peer=server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.312Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: added peer, starting replication: peer=server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 testLoggerAdapter verbose: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.312Z [WARN] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: 2021-08-25T09:31:13.312Z [WARN] server-92450e97-8bc9-f45e-0f54-459f2edf8aa1: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.313Z [WARN] b7e00268-87d5-10b0-607a-c947482187de: appendEntries rejected, sending older logs: peer="{Voter server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 92450e97-8bc9-f45e-0f54-459f2edf8aa1}" next=1 testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.313Z [WARN] b7e00268-87d5-10b0-607a-c947482187de: appendEntries rejected, sending older logs: peer="{Voter server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 92450e97-8bc9-f45e-0f54-459f2edf8aa1}" next=1 testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.313Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: pipelining replication: peer="{Voter server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 92450e97-8bc9-f45e-0f54-459f2edf8aa1}" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.313Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: pipelining replication: peer="{Voter server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 92450e97-8bc9-f45e-0f54-459f2edf8aa1}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.318Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:13.318Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.423Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.318405976 +0000 UTC m=+17.584622380 (1 nodes), 2.36µs from start of poll, 310.563311ms from cluster start. Timeout at 2021-08-25 09:31:13.42355154 +0000 UTC m=+17.689768044, 105.145664ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.423Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.318405976 +0000 UTC m=+17.584622380 (1 nodes), 2.36µs from start of poll, 310.563311ms from cluster start. Timeout at 2021-08-25 09:31:13.42355154 +0000 UTC m=+17.689768044, 105.145664ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.423Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:13.423Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.529Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.423892908 +0000 UTC m=+17.690109292 (1 nodes), 2.38µs from start of poll, 416.050223ms from cluster start. Timeout at 2021-08-25 09:31:13.529041372 +0000 UTC m=+17.795257836, 105.148544ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.529Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.423892908 +0000 UTC m=+17.690109292 (1 nodes), 2.38µs from start of poll, 416.050223ms from cluster start. Timeout at 2021-08-25 09:31:13.529041372 +0000 UTC m=+17.795257836, 105.148544ms after stability testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: updating configuration: command=RemoveServer server-id=4149dc47-5924-ffe4-78f2-b35df55172f5 server-addr= servers="[{Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de} {Suffrage:Voter ID:server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 Address:92450e97-8bc9-f45e-0f54-459f2edf8aa1}]" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: updating configuration: command=RemoveServer server-id=4149dc47-5924-ffe4-78f2-b35df55172f5 server-addr= servers="[{Suffrage:Voter ID:b7e00268-87d5-10b0-607a-c947482187de Address:b7e00268-87d5-10b0-607a-c947482187de} {Suffrage:Voter ID:server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 Address:92450e97-8bc9-f45e-0f54-459f2edf8aa1}]" testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: removed peer, stopping replication: peer=4149dc47-5924-ffe4-78f2-b35df55172f5 last-index=4 testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: removed peer, stopping replication: peer=4149dc47-5924-ffe4-78f2-b35df55172f5 last-index=4 testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: aborting pipeline replication: peer="{Voter 4149dc47-5924-ffe4-78f2-b35df55172f5 4149dc47-5924-ffe4-78f2-b35df55172f5}" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: aborting pipeline replication: peer="{Voter 4149dc47-5924-ffe4-78f2-b35df55172f5 4149dc47-5924-ffe4-78f2-b35df55172f5}" testLoggerAdapter verbose: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: aborting pipeline replication: peer="{Voter server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 92450e97-8bc9-f45e-0f54-459f2edf8aa1}" testing.go:149: b7e00268-87d5-10b0-607a-c947482187de: 2021-08-25T09:31:13.529Z [INFO] b7e00268-87d5-10b0-607a-c947482187de: aborting pipeline replication: peer="{Voter server-92450e97-8bc9-f45e-0f54-459f2edf8aa1 92450e97-8bc9-f45e-0f54-459f2edf8aa1}" --- PASS: TestRaft_ProtocolVersion_Upgrade_2_3 (0.52s) === RUN TestRaft_LeadershipTransferInProgress --- PASS: TestRaft_LeadershipTransferInProgress (0.00s) === RUN TestRaft_LeadershipTransferPickServer --- PASS: TestRaft_LeadershipTransferPickServer (0.00s) === RUN TestRaft_LeadershipTransfer testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.533Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 Address:737f9c17-f66c-2185-c80b-e9c02e1a04b5} {Suffrage:Voter ID:server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 Address:2661fbe6-8288-be2d-1199-d0e4b66b4a35} {Suffrage:Voter ID:server-c4a599c6-bd42-5ade-4559-1bdf940e5492 Address:c4a599c6-bd42-5ade-4559-1bdf940e5492}]" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.533Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 Address:737f9c17-f66c-2185-c80b-e9c02e1a04b5} {Suffrage:Voter ID:server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 Address:2661fbe6-8288-be2d-1199-d0e4b66b4a35} {Suffrage:Voter ID:server-c4a599c6-bd42-5ade-4559-1bdf940e5492 Address:c4a599c6-bd42-5ade-4559-1bdf940e5492}]" testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.533Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: entering follower state: follower="Node at 737f9c17-f66c-2185-c80b-e9c02e1a04b5 [Follower]" leader= testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.533Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: entering follower state: follower="Node at 737f9c17-f66c-2185-c80b-e9c02e1a04b5 [Follower]" leader= testLoggerAdapter verbose: server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: 2021-08-25T09:31:13.534Z [INFO] server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 Address:737f9c17-f66c-2185-c80b-e9c02e1a04b5} {Suffrage:Voter ID:server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 Address:2661fbe6-8288-be2d-1199-d0e4b66b4a35} {Suffrage:Voter ID:server-c4a599c6-bd42-5ade-4559-1bdf940e5492 Address:c4a599c6-bd42-5ade-4559-1bdf940e5492}]" testing.go:149: server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: 2021-08-25T09:31:13.534Z [INFO] server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 Address:737f9c17-f66c-2185-c80b-e9c02e1a04b5} {Suffrage:Voter ID:server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 Address:2661fbe6-8288-be2d-1199-d0e4b66b4a35} {Suffrage:Voter ID:server-c4a599c6-bd42-5ade-4559-1bdf940e5492 Address:c4a599c6-bd42-5ade-4559-1bdf940e5492}]" testLoggerAdapter verbose: server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: 2021-08-25T09:31:13.534Z [INFO] server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: entering follower state: follower="Node at 2661fbe6-8288-be2d-1199-d0e4b66b4a35 [Follower]" leader= testing.go:149: server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: 2021-08-25T09:31:13.534Z [INFO] server-2661fbe6-8288-be2d-1199-d0e4b66b4a35: entering follower state: follower="Node at 2661fbe6-8288-be2d-1199-d0e4b66b4a35 [Follower]" leader= testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.534Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 Address:737f9c17-f66c-2185-c80b-e9c02e1a04b5} {Suffrage:Voter ID:server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 Address:2661fbe6-8288-be2d-1199-d0e4b66b4a35} {Suffrage:Voter ID:server-c4a599c6-bd42-5ade-4559-1bdf940e5492 Address:c4a599c6-bd42-5ade-4559-1bdf940e5492}]" testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.534Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 Address:737f9c17-f66c-2185-c80b-e9c02e1a04b5} {Suffrage:Voter ID:server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 Address:2661fbe6-8288-be2d-1199-d0e4b66b4a35} {Suffrage:Voter ID:server-c4a599c6-bd42-5ade-4559-1bdf940e5492 Address:c4a599c6-bd42-5ade-4559-1bdf940e5492}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.534Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:13.534Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.534Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering follower state: follower="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Follower]" leader= testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.534Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering follower state: follower="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Follower]" leader= testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [WARN] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [WARN] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering candidate state: node="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Candidate]" term=2 testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering candidate state: node="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Candidate]" term=2 testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: election won: tally=2 testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: election won: tally=2 testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering leader state: leader="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Leader]" testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.587Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering leader state: leader="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Leader]" testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: added peer, starting replication: peer=server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: added peer, starting replication: peer=server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: added peer, starting replication: peer=server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: added peer, starting replication: peer=server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: pipelining replication: peer="{Voter server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 737f9c17-f66c-2185-c80b-e9c02e1a04b5}" testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: pipelining replication: peer="{Voter server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 737f9c17-f66c-2185-c80b-e9c02e1a04b5}" testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: pipelining replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.588Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: pipelining replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.693Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.588074716 +0000 UTC m=+17.854291180 (1 nodes), 53.377456ms from start of poll, 54.658176ms from cluster start. Timeout at 2021-08-25 09:31:13.693187743 +0000 UTC m=+17.959404147, 105.112967ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.693Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.588074716 +0000 UTC m=+17.854291180 (1 nodes), 53.377456ms from start of poll, 54.658176ms from cluster start. Timeout at 2021-08-25 09:31:13.693187743 +0000 UTC m=+17.959404147, 105.112967ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.693Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:13.693Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.798Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.69343666 +0000 UTC m=+17.959653064 (1 nodes), 2.56µs from start of poll, 160.02006ms from cluster start. Timeout at 2021-08-25 09:31:13.798559386 +0000 UTC m=+18.064775850, 105.122786ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.798Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.69343666 +0000 UTC m=+17.959653064 (1 nodes), 2.56µs from start of poll, 160.02006ms from cluster start. Timeout at 2021-08-25 09:31:13.798559386 +0000 UTC m=+18.064775850, 105.122786ms after stability testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.798Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: entering candidate state: node="Node at 737f9c17-f66c-2185-c80b-e9c02e1a04b5 [Candidate]" term=3 testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.798Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: entering candidate state: node="Node at 737f9c17-f66c-2185-c80b-e9c02e1a04b5 [Candidate]" term=3 testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.799Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering follower state: follower="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Follower]" leader= testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.799Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.799Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: entering follower state: follower="Node at c4a599c6-bd42-5ade-4559-1bdf940e5492 [Follower]" leader= testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: election won: tally=2 testing.go:149: cluster: 2021-08-25T09:31:13.799Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.799Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: aborting pipeline replication: peer="{Voter server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 737f9c17-f66c-2185-c80b-e9c02e1a04b5}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: election won: tally=2 testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: entering leader state: leader="Node at 737f9c17-f66c-2185-c80b-e9c02e1a04b5 [Leader]" testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.799Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: aborting pipeline replication: peer="{Voter server-737f9c17-f66c-2185-c80b-e9c02e1a04b5 737f9c17-f66c-2185-c80b-e9c02e1a04b5}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: entering leader state: leader="Node at 737f9c17-f66c-2185-c80b-e9c02e1a04b5 [Leader]" testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: added peer, starting replication: peer=server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 testLoggerAdapter verbose: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.799Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: aborting pipeline replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: added peer, starting replication: peer=server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: added peer, starting replication: peer=server-c4a599c6-bd42-5ade-4559-1bdf940e5492 testing.go:149: server-c4a599c6-bd42-5ade-4559-1bdf940e5492: 2021-08-25T09:31:13.799Z [INFO] server-c4a599c6-bd42-5ade-4559-1bdf940e5492: aborting pipeline replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: added peer, starting replication: peer=server-c4a599c6-bd42-5ade-4559-1bdf940e5492 testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: pipelining replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: pipelining replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: pipelining replication: peer="{Voter server-c4a599c6-bd42-5ade-4559-1bdf940e5492 c4a599c6-bd42-5ade-4559-1bdf940e5492}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.799Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: pipelining replication: peer="{Voter server-c4a599c6-bd42-5ade-4559-1bdf940e5492 c4a599c6-bd42-5ade-4559-1bdf940e5492}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.904Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.799195906 +0000 UTC m=+18.065412290 (1 nodes), 79.492µs from start of poll, 265.779286ms from cluster start. Timeout at 2021-08-25 09:31:13.904354069 +0000 UTC m=+18.170570513, 105.158223ms after stability testing.go:149: cluster: 2021-08-25T09:31:13.904Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.799195906 +0000 UTC m=+18.065412290 (1 nodes), 79.492µs from start of poll, 265.779286ms from cluster start. Timeout at 2021-08-25 09:31:13.904354069 +0000 UTC m=+18.170570513, 105.158223ms after stability testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.904Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: aborting pipeline replication: peer="{Voter server-c4a599c6-bd42-5ade-4559-1bdf940e5492 c4a599c6-bd42-5ade-4559-1bdf940e5492}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.904Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: aborting pipeline replication: peer="{Voter server-c4a599c6-bd42-5ade-4559-1bdf940e5492 c4a599c6-bd42-5ade-4559-1bdf940e5492}" testLoggerAdapter verbose: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.904Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: aborting pipeline replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" testing.go:149: server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: 2021-08-25T09:31:13.904Z [INFO] server-737f9c17-f66c-2185-c80b-e9c02e1a04b5: aborting pipeline replication: peer="{Voter server-2661fbe6-8288-be2d-1199-d0e4b66b4a35 2661fbe6-8288-be2d-1199-d0e4b66b4a35}" --- PASS: TestRaft_LeadershipTransfer (0.37s) === RUN TestRaft_LeadershipTransferWithOneNode testLoggerAdapter verbose: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.908Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5 Address:d1ec148f-980d-2f1d-5f32-05ba79c50fe5}]" testing.go:149: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.908Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5 Address:d1ec148f-980d-2f1d-5f32-05ba79c50fe5}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:13.909Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:13.909Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.909Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: entering follower state: follower="Node at d1ec148f-980d-2f1d-5f32-05ba79c50fe5 [Follower]" leader= testing.go:149: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.909Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: entering follower state: follower="Node at d1ec148f-980d-2f1d-5f32-05ba79c50fe5 [Follower]" leader= testLoggerAdapter verbose: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [WARN] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [WARN] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: entering candidate state: node="Node at d1ec148f-980d-2f1d-5f32-05ba79c50fe5 [Candidate]" term=2 testing.go:149: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: entering candidate state: node="Node at d1ec148f-980d-2f1d-5f32-05ba79c50fe5 [Candidate]" term=2 testLoggerAdapter verbose: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: election won: tally=1 testing.go:149: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: election won: tally=1 testLoggerAdapter verbose: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: entering leader state: leader="Node at d1ec148f-980d-2f1d-5f32-05ba79c50fe5 [Leader]" testing.go:149: server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: 2021-08-25T09:31:13.970Z [INFO] server-d1ec148f-980d-2f1d-5f32-05ba79c50fe5: entering leader state: leader="Node at d1ec148f-980d-2f1d-5f32-05ba79c50fe5 [Leader]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.076Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.970809515 +0000 UTC m=+18.237025919 (1 nodes), 61.708592ms from start of poll, 62.735436ms from cluster start. Timeout at 2021-08-25 09:31:14.075931321 +0000 UTC m=+18.342147765, 105.121846ms after stability testing.go:149: cluster: 2021-08-25T09:31:14.076Z [INFO] stable state for Leader reached at 2021-08-25 09:31:13.970809515 +0000 UTC m=+18.237025919 (1 nodes), 61.708592ms from start of poll, 62.735436ms from cluster start. Timeout at 2021-08-25 09:31:14.075931321 +0000 UTC m=+18.342147765, 105.121846ms after stability --- PASS: TestRaft_LeadershipTransferWithOneNode (0.17s) === RUN TestRaft_LeadershipTransferWithSevenNodes testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.081Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.081Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.081Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: entering follower state: follower="Node at 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 [Follower]" leader= testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.081Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: entering follower state: follower="Node at 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 [Follower]" leader= testLoggerAdapter verbose: server-41e0b663-49fe-ffe9-c040-225723f99c1a: 2021-08-25T09:31:14.082Z [INFO] server-41e0b663-49fe-ffe9-c040-225723f99c1a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testing.go:149: server-41e0b663-49fe-ffe9-c040-225723f99c1a: 2021-08-25T09:31:14.082Z [INFO] server-41e0b663-49fe-ffe9-c040-225723f99c1a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testLoggerAdapter verbose: server-41e0b663-49fe-ffe9-c040-225723f99c1a: 2021-08-25T09:31:14.082Z [INFO] server-41e0b663-49fe-ffe9-c040-225723f99c1a: entering follower state: follower="Node at 41e0b663-49fe-ffe9-c040-225723f99c1a [Follower]" leader= testing.go:149: server-41e0b663-49fe-ffe9-c040-225723f99c1a: 2021-08-25T09:31:14.082Z [INFO] server-41e0b663-49fe-ffe9-c040-225723f99c1a: entering follower state: follower="Node at 41e0b663-49fe-ffe9-c040-225723f99c1a [Follower]" leader= testLoggerAdapter verbose: server-588a5d96-ef8f-731a-9c27-b4567203686d: 2021-08-25T09:31:14.082Z [INFO] server-588a5d96-ef8f-731a-9c27-b4567203686d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testing.go:149: server-588a5d96-ef8f-731a-9c27-b4567203686d: 2021-08-25T09:31:14.082Z [INFO] server-588a5d96-ef8f-731a-9c27-b4567203686d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testLoggerAdapter verbose: server-588a5d96-ef8f-731a-9c27-b4567203686d: 2021-08-25T09:31:14.082Z [INFO] server-588a5d96-ef8f-731a-9c27-b4567203686d: entering follower state: follower="Node at 588a5d96-ef8f-731a-9c27-b4567203686d [Follower]" leader= testing.go:149: server-588a5d96-ef8f-731a-9c27-b4567203686d: 2021-08-25T09:31:14.082Z [INFO] server-588a5d96-ef8f-731a-9c27-b4567203686d: entering follower state: follower="Node at 588a5d96-ef8f-731a-9c27-b4567203686d [Follower]" leader= testLoggerAdapter verbose: server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: 2021-08-25T09:31:14.082Z [INFO] server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testing.go:149: server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: 2021-08-25T09:31:14.082Z [INFO] server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testLoggerAdapter verbose: server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: 2021-08-25T09:31:14.083Z [INFO] server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: entering follower state: follower="Node at 1fe4a98f-3e54-1fd5-cf32-62fafa19c811 [Follower]" leader= testing.go:149: server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: 2021-08-25T09:31:14.083Z [INFO] server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811: entering follower state: follower="Node at 1fe4a98f-3e54-1fd5-cf32-62fafa19c811 [Follower]" leader= testLoggerAdapter verbose: server-2003ea67-0ece-50a0-7a36-e2804e724e41: 2021-08-25T09:31:14.083Z [INFO] server-2003ea67-0ece-50a0-7a36-e2804e724e41: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testing.go:149: server-2003ea67-0ece-50a0-7a36-e2804e724e41: 2021-08-25T09:31:14.083Z [INFO] server-2003ea67-0ece-50a0-7a36-e2804e724e41: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testLoggerAdapter verbose: server-2003ea67-0ece-50a0-7a36-e2804e724e41: 2021-08-25T09:31:14.083Z [INFO] server-2003ea67-0ece-50a0-7a36-e2804e724e41: entering follower state: follower="Node at 2003ea67-0ece-50a0-7a36-e2804e724e41 [Follower]" leader= testing.go:149: server-2003ea67-0ece-50a0-7a36-e2804e724e41: 2021-08-25T09:31:14.083Z [INFO] server-2003ea67-0ece-50a0-7a36-e2804e724e41: entering follower state: follower="Node at 2003ea67-0ece-50a0-7a36-e2804e724e41 [Follower]" leader= testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.083Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.083Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.083Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering follower state: follower="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Follower]" leader= testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.083Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering follower state: follower="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Follower]" leader= testLoggerAdapter verbose: server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: 2021-08-25T09:31:14.084Z [INFO] server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testing.go:149: server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: 2021-08-25T09:31:14.084Z [INFO] server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 Address:42c1f716-2b47-4f4e-80a2-3ca8cc11dea6} {Suffrage:Voter ID:server-41e0b663-49fe-ffe9-c040-225723f99c1a Address:41e0b663-49fe-ffe9-c040-225723f99c1a} {Suffrage:Voter ID:server-588a5d96-ef8f-731a-9c27-b4567203686d Address:588a5d96-ef8f-731a-9c27-b4567203686d} {Suffrage:Voter ID:server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 Address:1fe4a98f-3e54-1fd5-cf32-62fafa19c811} {Suffrage:Voter ID:server-2003ea67-0ece-50a0-7a36-e2804e724e41 Address:2003ea67-0ece-50a0-7a36-e2804e724e41} {Suffrage:Voter ID:server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 Address:31286ec1-9bfe-8cf5-f555-4af451e5a6b2} {Suffrage:Voter ID:server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 Address:acdc9fbe-34fc-f5e5-5372-579b9c5ae972}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.084Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: 2021-08-25T09:31:14.084Z [INFO] server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: entering follower state: follower="Node at acdc9fbe-34fc-f5e5-5372-579b9c5ae972 [Follower]" leader= testing.go:149: cluster: 2021-08-25T09:31:14.084Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: 2021-08-25T09:31:14.084Z [INFO] server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972: entering follower state: follower="Node at acdc9fbe-34fc-f5e5-5372-579b9c5ae972 [Follower]" leader= testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.135Z [WARN] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.135Z [WARN] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.135Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering candidate state: node="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Candidate]" term=2 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.135Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering candidate state: node="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Candidate]" term=2 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.135Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: election won: tally=4 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.135Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: election won: tally=4 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering leader state: leader="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Leader]" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering leader state: leader="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Leader]" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-41e0b663-49fe-ffe9-c040-225723f99c1a testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-41e0b663-49fe-ffe9-c040-225723f99c1a testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-588a5d96-ef8f-731a-9c27-b4567203686d testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-588a5d96-ef8f-731a-9c27-b4567203686d testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-2003ea67-0ece-50a0-7a36-e2804e724e41 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-2003ea67-0ece-50a0-7a36-e2804e724e41 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: added peer, starting replication: peer=server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.136Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: pipelining replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.241Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.136097439 +0000 UTC m=+18.402313883 (1 nodes), 51.8555ms from start of poll, 54.802682ms from cluster start. Timeout at 2021-08-25 09:31:14.241154991 +0000 UTC m=+18.507371395, 105.057512ms after stability testing.go:149: cluster: 2021-08-25T09:31:14.241Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.136097439 +0000 UTC m=+18.402313883 (1 nodes), 51.8555ms from start of poll, 54.802682ms from cluster start. Timeout at 2021-08-25 09:31:14.241154991 +0000 UTC m=+18.507371395, 105.057512ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.241Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:14.241Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.346Z [INFO] stable state for Follower reached at 2021-08-25 09:31:14.241418166 +0000 UTC m=+18.507634570 (6 nodes), 4.959µs from start of poll, 160.123369ms from cluster start. Timeout at 2021-08-25 09:31:14.346476679 +0000 UTC m=+18.612693083, 105.058513ms after stability testing.go:149: cluster: 2021-08-25T09:31:14.346Z [INFO] stable state for Follower reached at 2021-08-25 09:31:14.241418166 +0000 UTC m=+18.507634570 (6 nodes), 4.959µs from start of poll, 160.123369ms from cluster start. Timeout at 2021-08-25 09:31:14.346476679 +0000 UTC m=+18.612693083, 105.058513ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.346Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:14.346Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.452Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.346753113 +0000 UTC m=+18.612969557 (1 nodes), 3.44µs from start of poll, 265.458356ms from cluster start. Timeout at 2021-08-25 09:31:14.451965991 +0000 UTC m=+18.718182475, 105.212918ms after stability testing.go:149: cluster: 2021-08-25T09:31:14.452Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.346753113 +0000 UTC m=+18.612969557 (1 nodes), 3.44µs from start of poll, 265.458356ms from cluster start. Timeout at 2021-08-25 09:31:14.451965991 +0000 UTC m=+18.718182475, 105.212918ms after stability testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: entering candidate state: node="Node at 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 [Candidate]" term=3 testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.452Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: entering candidate state: node="Node at 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 [Candidate]" term=3 testing.go:149: cluster: 2021-08-25T09:31:14.452Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: election won: tally=4 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering follower state: follower="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Follower]" leader= testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: election won: tally=4 testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: entering leader state: leader="Node at 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 [Leader]" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: entering leader state: leader="Node at 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 [Leader]" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-41e0b663-49fe-ffe9-c040-225723f99c1a testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-41e0b663-49fe-ffe9-c040-225723f99c1a testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-588a5d96-ef8f-731a-9c27-b4567203686d testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: entering follower state: follower="Node at 31286ec1-9bfe-8cf5-f555-4af451e5a6b2 [Follower]" leader= testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-588a5d96-ef8f-731a-9c27-b4567203686d testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-2003ea67-0ece-50a0-7a36-e2804e724e41 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-2003ea67-0ece-50a0-7a36-e2804e724e41 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.452Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6 42c1f716-2b47-4f4e-80a2-3ca8cc11dea6}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 testLoggerAdapter verbose: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 testing.go:149: server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: 2021-08-25T09:31:14.452Z [INFO] server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2: aborting pipeline replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: added peer, starting replication: peer=server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 31286ec1-9bfe-8cf5-f555-4af451e5a6b2}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 31286ec1-9bfe-8cf5-f555-4af451e5a6b2}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.453Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: pipelining replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.557Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.452702441 +0000 UTC m=+18.718918845 (1 nodes), 95.331µs from start of poll, 371.407644ms from cluster start. Timeout at 2021-08-25 09:31:14.557740016 +0000 UTC m=+18.823956500, 105.037655ms after stability testing.go:149: cluster: 2021-08-25T09:31:14.557Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.452702441 +0000 UTC m=+18.718918845 (1 nodes), 95.331µs from start of poll, 371.407644ms from cluster start. Timeout at 2021-08-25 09:31:14.557740016 +0000 UTC m=+18.823956500, 105.037655ms after stability testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-acdc9fbe-34fc-f5e5-5372-579b9c5ae972 acdc9fbe-34fc-f5e5-5372-579b9c5ae972}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-41e0b663-49fe-ffe9-c040-225723f99c1a 41e0b663-49fe-ffe9-c040-225723f99c1a}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-1fe4a98f-3e54-1fd5-cf32-62fafa19c811 1fe4a98f-3e54-1fd5-cf32-62fafa19c811}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-588a5d96-ef8f-731a-9c27-b4567203686d 588a5d96-ef8f-731a-9c27-b4567203686d}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-2003ea67-0ece-50a0-7a36-e2804e724e41 2003ea67-0ece-50a0-7a36-e2804e724e41}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 31286ec1-9bfe-8cf5-f555-4af451e5a6b2}" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.558Z [INFO] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: aborting pipeline replication: peer="{Voter server-31286ec1-9bfe-8cf5-f555-4af451e5a6b2 31286ec1-9bfe-8cf5-f555-4af451e5a6b2}" testLoggerAdapter verbose: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.608Z [ERROR] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: failed to heartbeat to: peer=1fe4a98f-3e54-1fd5-cf32-62fafa19c811 error="command timed out" testing.go:149: server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: 2021-08-25T09:31:14.608Z [ERROR] server-42c1f716-2b47-4f4e-80a2-3ca8cc11dea6: failed to heartbeat to: peer=1fe4a98f-3e54-1fd5-cf32-62fafa19c811 error="command timed out" --- PASS: TestRaft_LeadershipTransferWithSevenNodes (0.54s) === RUN TestRaft_LeadershipTransferToInvalidID testLoggerAdapter verbose: server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: 2021-08-25T09:31:14.615Z [INFO] server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf Address:0b0e4fc4-3705-56ec-c1ec-997db03f9ebf} {Suffrage:Voter ID:server-25774ff1-1b28-6a84-d8b1-feda5ab95927 Address:25774ff1-1b28-6a84-d8b1-feda5ab95927} {Suffrage:Voter ID:server-30796896-b483-4c89-9cab-b3ed5be09305 Address:30796896-b483-4c89-9cab-b3ed5be09305}]" testing.go:149: server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: 2021-08-25T09:31:14.615Z [INFO] server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf Address:0b0e4fc4-3705-56ec-c1ec-997db03f9ebf} {Suffrage:Voter ID:server-25774ff1-1b28-6a84-d8b1-feda5ab95927 Address:25774ff1-1b28-6a84-d8b1-feda5ab95927} {Suffrage:Voter ID:server-30796896-b483-4c89-9cab-b3ed5be09305 Address:30796896-b483-4c89-9cab-b3ed5be09305}]" testLoggerAdapter verbose: server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: 2021-08-25T09:31:14.616Z [INFO] server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: entering follower state: follower="Node at 0b0e4fc4-3705-56ec-c1ec-997db03f9ebf [Follower]" leader= testing.go:149: server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: 2021-08-25T09:31:14.616Z [INFO] server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf: entering follower state: follower="Node at 0b0e4fc4-3705-56ec-c1ec-997db03f9ebf [Follower]" leader= testLoggerAdapter verbose: server-25774ff1-1b28-6a84-d8b1-feda5ab95927: 2021-08-25T09:31:14.616Z [INFO] server-25774ff1-1b28-6a84-d8b1-feda5ab95927: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf Address:0b0e4fc4-3705-56ec-c1ec-997db03f9ebf} {Suffrage:Voter ID:server-25774ff1-1b28-6a84-d8b1-feda5ab95927 Address:25774ff1-1b28-6a84-d8b1-feda5ab95927} {Suffrage:Voter ID:server-30796896-b483-4c89-9cab-b3ed5be09305 Address:30796896-b483-4c89-9cab-b3ed5be09305}]" testing.go:149: server-25774ff1-1b28-6a84-d8b1-feda5ab95927: 2021-08-25T09:31:14.616Z [INFO] server-25774ff1-1b28-6a84-d8b1-feda5ab95927: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf Address:0b0e4fc4-3705-56ec-c1ec-997db03f9ebf} {Suffrage:Voter ID:server-25774ff1-1b28-6a84-d8b1-feda5ab95927 Address:25774ff1-1b28-6a84-d8b1-feda5ab95927} {Suffrage:Voter ID:server-30796896-b483-4c89-9cab-b3ed5be09305 Address:30796896-b483-4c89-9cab-b3ed5be09305}]" testLoggerAdapter verbose: server-25774ff1-1b28-6a84-d8b1-feda5ab95927: 2021-08-25T09:31:14.616Z [INFO] server-25774ff1-1b28-6a84-d8b1-feda5ab95927: entering follower state: follower="Node at 25774ff1-1b28-6a84-d8b1-feda5ab95927 [Follower]" leader= testing.go:149: server-25774ff1-1b28-6a84-d8b1-feda5ab95927: 2021-08-25T09:31:14.616Z [INFO] server-25774ff1-1b28-6a84-d8b1-feda5ab95927: entering follower state: follower="Node at 25774ff1-1b28-6a84-d8b1-feda5ab95927 [Follower]" leader= testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.616Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf Address:0b0e4fc4-3705-56ec-c1ec-997db03f9ebf} {Suffrage:Voter ID:server-25774ff1-1b28-6a84-d8b1-feda5ab95927 Address:25774ff1-1b28-6a84-d8b1-feda5ab95927} {Suffrage:Voter ID:server-30796896-b483-4c89-9cab-b3ed5be09305 Address:30796896-b483-4c89-9cab-b3ed5be09305}]" testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.616Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf Address:0b0e4fc4-3705-56ec-c1ec-997db03f9ebf} {Suffrage:Voter ID:server-25774ff1-1b28-6a84-d8b1-feda5ab95927 Address:25774ff1-1b28-6a84-d8b1-feda5ab95927} {Suffrage:Voter ID:server-30796896-b483-4c89-9cab-b3ed5be09305 Address:30796896-b483-4c89-9cab-b3ed5be09305}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.616Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:14.616Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.616Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: entering follower state: follower="Node at 30796896-b483-4c89-9cab-b3ed5be09305 [Follower]" leader= testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.616Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: entering follower state: follower="Node at 30796896-b483-4c89-9cab-b3ed5be09305 [Follower]" leader= testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [WARN] server-30796896-b483-4c89-9cab-b3ed5be09305: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [WARN] server-30796896-b483-4c89-9cab-b3ed5be09305: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: entering candidate state: node="Node at 30796896-b483-4c89-9cab-b3ed5be09305 [Candidate]" term=2 testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: entering candidate state: node="Node at 30796896-b483-4c89-9cab-b3ed5be09305 [Candidate]" term=2 testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: election won: tally=2 testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: election won: tally=2 testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: entering leader state: leader="Node at 30796896-b483-4c89-9cab-b3ed5be09305 [Leader]" testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: entering leader state: leader="Node at 30796896-b483-4c89-9cab-b3ed5be09305 [Leader]" testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: added peer, starting replication: peer=server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: added peer, starting replication: peer=server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: added peer, starting replication: peer=server-25774ff1-1b28-6a84-d8b1-feda5ab95927 testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: added peer, starting replication: peer=server-25774ff1-1b28-6a84-d8b1-feda5ab95927 testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: pipelining replication: peer="{Voter server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf 0b0e4fc4-3705-56ec-c1ec-997db03f9ebf}" testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: pipelining replication: peer="{Voter server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf 0b0e4fc4-3705-56ec-c1ec-997db03f9ebf}" testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: pipelining replication: peer="{Voter server-25774ff1-1b28-6a84-d8b1-feda5ab95927 25774ff1-1b28-6a84-d8b1-feda5ab95927}" testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.681Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: pipelining replication: peer="{Voter server-25774ff1-1b28-6a84-d8b1-feda5ab95927 25774ff1-1b28-6a84-d8b1-feda5ab95927}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.786Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.681787741 +0000 UTC m=+18.948004145 (1 nodes), 65.126491ms from start of poll, 66.19117ms from cluster start. Timeout at 2021-08-25 09:31:14.786818456 +0000 UTC m=+19.053034920, 105.030775ms after stability testing.go:149: cluster: 2021-08-25T09:31:14.786Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.681787741 +0000 UTC m=+18.948004145 (1 nodes), 65.126491ms from start of poll, 66.19117ms from cluster start. Timeout at 2021-08-25 09:31:14.786818456 +0000 UTC m=+19.053034920, 105.030775ms after stability testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.787Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: aborting pipeline replication: peer="{Voter server-25774ff1-1b28-6a84-d8b1-feda5ab95927 25774ff1-1b28-6a84-d8b1-feda5ab95927}" testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.787Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: aborting pipeline replication: peer="{Voter server-25774ff1-1b28-6a84-d8b1-feda5ab95927 25774ff1-1b28-6a84-d8b1-feda5ab95927}" testLoggerAdapter verbose: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.787Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: aborting pipeline replication: peer="{Voter server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf 0b0e4fc4-3705-56ec-c1ec-997db03f9ebf}" testing.go:149: server-30796896-b483-4c89-9cab-b3ed5be09305: 2021-08-25T09:31:14.787Z [INFO] server-30796896-b483-4c89-9cab-b3ed5be09305: aborting pipeline replication: peer="{Voter server-0b0e4fc4-3705-56ec-c1ec-997db03f9ebf 0b0e4fc4-3705-56ec-c1ec-997db03f9ebf}" --- PASS: TestRaft_LeadershipTransferToInvalidID (0.18s) === RUN TestRaft_LeadershipTransferToInvalidAddress testLoggerAdapter verbose: server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: 2021-08-25T09:31:14.791Z [INFO] server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 Address:b53d6d52-0ad5-72c0-ae98-e594ec3a5080} {Suffrage:Voter ID:server-89f5186a-fa60-689c-2325-d07daafc52b2 Address:89f5186a-fa60-689c-2325-d07daafc52b2} {Suffrage:Voter ID:server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 Address:0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}]" testing.go:149: server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: 2021-08-25T09:31:14.791Z [INFO] server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 Address:b53d6d52-0ad5-72c0-ae98-e594ec3a5080} {Suffrage:Voter ID:server-89f5186a-fa60-689c-2325-d07daafc52b2 Address:89f5186a-fa60-689c-2325-d07daafc52b2} {Suffrage:Voter ID:server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 Address:0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}]" testLoggerAdapter verbose: server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: 2021-08-25T09:31:14.791Z [INFO] server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: entering follower state: follower="Node at b53d6d52-0ad5-72c0-ae98-e594ec3a5080 [Follower]" leader= testing.go:149: server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: 2021-08-25T09:31:14.791Z [INFO] server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080: entering follower state: follower="Node at b53d6d52-0ad5-72c0-ae98-e594ec3a5080 [Follower]" leader= testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.791Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 Address:b53d6d52-0ad5-72c0-ae98-e594ec3a5080} {Suffrage:Voter ID:server-89f5186a-fa60-689c-2325-d07daafc52b2 Address:89f5186a-fa60-689c-2325-d07daafc52b2} {Suffrage:Voter ID:server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 Address:0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}]" testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.791Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 Address:b53d6d52-0ad5-72c0-ae98-e594ec3a5080} {Suffrage:Voter ID:server-89f5186a-fa60-689c-2325-d07daafc52b2 Address:89f5186a-fa60-689c-2325-d07daafc52b2} {Suffrage:Voter ID:server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 Address:0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}]" testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.791Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: entering follower state: follower="Node at 89f5186a-fa60-689c-2325-d07daafc52b2 [Follower]" leader= testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.791Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: entering follower state: follower="Node at 89f5186a-fa60-689c-2325-d07daafc52b2 [Follower]" leader= testLoggerAdapter verbose: server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: 2021-08-25T09:31:14.791Z [INFO] server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 Address:b53d6d52-0ad5-72c0-ae98-e594ec3a5080} {Suffrage:Voter ID:server-89f5186a-fa60-689c-2325-d07daafc52b2 Address:89f5186a-fa60-689c-2325-d07daafc52b2} {Suffrage:Voter ID:server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 Address:0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}]" testing.go:149: server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: 2021-08-25T09:31:14.791Z [INFO] server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 Address:b53d6d52-0ad5-72c0-ae98-e594ec3a5080} {Suffrage:Voter ID:server-89f5186a-fa60-689c-2325-d07daafc52b2 Address:89f5186a-fa60-689c-2325-d07daafc52b2} {Suffrage:Voter ID:server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 Address:0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.792Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:14.792Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: 2021-08-25T09:31:14.792Z [INFO] server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: entering follower state: follower="Node at 0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 [Follower]" leader= testing.go:149: server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: 2021-08-25T09:31:14.792Z [INFO] server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865: entering follower state: follower="Node at 0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 [Follower]" leader= testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [WARN] server-89f5186a-fa60-689c-2325-d07daafc52b2: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [WARN] server-89f5186a-fa60-689c-2325-d07daafc52b2: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: entering candidate state: node="Node at 89f5186a-fa60-689c-2325-d07daafc52b2 [Candidate]" term=2 testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: entering candidate state: node="Node at 89f5186a-fa60-689c-2325-d07daafc52b2 [Candidate]" term=2 testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: election won: tally=2 testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: election won: tally=2 testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: entering leader state: leader="Node at 89f5186a-fa60-689c-2325-d07daafc52b2 [Leader]" testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: entering leader state: leader="Node at 89f5186a-fa60-689c-2325-d07daafc52b2 [Leader]" testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: added peer, starting replication: peer=server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: added peer, starting replication: peer=server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: added peer, starting replication: peer=server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: added peer, starting replication: peer=server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: pipelining replication: peer="{Voter server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 b53d6d52-0ad5-72c0-ae98-e594ec3a5080}" testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: pipelining replication: peer="{Voter server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 b53d6d52-0ad5-72c0-ae98-e594ec3a5080}" testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: pipelining replication: peer="{Voter server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}" testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:14.847Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: pipelining replication: peer="{Voter server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.952Z [INFO] stable state for Follower reached at 2021-08-25 09:31:14.847494885 +0000 UTC m=+19.113711309 (2 nodes), 55.333492ms from start of poll, 56.560996ms from cluster start. Timeout at 2021-08-25 09:31:14.952609353 +0000 UTC m=+19.218825836, 105.114527ms after stability testing.go:149: cluster: 2021-08-25T09:31:14.952Z [INFO] stable state for Follower reached at 2021-08-25 09:31:14.847494885 +0000 UTC m=+19.113711309 (2 nodes), 55.333492ms from start of poll, 56.560996ms from cluster start. Timeout at 2021-08-25 09:31:14.952609353 +0000 UTC m=+19.218825836, 105.114527ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:14.952Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:14.952Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.058Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.952929842 +0000 UTC m=+19.219146246 (1 nodes), 2.319µs from start of poll, 161.995933ms from cluster start. Timeout at 2021-08-25 09:31:15.058016992 +0000 UTC m=+19.324233496, 105.08725ms after stability testing.go:149: cluster: 2021-08-25T09:31:15.058Z [INFO] stable state for Leader reached at 2021-08-25 09:31:14.952929842 +0000 UTC m=+19.219146246 (1 nodes), 2.319µs from start of poll, 161.995933ms from cluster start. Timeout at 2021-08-25 09:31:15.058016992 +0000 UTC m=+19.324233496, 105.08725ms after stability testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:15.058Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: aborting pipeline replication: peer="{Voter server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}" testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:15.058Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: aborting pipeline replication: peer="{Voter server-0931e6f0-ca2c-1493-3aa3-ed47b8ae2865 0931e6f0-ca2c-1493-3aa3-ed47b8ae2865}" testLoggerAdapter verbose: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:15.058Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: aborting pipeline replication: peer="{Voter server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 b53d6d52-0ad5-72c0-ae98-e594ec3a5080}" testing.go:149: server-89f5186a-fa60-689c-2325-d07daafc52b2: 2021-08-25T09:31:15.058Z [INFO] server-89f5186a-fa60-689c-2325-d07daafc52b2: aborting pipeline replication: peer="{Voter server-b53d6d52-0ad5-72c0-ae98-e594ec3a5080 b53d6d52-0ad5-72c0-ae98-e594ec3a5080}" --- PASS: TestRaft_LeadershipTransferToInvalidAddress (0.27s) === RUN TestRaft_LeadershipTransferToBehindServer testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.062Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1c188701-f54a-d72f-8c33-efb56875b3c5 Address:1c188701-f54a-d72f-8c33-efb56875b3c5} {Suffrage:Voter ID:server-72c2c775-6105-6649-87b2-62bdbb54ab42 Address:72c2c775-6105-6649-87b2-62bdbb54ab42} {Suffrage:Voter ID:server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 Address:5971c68c-2edf-c1f2-b958-95bf0e2008a0}]" testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.062Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1c188701-f54a-d72f-8c33-efb56875b3c5 Address:1c188701-f54a-d72f-8c33-efb56875b3c5} {Suffrage:Voter ID:server-72c2c775-6105-6649-87b2-62bdbb54ab42 Address:72c2c775-6105-6649-87b2-62bdbb54ab42} {Suffrage:Voter ID:server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 Address:5971c68c-2edf-c1f2-b958-95bf0e2008a0}]" testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.063Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: entering follower state: follower="Node at 1c188701-f54a-d72f-8c33-efb56875b3c5 [Follower]" leader= testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.063Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: entering follower state: follower="Node at 1c188701-f54a-d72f-8c33-efb56875b3c5 [Follower]" leader= testLoggerAdapter verbose: server-72c2c775-6105-6649-87b2-62bdbb54ab42: 2021-08-25T09:31:15.063Z [INFO] server-72c2c775-6105-6649-87b2-62bdbb54ab42: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1c188701-f54a-d72f-8c33-efb56875b3c5 Address:1c188701-f54a-d72f-8c33-efb56875b3c5} {Suffrage:Voter ID:server-72c2c775-6105-6649-87b2-62bdbb54ab42 Address:72c2c775-6105-6649-87b2-62bdbb54ab42} {Suffrage:Voter ID:server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 Address:5971c68c-2edf-c1f2-b958-95bf0e2008a0}]" testing.go:149: server-72c2c775-6105-6649-87b2-62bdbb54ab42: 2021-08-25T09:31:15.063Z [INFO] server-72c2c775-6105-6649-87b2-62bdbb54ab42: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1c188701-f54a-d72f-8c33-efb56875b3c5 Address:1c188701-f54a-d72f-8c33-efb56875b3c5} {Suffrage:Voter ID:server-72c2c775-6105-6649-87b2-62bdbb54ab42 Address:72c2c775-6105-6649-87b2-62bdbb54ab42} {Suffrage:Voter ID:server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 Address:5971c68c-2edf-c1f2-b958-95bf0e2008a0}]" testLoggerAdapter verbose: server-72c2c775-6105-6649-87b2-62bdbb54ab42: 2021-08-25T09:31:15.063Z [INFO] server-72c2c775-6105-6649-87b2-62bdbb54ab42: entering follower state: follower="Node at 72c2c775-6105-6649-87b2-62bdbb54ab42 [Follower]" leader= testing.go:149: server-72c2c775-6105-6649-87b2-62bdbb54ab42: 2021-08-25T09:31:15.063Z [INFO] server-72c2c775-6105-6649-87b2-62bdbb54ab42: entering follower state: follower="Node at 72c2c775-6105-6649-87b2-62bdbb54ab42 [Follower]" leader= testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.063Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1c188701-f54a-d72f-8c33-efb56875b3c5 Address:1c188701-f54a-d72f-8c33-efb56875b3c5} {Suffrage:Voter ID:server-72c2c775-6105-6649-87b2-62bdbb54ab42 Address:72c2c775-6105-6649-87b2-62bdbb54ab42} {Suffrage:Voter ID:server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 Address:5971c68c-2edf-c1f2-b958-95bf0e2008a0}]" testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.063Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1c188701-f54a-d72f-8c33-efb56875b3c5 Address:1c188701-f54a-d72f-8c33-efb56875b3c5} {Suffrage:Voter ID:server-72c2c775-6105-6649-87b2-62bdbb54ab42 Address:72c2c775-6105-6649-87b2-62bdbb54ab42} {Suffrage:Voter ID:server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 Address:5971c68c-2edf-c1f2-b958-95bf0e2008a0}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.063Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:15.063Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.063Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering follower state: follower="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Follower]" leader= testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.063Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering follower state: follower="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Follower]" leader= testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [WARN] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [WARN] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering candidate state: node="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Candidate]" term=2 testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering candidate state: node="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Candidate]" term=2 testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: election won: tally=2 testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: election won: tally=2 testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering leader state: leader="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Leader]" testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering leader state: leader="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Leader]" testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: added peer, starting replication: peer=server-1c188701-f54a-d72f-8c33-efb56875b3c5 testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: added peer, starting replication: peer=server-1c188701-f54a-d72f-8c33-efb56875b3c5 testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: added peer, starting replication: peer=server-72c2c775-6105-6649-87b2-62bdbb54ab42 testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.135Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: added peer, starting replication: peer=server-72c2c775-6105-6649-87b2-62bdbb54ab42 testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.136Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: pipelining replication: peer="{Voter server-1c188701-f54a-d72f-8c33-efb56875b3c5 1c188701-f54a-d72f-8c33-efb56875b3c5}" testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.136Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: pipelining replication: peer="{Voter server-1c188701-f54a-d72f-8c33-efb56875b3c5 1c188701-f54a-d72f-8c33-efb56875b3c5}" testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.136Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: pipelining replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.136Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: pipelining replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.241Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.135962816 +0000 UTC m=+19.402179220 (1 nodes), 72.045799ms from start of poll, 73.276283ms from cluster start. Timeout at 2021-08-25 09:31:15.241088422 +0000 UTC m=+19.507304866, 105.125646ms after stability testing.go:149: cluster: 2021-08-25T09:31:15.241Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.135962816 +0000 UTC m=+19.402179220 (1 nodes), 72.045799ms from start of poll, 73.276283ms from cluster start. Timeout at 2021-08-25 09:31:15.241088422 +0000 UTC m=+19.507304866, 105.125646ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.241Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:15.241Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.346Z [INFO] stable state for Follower reached at 2021-08-25 09:31:15.241389694 +0000 UTC m=+19.507606078 (2 nodes), 3.66µs from start of poll, 178.703141ms from cluster start. Timeout at 2021-08-25 09:31:15.346504461 +0000 UTC m=+19.612720865, 105.114787ms after stability testing.go:149: cluster: 2021-08-25T09:31:15.346Z [INFO] stable state for Follower reached at 2021-08-25 09:31:15.241389694 +0000 UTC m=+19.507606078 (2 nodes), 3.66µs from start of poll, 178.703141ms from cluster start. Timeout at 2021-08-25 09:31:15.346504461 +0000 UTC m=+19.612720865, 105.114787ms after stability testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: entering candidate state: node="Node at 1c188701-f54a-d72f-8c33-efb56875b3c5 [Candidate]" term=3 testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.380Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: entering candidate state: node="Node at 1c188701-f54a-d72f-8c33-efb56875b3c5 [Candidate]" term=3 testing.go:149: cluster: 2021-08-25T09:31:15.380Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.380Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering follower state: follower="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Follower]" leader= testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: election won: tally=2 testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.380Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: entering follower state: follower="Node at 5971c68c-2edf-c1f2-b958-95bf0e2008a0 [Follower]" leader= testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: election won: tally=2 testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.380Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: aborting pipeline replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: entering leader state: leader="Node at 1c188701-f54a-d72f-8c33-efb56875b3c5 [Leader]" testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.380Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: aborting pipeline replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" testLoggerAdapter verbose: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.380Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: aborting pipeline replication: peer="{Voter server-1c188701-f54a-d72f-8c33-efb56875b3c5 1c188701-f54a-d72f-8c33-efb56875b3c5}" testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: entering leader state: leader="Node at 1c188701-f54a-d72f-8c33-efb56875b3c5 [Leader]" testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: added peer, starting replication: peer=server-72c2c775-6105-6649-87b2-62bdbb54ab42 testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: added peer, starting replication: peer=server-72c2c775-6105-6649-87b2-62bdbb54ab42 testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: added peer, starting replication: peer=server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: added peer, starting replication: peer=server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: pipelining replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" testing.go:149: server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: 2021-08-25T09:31:15.380Z [INFO] server-5971c68c-2edf-c1f2-b958-95bf0e2008a0: aborting pipeline replication: peer="{Voter server-1c188701-f54a-d72f-8c33-efb56875b3c5 1c188701-f54a-d72f-8c33-efb56875b3c5}" testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: pipelining replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: pipelining replication: peer="{Voter server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 5971c68c-2edf-c1f2-b958-95bf0e2008a0}" testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.380Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: pipelining replication: peer="{Voter server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 5971c68c-2edf-c1f2-b958-95bf0e2008a0}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.485Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.380616791 +0000 UTC m=+19.646833195 (1 nodes), 120.669µs from start of poll, 317.930258ms from cluster start. Timeout at 2021-08-25 09:31:15.485726138 +0000 UTC m=+19.751942542, 105.109347ms after stability testing.go:149: cluster: 2021-08-25T09:31:15.485Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.380616791 +0000 UTC m=+19.646833195 (1 nodes), 120.669µs from start of poll, 317.930258ms from cluster start. Timeout at 2021-08-25 09:31:15.485726138 +0000 UTC m=+19.751942542, 105.109347ms after stability testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.486Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: aborting pipeline replication: peer="{Voter server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 5971c68c-2edf-c1f2-b958-95bf0e2008a0}" testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.486Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: aborting pipeline replication: peer="{Voter server-5971c68c-2edf-c1f2-b958-95bf0e2008a0 5971c68c-2edf-c1f2-b958-95bf0e2008a0}" testLoggerAdapter verbose: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.486Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: aborting pipeline replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" testing.go:149: server-1c188701-f54a-d72f-8c33-efb56875b3c5: 2021-08-25T09:31:15.486Z [INFO] server-1c188701-f54a-d72f-8c33-efb56875b3c5: aborting pipeline replication: peer="{Voter server-72c2c775-6105-6649-87b2-62bdbb54ab42 72c2c775-6105-6649-87b2-62bdbb54ab42}" --- PASS: TestRaft_LeadershipTransferToBehindServer (0.43s) === RUN TestRaft_LeadershipTransferToItself testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.490Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-af8e90da-a68e-7707-3f85-a7573c1bc576 Address:af8e90da-a68e-7707-3f85-a7573c1bc576} {Suffrage:Voter ID:server-556cd876-19b4-7074-ebf5-ea51065c65b3 Address:556cd876-19b4-7074-ebf5-ea51065c65b3} {Suffrage:Voter ID:server-f28d238d-b992-d629-c208-c9c9b4cc3218 Address:f28d238d-b992-d629-c208-c9c9b4cc3218}]" testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.490Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-af8e90da-a68e-7707-3f85-a7573c1bc576 Address:af8e90da-a68e-7707-3f85-a7573c1bc576} {Suffrage:Voter ID:server-556cd876-19b4-7074-ebf5-ea51065c65b3 Address:556cd876-19b4-7074-ebf5-ea51065c65b3} {Suffrage:Voter ID:server-f28d238d-b992-d629-c208-c9c9b4cc3218 Address:f28d238d-b992-d629-c208-c9c9b4cc3218}]" testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.490Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: entering follower state: follower="Node at af8e90da-a68e-7707-3f85-a7573c1bc576 [Follower]" leader= testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.490Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: entering follower state: follower="Node at af8e90da-a68e-7707-3f85-a7573c1bc576 [Follower]" leader= testLoggerAdapter verbose: server-556cd876-19b4-7074-ebf5-ea51065c65b3: 2021-08-25T09:31:15.490Z [INFO] server-556cd876-19b4-7074-ebf5-ea51065c65b3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-af8e90da-a68e-7707-3f85-a7573c1bc576 Address:af8e90da-a68e-7707-3f85-a7573c1bc576} {Suffrage:Voter ID:server-556cd876-19b4-7074-ebf5-ea51065c65b3 Address:556cd876-19b4-7074-ebf5-ea51065c65b3} {Suffrage:Voter ID:server-f28d238d-b992-d629-c208-c9c9b4cc3218 Address:f28d238d-b992-d629-c208-c9c9b4cc3218}]" testing.go:149: server-556cd876-19b4-7074-ebf5-ea51065c65b3: 2021-08-25T09:31:15.490Z [INFO] server-556cd876-19b4-7074-ebf5-ea51065c65b3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-af8e90da-a68e-7707-3f85-a7573c1bc576 Address:af8e90da-a68e-7707-3f85-a7573c1bc576} {Suffrage:Voter ID:server-556cd876-19b4-7074-ebf5-ea51065c65b3 Address:556cd876-19b4-7074-ebf5-ea51065c65b3} {Suffrage:Voter ID:server-f28d238d-b992-d629-c208-c9c9b4cc3218 Address:f28d238d-b992-d629-c208-c9c9b4cc3218}]" testLoggerAdapter verbose: server-556cd876-19b4-7074-ebf5-ea51065c65b3: 2021-08-25T09:31:15.490Z [INFO] server-556cd876-19b4-7074-ebf5-ea51065c65b3: entering follower state: follower="Node at 556cd876-19b4-7074-ebf5-ea51065c65b3 [Follower]" leader= testing.go:149: server-556cd876-19b4-7074-ebf5-ea51065c65b3: 2021-08-25T09:31:15.490Z [INFO] server-556cd876-19b4-7074-ebf5-ea51065c65b3: entering follower state: follower="Node at 556cd876-19b4-7074-ebf5-ea51065c65b3 [Follower]" leader= testLoggerAdapter verbose: server-f28d238d-b992-d629-c208-c9c9b4cc3218: 2021-08-25T09:31:15.491Z [INFO] server-f28d238d-b992-d629-c208-c9c9b4cc3218: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-af8e90da-a68e-7707-3f85-a7573c1bc576 Address:af8e90da-a68e-7707-3f85-a7573c1bc576} {Suffrage:Voter ID:server-556cd876-19b4-7074-ebf5-ea51065c65b3 Address:556cd876-19b4-7074-ebf5-ea51065c65b3} {Suffrage:Voter ID:server-f28d238d-b992-d629-c208-c9c9b4cc3218 Address:f28d238d-b992-d629-c208-c9c9b4cc3218}]" testing.go:149: server-f28d238d-b992-d629-c208-c9c9b4cc3218: 2021-08-25T09:31:15.491Z [INFO] server-f28d238d-b992-d629-c208-c9c9b4cc3218: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-af8e90da-a68e-7707-3f85-a7573c1bc576 Address:af8e90da-a68e-7707-3f85-a7573c1bc576} {Suffrage:Voter ID:server-556cd876-19b4-7074-ebf5-ea51065c65b3 Address:556cd876-19b4-7074-ebf5-ea51065c65b3} {Suffrage:Voter ID:server-f28d238d-b992-d629-c208-c9c9b4cc3218 Address:f28d238d-b992-d629-c208-c9c9b4cc3218}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.491Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:15.491Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-f28d238d-b992-d629-c208-c9c9b4cc3218: 2021-08-25T09:31:15.491Z [INFO] server-f28d238d-b992-d629-c208-c9c9b4cc3218: entering follower state: follower="Node at f28d238d-b992-d629-c208-c9c9b4cc3218 [Follower]" leader= testing.go:149: server-f28d238d-b992-d629-c208-c9c9b4cc3218: 2021-08-25T09:31:15.491Z [INFO] server-f28d238d-b992-d629-c208-c9c9b4cc3218: entering follower state: follower="Node at f28d238d-b992-d629-c208-c9c9b4cc3218 [Follower]" leader= testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.544Z [WARN] server-af8e90da-a68e-7707-3f85-a7573c1bc576: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.544Z [WARN] server-af8e90da-a68e-7707-3f85-a7573c1bc576: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.544Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: entering candidate state: node="Node at af8e90da-a68e-7707-3f85-a7573c1bc576 [Candidate]" term=2 testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.544Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: entering candidate state: node="Node at af8e90da-a68e-7707-3f85-a7573c1bc576 [Candidate]" term=2 testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.544Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: election won: tally=2 testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.544Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: election won: tally=2 testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: entering leader state: leader="Node at af8e90da-a68e-7707-3f85-a7573c1bc576 [Leader]" testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: entering leader state: leader="Node at af8e90da-a68e-7707-3f85-a7573c1bc576 [Leader]" testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: added peer, starting replication: peer=server-556cd876-19b4-7074-ebf5-ea51065c65b3 testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: added peer, starting replication: peer=server-556cd876-19b4-7074-ebf5-ea51065c65b3 testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: added peer, starting replication: peer=server-f28d238d-b992-d629-c208-c9c9b4cc3218 testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: added peer, starting replication: peer=server-f28d238d-b992-d629-c208-c9c9b4cc3218 testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: pipelining replication: peer="{Voter server-556cd876-19b4-7074-ebf5-ea51065c65b3 556cd876-19b4-7074-ebf5-ea51065c65b3}" testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: pipelining replication: peer="{Voter server-556cd876-19b4-7074-ebf5-ea51065c65b3 556cd876-19b4-7074-ebf5-ea51065c65b3}" testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: pipelining replication: peer="{Voter server-f28d238d-b992-d629-c208-c9c9b4cc3218 f28d238d-b992-d629-c208-c9c9b4cc3218}" testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.545Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: pipelining replication: peer="{Voter server-f28d238d-b992-d629-c208-c9c9b4cc3218 f28d238d-b992-d629-c208-c9c9b4cc3218}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.650Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.545090651 +0000 UTC m=+19.811307055 (1 nodes), 53.886468ms from start of poll, 55.14121ms from cluster start. Timeout at 2021-08-25 09:31:15.650212398 +0000 UTC m=+19.916428862, 105.121807ms after stability testing.go:149: cluster: 2021-08-25T09:31:15.650Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.545090651 +0000 UTC m=+19.811307055 (1 nodes), 53.886468ms from start of poll, 55.14121ms from cluster start. Timeout at 2021-08-25 09:31:15.650212398 +0000 UTC m=+19.916428862, 105.121807ms after stability testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.650Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: cannot transfer leadership to itself testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.650Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: cannot transfer leadership to itself testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.650Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: aborting pipeline replication: peer="{Voter server-556cd876-19b4-7074-ebf5-ea51065c65b3 556cd876-19b4-7074-ebf5-ea51065c65b3}" testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.650Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: aborting pipeline replication: peer="{Voter server-556cd876-19b4-7074-ebf5-ea51065c65b3 556cd876-19b4-7074-ebf5-ea51065c65b3}" testLoggerAdapter verbose: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.650Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: aborting pipeline replication: peer="{Voter server-f28d238d-b992-d629-c208-c9c9b4cc3218 f28d238d-b992-d629-c208-c9c9b4cc3218}" testing.go:149: server-af8e90da-a68e-7707-3f85-a7573c1bc576: 2021-08-25T09:31:15.650Z [INFO] server-af8e90da-a68e-7707-3f85-a7573c1bc576: aborting pipeline replication: peer="{Voter server-f28d238d-b992-d629-c208-c9c9b4cc3218 f28d238d-b992-d629-c208-c9c9b4cc3218}" --- PASS: TestRaft_LeadershipTransferToItself (0.16s) === RUN TestRaft_LeadershipTransferLeaderRejectsClientRequests raft_test.go:2064: Disabled unreliable test --- SKIP: TestRaft_LeadershipTransferLeaderRejectsClientRequests (0.00s) === RUN TestRaft_LeadershipTransferLeaderReplicationTimeout testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.655Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1d8f6168-91a1-620d-c754-079ce18bbab7 Address:1d8f6168-91a1-620d-c754-079ce18bbab7} {Suffrage:Voter ID:server-56be31ef-e709-41a1-e1de-04689d3bf5f7 Address:56be31ef-e709-41a1-e1de-04689d3bf5f7} {Suffrage:Voter ID:server-0af2c5a3-3705-949c-6491-81f422b3e4fa Address:0af2c5a3-3705-949c-6491-81f422b3e4fa}]" testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.655Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1d8f6168-91a1-620d-c754-079ce18bbab7 Address:1d8f6168-91a1-620d-c754-079ce18bbab7} {Suffrage:Voter ID:server-56be31ef-e709-41a1-e1de-04689d3bf5f7 Address:56be31ef-e709-41a1-e1de-04689d3bf5f7} {Suffrage:Voter ID:server-0af2c5a3-3705-949c-6491-81f422b3e4fa Address:0af2c5a3-3705-949c-6491-81f422b3e4fa}]" testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.655Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: entering follower state: follower="Node at 1d8f6168-91a1-620d-c754-079ce18bbab7 [Follower]" leader= testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.655Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: entering follower state: follower="Node at 1d8f6168-91a1-620d-c754-079ce18bbab7 [Follower]" leader= testLoggerAdapter verbose: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:15.656Z [INFO] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1d8f6168-91a1-620d-c754-079ce18bbab7 Address:1d8f6168-91a1-620d-c754-079ce18bbab7} {Suffrage:Voter ID:server-56be31ef-e709-41a1-e1de-04689d3bf5f7 Address:56be31ef-e709-41a1-e1de-04689d3bf5f7} {Suffrage:Voter ID:server-0af2c5a3-3705-949c-6491-81f422b3e4fa Address:0af2c5a3-3705-949c-6491-81f422b3e4fa}]" testing.go:149: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:15.656Z [INFO] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1d8f6168-91a1-620d-c754-079ce18bbab7 Address:1d8f6168-91a1-620d-c754-079ce18bbab7} {Suffrage:Voter ID:server-56be31ef-e709-41a1-e1de-04689d3bf5f7 Address:56be31ef-e709-41a1-e1de-04689d3bf5f7} {Suffrage:Voter ID:server-0af2c5a3-3705-949c-6491-81f422b3e4fa Address:0af2c5a3-3705-949c-6491-81f422b3e4fa}]" testLoggerAdapter verbose: server-0af2c5a3-3705-949c-6491-81f422b3e4fa: 2021-08-25T09:31:15.657Z [INFO] server-0af2c5a3-3705-949c-6491-81f422b3e4fa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1d8f6168-91a1-620d-c754-079ce18bbab7 Address:1d8f6168-91a1-620d-c754-079ce18bbab7} {Suffrage:Voter ID:server-56be31ef-e709-41a1-e1de-04689d3bf5f7 Address:56be31ef-e709-41a1-e1de-04689d3bf5f7} {Suffrage:Voter ID:server-0af2c5a3-3705-949c-6491-81f422b3e4fa Address:0af2c5a3-3705-949c-6491-81f422b3e4fa}]" testing.go:149: server-0af2c5a3-3705-949c-6491-81f422b3e4fa: 2021-08-25T09:31:15.657Z [INFO] server-0af2c5a3-3705-949c-6491-81f422b3e4fa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1d8f6168-91a1-620d-c754-079ce18bbab7 Address:1d8f6168-91a1-620d-c754-079ce18bbab7} {Suffrage:Voter ID:server-56be31ef-e709-41a1-e1de-04689d3bf5f7 Address:56be31ef-e709-41a1-e1de-04689d3bf5f7} {Suffrage:Voter ID:server-0af2c5a3-3705-949c-6491-81f422b3e4fa Address:0af2c5a3-3705-949c-6491-81f422b3e4fa}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.657Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:15.657Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:15.657Z [INFO] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: entering follower state: follower="Node at 56be31ef-e709-41a1-e1de-04689d3bf5f7 [Follower]" leader= testing.go:149: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:15.657Z [INFO] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: entering follower state: follower="Node at 56be31ef-e709-41a1-e1de-04689d3bf5f7 [Follower]" leader= testLoggerAdapter verbose: server-0af2c5a3-3705-949c-6491-81f422b3e4fa: 2021-08-25T09:31:15.657Z [INFO] server-0af2c5a3-3705-949c-6491-81f422b3e4fa: entering follower state: follower="Node at 0af2c5a3-3705-949c-6491-81f422b3e4fa [Follower]" leader= testing.go:149: server-0af2c5a3-3705-949c-6491-81f422b3e4fa: 2021-08-25T09:31:15.657Z [INFO] server-0af2c5a3-3705-949c-6491-81f422b3e4fa: entering follower state: follower="Node at 0af2c5a3-3705-949c-6491-81f422b3e4fa [Follower]" leader= testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.725Z [WARN] server-1d8f6168-91a1-620d-c754-079ce18bbab7: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.725Z [WARN] server-1d8f6168-91a1-620d-c754-079ce18bbab7: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: entering candidate state: node="Node at 1d8f6168-91a1-620d-c754-079ce18bbab7 [Candidate]" term=2 testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: entering candidate state: node="Node at 1d8f6168-91a1-620d-c754-079ce18bbab7 [Candidate]" term=2 testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: election won: tally=2 testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: election won: tally=2 testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: entering leader state: leader="Node at 1d8f6168-91a1-620d-c754-079ce18bbab7 [Leader]" testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: entering leader state: leader="Node at 1d8f6168-91a1-620d-c754-079ce18bbab7 [Leader]" testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: added peer, starting replication: peer=server-56be31ef-e709-41a1-e1de-04689d3bf5f7 testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: added peer, starting replication: peer=server-56be31ef-e709-41a1-e1de-04689d3bf5f7 testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: added peer, starting replication: peer=server-0af2c5a3-3705-949c-6491-81f422b3e4fa testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: added peer, starting replication: peer=server-0af2c5a3-3705-949c-6491-81f422b3e4fa testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: pipelining replication: peer="{Voter server-56be31ef-e709-41a1-e1de-04689d3bf5f7 56be31ef-e709-41a1-e1de-04689d3bf5f7}" testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: pipelining replication: peer="{Voter server-56be31ef-e709-41a1-e1de-04689d3bf5f7 56be31ef-e709-41a1-e1de-04689d3bf5f7}" testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: pipelining replication: peer="{Voter server-0af2c5a3-3705-949c-6491-81f422b3e4fa 0af2c5a3-3705-949c-6491-81f422b3e4fa}" testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:15.726Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: pipelining replication: peer="{Voter server-0af2c5a3-3705-949c-6491-81f422b3e4fa 0af2c5a3-3705-949c-6491-81f422b3e4fa}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.831Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.726345972 +0000 UTC m=+19.992562416 (1 nodes), 69.018724ms from start of poll, 71.782884ms from cluster start. Timeout at 2021-08-25 09:31:15.831477678 +0000 UTC m=+20.097694122, 105.131706ms after stability testing.go:149: cluster: 2021-08-25T09:31:15.831Z [INFO] stable state for Leader reached at 2021-08-25 09:31:15.726345972 +0000 UTC m=+19.992562416 (1 nodes), 69.018724ms from start of poll, 71.782884ms from cluster start. Timeout at 2021-08-25 09:31:15.831477678 +0000 UTC m=+20.097694122, 105.131706ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.831Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:15.831Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2021-08-25T09:31:15.937Z [INFO] stable state for Follower reached at 2021-08-25 09:31:15.831805167 +0000 UTC m=+20.098021571 (2 nodes), 2.7µs from start of poll, 177.242039ms from cluster start. Timeout at 2021-08-25 09:31:15.936943212 +0000 UTC m=+20.203160256, 105.138685ms after stability testing.go:149: cluster: 2021-08-25T09:31:15.937Z [INFO] stable state for Follower reached at 2021-08-25 09:31:15.831805167 +0000 UTC m=+20.098021571 (2 nodes), 2.7µs from start of poll, 177.242039ms from cluster start. Timeout at 2021-08-25 09:31:15.936943212 +0000 UTC m=+20.203160256, 105.138685ms after stability testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:16.322Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: aborting pipeline replication: peer="{Voter server-0af2c5a3-3705-949c-6491-81f422b3e4fa 0af2c5a3-3705-949c-6491-81f422b3e4fa}" testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:16.322Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: aborting pipeline replication: peer="{Voter server-0af2c5a3-3705-949c-6491-81f422b3e4fa 0af2c5a3-3705-949c-6491-81f422b3e4fa}" testLoggerAdapter verbose: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:16.322Z [INFO] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: entering candidate state: node="Node at 56be31ef-e709-41a1-e1de-04689d3bf5f7 [Shutdown]" term=3 testing.go:149: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:16.322Z [INFO] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: entering candidate state: node="Node at 56be31ef-e709-41a1-e1de-04689d3bf5f7 [Shutdown]" term=3 testLoggerAdapter verbose: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:16.322Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: aborting pipeline replication: peer="{Voter server-56be31ef-e709-41a1-e1de-04689d3bf5f7 56be31ef-e709-41a1-e1de-04689d3bf5f7}" testing.go:149: server-1d8f6168-91a1-620d-c754-079ce18bbab7: 2021-08-25T09:31:16.322Z [INFO] server-1d8f6168-91a1-620d-c754-079ce18bbab7: aborting pipeline replication: peer="{Voter server-56be31ef-e709-41a1-e1de-04689d3bf5f7 56be31ef-e709-41a1-e1de-04689d3bf5f7}" testLoggerAdapter verbose: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:16.372Z [ERROR] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: failed to make requestVote RPC: target="{Voter server-1d8f6168-91a1-620d-c754-079ce18bbab7 1d8f6168-91a1-620d-c754-079ce18bbab7}" error="command timed out" testing.go:149: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:16.372Z [ERROR] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: failed to make requestVote RPC: target="{Voter server-1d8f6168-91a1-620d-c754-079ce18bbab7 1d8f6168-91a1-620d-c754-079ce18bbab7}" error="command timed out" testLoggerAdapter verbose: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:16.372Z [ERROR] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: failed to make requestVote RPC: target="{Voter server-0af2c5a3-3705-949c-6491-81f422b3e4fa 0af2c5a3-3705-949c-6491-81f422b3e4fa}" error="command timed out" testing.go:149: server-56be31ef-e709-41a1-e1de-04689d3bf5f7: 2021-08-25T09:31:16.372Z [ERROR] server-56be31ef-e709-41a1-e1de-04689d3bf5f7: failed to make requestVote RPC: target="{Voter server-0af2c5a3-3705-949c-6491-81f422b3e4fa 0af2c5a3-3705-949c-6491-81f422b3e4fa}" error="command timed out" --- PASS: TestRaft_LeadershipTransferLeaderReplicationTimeout (0.72s) === RUN TestRaft_LeadershipTransferStopRightAway --- PASS: TestRaft_LeadershipTransferStopRightAway (0.00s) === RUN TestRaft_GetConfigurationNoBootstrap testLoggerAdapter verbose: server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: 2021-08-25T09:31:16.376Z [INFO] server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a Address:00e96bd7-3d38-a6b8-e404-23bc45ebed3a} {Suffrage:Voter ID:server-c3d9dd09-9515-e49d-50fe-328d27ee0d20 Address:c3d9dd09-9515-e49d-50fe-328d27ee0d20}]" testing.go:149: server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: 2021-08-25T09:31:16.376Z [INFO] server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a Address:00e96bd7-3d38-a6b8-e404-23bc45ebed3a} {Suffrage:Voter ID:server-c3d9dd09-9515-e49d-50fe-328d27ee0d20 Address:c3d9dd09-9515-e49d-50fe-328d27ee0d20}]" testLoggerAdapter verbose: server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: 2021-08-25T09:31:16.376Z [INFO] server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: entering follower state: follower="Node at 00e96bd7-3d38-a6b8-e404-23bc45ebed3a [Follower]" leader= testing.go:149: server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: 2021-08-25T09:31:16.376Z [INFO] server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a: entering follower state: follower="Node at 00e96bd7-3d38-a6b8-e404-23bc45ebed3a [Follower]" leader= testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.376Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a Address:00e96bd7-3d38-a6b8-e404-23bc45ebed3a} {Suffrage:Voter ID:server-c3d9dd09-9515-e49d-50fe-328d27ee0d20 Address:c3d9dd09-9515-e49d-50fe-328d27ee0d20}]" testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.376Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a Address:00e96bd7-3d38-a6b8-e404-23bc45ebed3a} {Suffrage:Voter ID:server-c3d9dd09-9515-e49d-50fe-328d27ee0d20 Address:c3d9dd09-9515-e49d-50fe-328d27ee0d20}]" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:16.376Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2021-08-25T09:31:16.376Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.377Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: entering follower state: follower="Node at c3d9dd09-9515-e49d-50fe-328d27ee0d20 [Follower]" leader= testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.377Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: entering follower state: follower="Node at c3d9dd09-9515-e49d-50fe-328d27ee0d20 [Follower]" leader= testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.444Z [WARN] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.444Z [WARN] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.444Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: entering candidate state: node="Node at c3d9dd09-9515-e49d-50fe-328d27ee0d20 [Candidate]" term=2 testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.444Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: entering candidate state: node="Node at c3d9dd09-9515-e49d-50fe-328d27ee0d20 [Candidate]" term=2 testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: election won: tally=2 testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: election won: tally=2 testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: entering leader state: leader="Node at c3d9dd09-9515-e49d-50fe-328d27ee0d20 [Leader]" testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: entering leader state: leader="Node at c3d9dd09-9515-e49d-50fe-328d27ee0d20 [Leader]" testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: added peer, starting replication: peer=server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: added peer, starting replication: peer=server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: pipelining replication: peer="{Voter server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a 00e96bd7-3d38-a6b8-e404-23bc45ebed3a}" testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.445Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: pipelining replication: peer="{Voter server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a 00e96bd7-3d38-a6b8-e404-23bc45ebed3a}" testLoggerAdapter verbose: cluster: 2021-08-25T09:31:16.550Z [INFO] stable state for Follower reached at 2021-08-25 09:31:16.445417097 +0000 UTC m=+20.711633541 (1 nodes), 68.388783ms from start of poll, 69.233864ms from cluster start. Timeout at 2021-08-25 09:31:16.550554742 +0000 UTC m=+20.816771266, 105.137725ms after stability testing.go:149: cluster: 2021-08-25T09:31:16.550Z [INFO] stable state for Follower reached at 2021-08-25 09:31:16.445417097 +0000 UTC m=+20.711633541 (1 nodes), 68.388783ms from start of poll, 69.233864ms from cluster start. Timeout at 2021-08-25 09:31:16.550554742 +0000 UTC m=+20.816771266, 105.137725ms after stability testLoggerAdapter verbose: cluster: 2021-08-25T09:31:16.550Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2021-08-25T09:31:16.550Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2021-08-25T09:31:16.656Z [INFO] stable state for Leader reached at 2021-08-25 09:31:16.550839295 +0000 UTC m=+20.817055679 (1 nodes), 2.139µs from start of poll, 174.656002ms from cluster start. Timeout at 2021-08-25 09:31:16.655969561 +0000 UTC m=+20.922186025, 105.130346ms after stability testing.go:149: cluster: 2021-08-25T09:31:16.656Z [INFO] stable state for Leader reached at 2021-08-25 09:31:16.550839295 +0000 UTC m=+20.817055679 (1 nodes), 2.139µs from start of poll, 174.656002ms from cluster start. Timeout at 2021-08-25 09:31:16.655969561 +0000 UTC m=+20.922186025, 105.130346ms after stability testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.666Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a Address:00e96bd7-3d38-a6b8-e404-23bc45ebed3a} {Suffrage:Voter ID:server-c3d9dd09-9515-e49d-50fe-328d27ee0d20 Address:c3d9dd09-9515-e49d-50fe-328d27ee0d20}]" testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.666Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a Address:00e96bd7-3d38-a6b8-e404-23bc45ebed3a} {Suffrage:Voter ID:server-c3d9dd09-9515-e49d-50fe-328d27ee0d20 Address:c3d9dd09-9515-e49d-50fe-328d27ee0d20}]" testLoggerAdapter verbose: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.667Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: aborting pipeline replication: peer="{Voter server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a 00e96bd7-3d38-a6b8-e404-23bc45ebed3a}" testing.go:149: server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: 2021-08-25T09:31:16.667Z [INFO] server-c3d9dd09-9515-e49d-50fe-328d27ee0d20: aborting pipeline replication: peer="{Voter server-00e96bd7-3d38-a6b8-e404-23bc45ebed3a 00e96bd7-3d38-a6b8-e404-23bc45ebed3a}" --- PASS: TestRaft_GetConfigurationNoBootstrap (0.29s) === RUN TestTCPTransport_BadAddr --- PASS: TestTCPTransport_BadAddr (0.00s) === RUN TestTCPTransport_WithAdvertise --- PASS: TestTCPTransport_WithAdvertise (0.00s) === RUN TestTransport_StartStop --- PASS: TestTransport_StartStop (0.00s) === RUN TestTransport_AppendEntries --- PASS: TestTransport_AppendEntries (0.00s) === RUN TestTransport_AppendEntriesPipeline --- PASS: TestTransport_AppendEntriesPipeline (0.00s) === RUN TestTransport_RequestVote --- PASS: TestTransport_RequestVote (0.00s) === RUN TestTransport_InstallSnapshot --- PASS: TestTransport_InstallSnapshot (0.00s) === RUN TestTransport_EncodeDecode --- PASS: TestTransport_EncodeDecode (0.00s) === RUN TestRandomTimeout --- PASS: TestRandomTimeout (0.00s) === RUN TestNewSeed --- PASS: TestNewSeed (0.00s) === RUN TestRandomTimeout_NoTime --- PASS: TestRandomTimeout_NoTime (0.00s) === RUN TestMin --- PASS: TestMin (0.00s) === RUN TestMax --- PASS: TestMax (0.00s) === RUN TestGenerateUUID --- PASS: TestGenerateUUID (0.01s) === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok github.com/hashicorp/raft 20.962s ? github.com/hashicorp/raft/bench [no test files] create-stamp debian/debhelper-build-stamp fakeroot debian/rules binary dh binary --buildsystem=golang --with=golang dh_testroot -O--buildsystem=golang dh_prep -O--buildsystem=golang dh_auto_install -O--buildsystem=golang dh_installdocs -O--buildsystem=golang dh_installchangelogs -O--buildsystem=golang dh_installinit -O--buildsystem=golang dh_installsystemduser -O--buildsystem=golang dh_perl -O--buildsystem=golang dh_link -O--buildsystem=golang dh_strip_nondeterminism -O--buildsystem=golang dh_compress -O--buildsystem=golang dh_fixperms -O--buildsystem=golang dh_missing -O--buildsystem=golang dh_installdeb -O--buildsystem=golang dh_golang -O--buildsystem=golang dh_gencontrol -O--buildsystem=golang dpkg-gencontrol: warning: Depends field of package golang-github-hashicorp-raft-dev: substitution variable ${shlibs:Depends} used, but is not defined dh_md5sums -O--buildsystem=golang dh_builddeb -O--buildsystem=golang dpkg-deb: building package 'golang-github-hashicorp-raft-dev' in '../golang-github-hashicorp-raft-dev_1.1.2-1_all.deb'. dpkg-genbuildinfo --build=binary dpkg-genchanges --build=binary >../golang-github-hashicorp-raft_1.1.2-1_armhf.changes dpkg-genchanges: info: binary-only upload (no source code included) dpkg-source --after-build . dpkg-buildpackage: info: binary-only upload (no source included) dpkg-genchanges: info: including full source code in upload I: copying local configuration I: user script /srv/workspace/pbuilder/7057/tmp/hooks/B01_cleanup starting I: user script /srv/workspace/pbuilder/7057/tmp/hooks/B01_cleanup finished I: unmounting dev/ptmx filesystem I: unmounting dev/pts filesystem I: unmounting dev/shm filesystem I: unmounting proc filesystem I: unmounting sys filesystem I: cleaning the build env I: removing directory /srv/workspace/pbuilder/7057 and its subdirectories I: Current time: Wed Aug 25 23:31:30 +14 2021 I: pbuilder-time-stamp: 1629883890