Triggered by Gerrit: https://gerrit.osmocom.org/c/osmo-gsm-tester/+/27601 Running as SYSTEM Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_gerrit The recommended git tool is: NONE using credential d5eda5e9-b59d-44ba-88d2-43473cb6e42d > git rev-parse --resolve-git-dir /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/.git # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Fetching upstream changes from ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester > git --version # timeout=10 > git --version # 'git version 2.20.1' using GIT_SSH to set credentials > git fetch --tags --force --progress -- ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/01/27601/1 # timeout=10 > git rev-parse 5a80d769ec542c1b578a2924f372d41e2757b3c4^{commit} # timeout=10 JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://plugins.jenkins.io/git/#remove-git-plugin-buildsbybranch-builddata-script Checking out Revision 5a80d769ec542c1b578a2924f372d41e2757b3c4 (refs/changes/01/27601/1) > git config core.sparsecheckout # timeout=10 > git checkout -f 5a80d769ec542c1b578a2924f372d41e2757b3c4 # timeout=10 Commit message: "contrib: Fix osmocom-bb-fimware no longer available in order prebuilt pkg repo" > git rev-parse FETCH_HEAD^{commit} # timeout=10 > git rev-list --no-walk ae8aa17301137f70e586478f9db5e19ab5380ea9 # timeout=10 [osmo-gsm-tester_gerrit] $ /bin/sh -xe /tmp/jenkins3140653776722019423.sh + rm -f trial-2063-bin.tgz trial-2063-run.tgz *.md5 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bsc" build number 15570 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts" build number 6578 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-ggsn" build number 5814 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hlr" build number 7536 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-mgw" build number 7731 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hnbgw" build number 592 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-msc" build number 25979 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu" build number 4550 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-sgsn" build number 26313 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-trx" build number 3840 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-nitb" build number 11307 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-stp" build number 8062 Copied 2 artifacts from "osmo-gsm-tester_build-osmocom-bb" build number 4035 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-sysmo" build number 7108 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-sysmo" build number 4589 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-oc2g" build number 4604 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-oc2g" build number 3308 Copied 2 artifacts from "osmo-gsm-tester_build-open5gs" build number 8 Copied 2 artifacts from "osmo-gsm-tester_build-srslte" build number 19 [osmo-gsm-tester_gerrit] $ /bin/sh /tmp/jenkins8606377750804127806.sh + unlink osmo-gsm-tester/sysmocom/resources.conf unlink: cannot unlink 'osmo-gsm-tester/sysmocom/resources.conf': No such file or directory + true + ln -s resources.conf.prod osmo-gsm-tester/sysmocom/resources.conf + export OSMO_GSM_TESTER_OPTS=-s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo + ./osmo-gsm-tester/contrib/jenkins-make-check-and-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit + run_script=osmo-gsm-tester/contrib/jenkins-run.sh + test -x osmo-gsm-tester/contrib/jenkins-run.sh + cd osmo-gsm-tester + make deps ./contrib/check_dependencies.py Skip checking modules: [] Changing workdir dir to /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_ms_driver importing src.osmo_ms_driver.location_update_test importing src.osmo_ms_driver.event_server importing src.osmo_ms_driver.cdf importing src.osmo_ms_driver.__main__ importing src.osmo_ms_driver.__init__ importing src.osmo_ms_driver.test_support importing src.osmo_ms_driver.starter importing src.osmo_ms_driver.simple_loop importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core importing src.osmo_gsm_tester.core.util importing src.osmo_gsm_tester.core.schema importing src.osmo_gsm_tester.core.trial importing src.osmo_gsm_tester.core.template importing src.osmo_gsm_tester.core.config importing src.osmo_gsm_tester.core.event_loop importing src.osmo_gsm_tester.core.log importing src.osmo_gsm_tester.core.scenario importing src.osmo_gsm_tester.core.remote importing src.osmo_gsm_tester.core.__init__ importing src.osmo_gsm_tester.core.process importing src.osmo_gsm_tester.core.report importing src.osmo_gsm_tester.core.test importing src.osmo_gsm_tester.core.resource importing src.osmo_gsm_tester.core.suite importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj importing src.osmo_gsm_tester.obj.android_apn importing src.osmo_gsm_tester.obj.bts importing src.osmo_gsm_tester.obj.mgcpgw_osmo importing src.osmo_gsm_tester.obj.sgsn_osmo importing src.osmo_gsm_tester.obj.epc_srs importing src.osmo_gsm_tester.obj.bts_nanobts importing src.osmo_gsm_tester.obj.enb_srs importing src.osmo_gsm_tester.obj.esme importing src.osmo_gsm_tester.obj.qc_diag importing src.osmo_gsm_tester.obj.stp_osmo importing src.osmo_gsm_tester.obj.ms_ofono importing src.osmo_gsm_tester.obj.ms_android importing src.osmo_gsm_tester.obj.osmo_vty importing src.osmo_gsm_tester.obj.hlr_osmo importing src.osmo_gsm_tester.obj.enb_amarisoft importing src.osmo_gsm_tester.obj.android_host importing src.osmo_gsm_tester.obj.sgwu_open5gs importing src.osmo_gsm_tester.obj.powersupply importing src.osmo_gsm_tester.obj.iperf3 importing src.osmo_gsm_tester.obj.epc importing src.osmo_gsm_tester.obj.rfemu_minicircuits importing src.osmo_gsm_tester.obj.bts_osmo importing src.osmo_gsm_tester.obj.upf_open5gs importing src.osmo_gsm_tester.obj.nitb_osmo importing src.osmo_gsm_tester.obj.android_bitrate_monitor importing src.osmo_gsm_tester.obj.pcu_oc2g importing src.osmo_gsm_tester.obj.ms_srs importing src.osmo_gsm_tester.obj.ms_driver importing src.osmo_gsm_tester.obj.bts_sysmo importing src.osmo_gsm_tester.obj.rfemu_srsenb_stdin importing src.osmo_gsm_tester.obj.pcu importing src.osmo_gsm_tester.obj.osmocon importing src.osmo_gsm_tester.obj.epc_amarisoft importing src.osmo_gsm_tester.obj.ggsn_osmo importing src.osmo_gsm_tester.obj.enb importing src.osmo_gsm_tester.obj.rfemu importing src.osmo_gsm_tester.obj.bts_octphy importing src.osmo_gsm_tester.obj.pcap_recorder importing src.osmo_gsm_tester.obj.sms importing src.osmo_gsm_tester.obj.bts_oc2g importing src.osmo_gsm_tester.obj.stress importing src.osmo_gsm_tester.obj.smsc importing src.osmo_gsm_tester.obj.msc_osmo importing src.osmo_gsm_tester.obj.sgwc_open5gs importing src.osmo_gsm_tester.obj.osmo_ctrl importing src.osmo_gsm_tester.obj.bts_osmotrx importing src.osmo_gsm_tester.obj.rfemu_amarisoftctrl importing src.osmo_gsm_tester.obj.ms importing src.osmo_gsm_tester.obj.powersupply_intellinet importing src.osmo_gsm_tester.obj.ms_amarisoft importing src.osmo_gsm_tester.obj.srslte_common importing src.osmo_gsm_tester.obj.powersupply_sispm importing src.osmo_gsm_tester.obj.gnuradio_zmq_broker importing src.osmo_gsm_tester.obj.pcu_sysmo importing src.osmo_gsm_tester.obj.bts_osmovirtual importing src.osmo_gsm_tester.obj.hss_open5gs importing src.osmo_gsm_tester.obj.mgw_osmo importing src.osmo_gsm_tester.obj.smf_open5gs importing src.osmo_gsm_tester.obj.pcrf_open5gs importing src.osmo_gsm_tester.obj.mme_open5gs importing src.osmo_gsm_tester.obj.bsc_osmo importing src.osmo_gsm_tester.obj.epc_open5gs importing src.osmo_gsm_tester.obj.run_node importing src.osmo_gsm_tester.obj.pcu_osmo importing src.osmo_gsm_tester.obj.rfemu_gnuradio_zmq importing src.osmo_gsm_tester.obj.ms_osmo_mobile importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester importing src.osmo_gsm_tester.testenv importing src.osmo_gsm_tester.__init__ Importing dependencies ok, all installed Retreiving list of imported modules... + make check make -C selftest check make[1]: Entering directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest' echo "export PYTHONPATH=\"/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/../src\"" > set_pythonpath ./all_tests.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/cdf_test/cdf_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/config_test/config_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/lock_test/lock_test.sh /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/log_test/log_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/process_test/process_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/rate_ctrs_test/rate_ctrs_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/report_test/report_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/resource_test/resource_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/scenario_test/scenario_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/schema_test/schema_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/sms_test/sms_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/suite_test/suite_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/template_test/template_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/trial_test/trial_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/util_test/util_test.py 15 tests ok make[1]: Leaving directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest' make check: success + ./contrib/jenkins-build-manuals.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester + export OSMO_GSM_MANUALS_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals ] + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals ] + git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals pull From https://git.osmocom.org/osmo-gsm-manuals b1318e3..5cba08b master -> origin/master Updating b1318e3..5cba08b Fast-forward common/chapters/gb-ns2.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) + git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals checkout -f HEAD Your branch is up to date with 'origin/master'. + temp=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp ] + cp -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + cd /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + make OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env) ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build build ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common common # a2x can't use a different output file. To support out-of-tree builds, # we create a symlink at /srcfile.adoc pointing at # /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc. osmo-gsm-tester-manual.adoc is the /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc, # is like basename from coreutils, and is WD. if ! [ -f osmo-gsm-tester-manual.adoc ]; then \ ln -s osmo-gsm-tester-manual.adoc osmo-gsm-tester-manual.adoc; \ fi # TEXINPUTS: find LaTeX includes like \includegraphics{./common/images/sysmocom.pdf} # in /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals. TEXINPUTS="/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals" \ a2x -vv -L --asciidoc-opts="-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'" --dblatex-opts="-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0" -a docinfo -a revnumber="DRAFT 0.1-1214-g5a80" -a revdate="2022-Mar-29" osmo-gsm-tester-manual.adoc a2x: args: ['-vv', '-L', "--asciidoc-opts=-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'", '--dblatex-opts=-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0', '-a', 'docinfo', '-a', 'revnumber=DRAFT 0.1-1214-g5a80', '-a', 'revdate=2022-Mar-29', 'osmo-gsm-tester-manual.adoc'] a2x: resource files: [] a2x: resource directories: ['/etc/asciidoc/images', '/etc/asciidoc/stylesheets'] a2x: executing: "/usr/bin/asciidoc" --backend docbook -a "a2x-format=pdf" -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common' --attribute "docinfo" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "revdate=2022-Mar-29" --verbose --out-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml" "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc" asciidoc: reading: /etc/asciidoc/asciidoc.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf asciidoc: reading: /etc/asciidoc/asciidoc.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc asciidoc: reading: /etc/asciidoc/docbook45.conf asciidoc: reading: /etc/asciidoc/filters/code/code-filter.conf asciidoc: reading: /etc/asciidoc/filters/graphviz/graphviz-filter.conf asciidoc: reading: /etc/asciidoc/filters/music/music-filter.conf asciidoc: reading: /etc/asciidoc/filters/source/source-highlight-filter.conf asciidoc: reading: /etc/asciidoc/filters/latex/latex-filter.conf asciidoc: reading: /etc/asciidoc/lang-en.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf asciidoc: writing: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml asciidoc: osmo-gsm-tester-manual.adoc: line 5: evaluating: {include:/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual-docinfo.xml} asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 13: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc asciidoc: intro.adoc: line 55: evaluating: {counter2:target-number} asciidoc: intro.adoc: line 55: evaluating: {set2:target:osmo-gsm-tester-manual__1.png} asciidoc: intro.adoc: line 103: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 15: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc asciidoc: trial.adoc: line 14: evaluating: {counter2:target-number} asciidoc: trial.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__2.png} asciidoc: trial.adoc: line 25: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 17: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 19: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc asciidoc: resource_pool.adoc: line 14: evaluating: {counter2:target-number} asciidoc: resource_pool.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__3.png} asciidoc: resource_pool.adoc: line 79: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc asciidoc: resource_pool.adoc: line 111: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc asciidoc: config_mangling.adoc: line 165: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Mar-29" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: config_mangling.adoc: line 255: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Mar-29" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: config_mangling.adoc: line 330: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Mar-29" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 21: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 23: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 25: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 27: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 29: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 31: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 33: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc a2x: executing: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0 -V "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml" Build the book set list... xsltproc -o doclist.txt --xinclude --xincludestyle doclist.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml Build the listings... xsltproc -o listings.xml --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' /usr/share/dblatex/xsl/common/mklistings.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml xsltproc -o osmo-gsm-tester-manual.rtex --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' --param listings.xml '/tmp/tmpnYNw_7/listings.xml' /tmp/tmpnYNw_7/custom.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml XSLT stylesheets DocBook - LaTeX 2e (0.3.10) =================================================== WARNING: xref to <simpara id="scenario_suite_params"> has no generated text. Trying its ancestor elements. Image 'dblatex' not found Cannot convert 'osmo-gsm-tester-manual__1.png' to pdf Cannot convert 'osmo-gsm-tester-manual__2.png' to pdf Cannot convert 'osmo-gsm-tester-manual__3.png' to pdf Build osmo-gsm-tester-manual.pdf built-in module pdftex registered no support found for ifthen no support found for ifxetex no support found for fontspec no support found for xltxtra no support found for fontenc no support found for inputenc no support found for fancybox built-in module makeidx registered no support found for custom-dblatex building additional files... checking if compiling is necessary... the output file doesn't exist pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex running post-compilation scripts... [index] the index file /tmp/tmpnYNw_7/osmo-gsm-tester-manual.idx is empty osmo-gsm-tester-manual.toc MD5 checksum changed osmo-gsm-tester-manual.aux MD5 checksum changed the /tmp/tmpnYNw_7/osmo-gsm-tester-manual.toc file has changed pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex running post-compilation scripts... [index] the index file /tmp/tmpnYNw_7/osmo-gsm-tester-manual.idx is empty no new compilation is needed running last-compilation scripts... 'osmo-gsm-tester-manual.pdf' successfully built a2x: deleting /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml + make check OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env) if [ -n "$ASCIIDOC_WARNINGS_CHECK" ]; then \ make osmo-gsm-tester-manual.check; \ fi + [ = 1 ] + rm -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + cd /home/jenkins/workspace/osmo-gsm-tester_gerrit + PATH=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games osmo-gsm-tester/contrib/jenkins-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit + dirname -- osmo-gsm-tester/contrib/jenkins-run.sh + CDPATH= cd -- osmo-gsm-tester/contrib + pwd -P + SCRIPT_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib + OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf + date +%F %T + time_start=2022-03-29 18:32:50 + trial_dir_prefix=trial- + rm -rf trial-2063 + trial_dir=trial-2064 + mkdir -p trial-2064 + mv open5gs.build-8.tgz osmo-bsc.build-15570.tgz osmo-bts-oc2g.build-4604.tgz osmo-bts-sysmo.build-7108.tgz osmo-bts.build-6578.tgz osmo-ggsn.build-5814.tgz osmo-hlr.build-7536.tgz osmo-hnbgw.build-592.tgz osmo-mgw.build-7731.tgz osmo-msc.build-25979.tgz osmo-nitb.build-11307.tgz osmo-pcu-oc2g.build-3308.tgz osmo-pcu-sysmo.build-4589.tgz osmo-pcu.build-4550.tgz osmo-sgsn.build-26313.tgz osmo-stp.build-8062.tgz osmo-trx.build-3840.tgz osmocom-bb.build-4035.tgz srslte.build-19.tgz trial-2064 + cat open5gs.build-8.md5 osmo-bsc.build-15570.md5 osmo-bts-oc2g.build-4604.md5 osmo-bts-sysmo.build-7108.md5 osmo-bts.build-6578.md5 osmo-ggsn.build-5814.md5 osmo-hlr.build-7536.md5 osmo-hnbgw.build-592.md5 osmo-mgw.build-7731.md5 osmo-msc.build-25979.md5 osmo-nitb.build-11307.md5 osmo-pcu-oc2g.build-3308.md5 osmo-pcu-sysmo.build-4589.md5 osmo-pcu.build-4550.md5 osmo-sgsn.build-26313.md5 osmo-stp.build-8062.md5 osmo-trx.build-3840.md5 osmocom-bb.build-4035.md5 srslte.build-19.md5 + rm open5gs.build-8.md5 osmo-bsc.build-15570.md5 osmo-bts-oc2g.build-4604.md5 osmo-bts-sysmo.build-7108.md5 osmo-bts.build-6578.md5 osmo-ggsn.build-5814.md5 osmo-hlr.build-7536.md5 osmo-hnbgw.build-592.md5 osmo-mgw.build-7731.md5 osmo-msc.build-25979.md5 osmo-nitb.build-11307.md5 osmo-pcu-oc2g.build-3308.md5 osmo-pcu-sysmo.build-4589.md5 osmo-pcu.build-4550.md5 osmo-sgsn.build-26313.md5 osmo-stp.build-8062.md5 osmo-trx.build-3840.md5 osmocom-bb.build-4035.md5 srslte.build-19.md5 + exit_code=1 + which osmo-gsm-tester.py + python3 -u /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo-gsm-tester.py -c /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf trial-2064 -s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo combinations: ['nitb_sms:sysmo', 'sms:sysmo', 'gprs:sysmo'] trial: 'trial-2064' tests: None -------------------------------------------------------------------------------------------- trial-2064 -------------------------------------------------------------------------------------------- 18:32:52.346499 tst trial-2064: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064/run.2022-03-29_18-32-52/log --------------------------------------------------------------------- trial-2064 nitb_sms:sysmo --------------------------------------------------------------------- 18:32:55.173840 tst nitb_sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 18:32:55.177601 tst nitb_sms:sysmo: Verifying 1 x bts (candidates: 11) 18:32:55.198970 tst nitb_sms:sysmo: Verifying 1 x ip_address (candidates: 9) 18:32:55.206105 tst nitb_sms:sysmo: Verifying 2 x modem (candidates: 12) 18:32:55.263287 tst nitb_sms:sysmo: Reserving 1 x bts (candidates: 11) 18:32:55.285008 tst nitb_sms:sysmo: Reserving 1 x ip_address (candidates: 9) 18:32:55.291918 tst nitb_sms:sysmo: Reserving 2 x modem (candidates: 12) 18:32:55.831054 tst nitb_sms:sysmo: RESERVED RESOURCES for nitb_sms: bts sysmoBTS 1002 ip_address 10.42.42.2 modem sierra_1st sierra_2nd ---------------------------------------------- trial-2064 nitb_sms:sysmo mo_mt_sms.py ---------------------------------------------- 18:32:55.838323 tst nitb_sms:sysmo: Using 1 x ip_address (candidates: 1) 18:32:55.849461 tst nitb_sms:sysmo: Using 1 x bts (candidates: 1) 18:32:56.226482 tst mo_mt_sms.py: using LAC 3943 18:32:56.472628 tst mo_mt_sms.py: using RAC 118 18:32:56.594157 tst mo_mt_sms.py: using CellId 3943 18:32:56.714254 tst mo_mt_sms.py: using BVCI 3944 18:32:56.826162 tst nitb_sms:sysmo: Using 1 x modem (candidates: 2) 18:32:57.032268 tst nitb_sms:sysmo: Using 1 x modem (candidates: 2) 18:32:57.148887 tst mo_mt_sms.py:9: start nitb and bts... 18:32:57.363011 run osmo-nitb_10.42.42.2: Starting osmo-nitb 18:32:59.729286 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=26639): Launched 18:33:00.052345 run osmo-nitb_10.42.42.2(pid=26640): Launched 18:33:00.228664 run osmo-nitb_10.42.42.2(pid=26640): ERR: Terminated: ERROR {rc=2} [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=26640)] 18:33:00.359508 run osmo-nitb_10.42.42.2(pid=26640): stdout: | (launched: 2022-03-29_18:32:59.992947) 18:33:00.492150 run osmo-nitb_10.42.42.2(pid=26640): stderr: | (launched: 2022-03-29_18:33:00.040289) | ====================================================================== | ERROR: You should not run osmo-nitb. It is ancient, unmaintained and | obsolete for several years. If you really want to ignore years of hard work on | the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp), | you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line | argument. DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!! | Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box | ====================================================================== 18:33:00.624403 run osmo-nitb_10.42.42.2(pid=26640): stdout: | (launched: 2022-03-29_18:32:59.992947) 18:33:00.751165 run osmo-nitb_10.42.42.2(pid=26640): stderr: | (launched: 2022-03-29_18:33:00.040289) | ====================================================================== | ERROR: You should not run osmo-nitb. It is ancient, unmaintained and | obsolete for several years. If you really want to ignore years of hard work on | the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp), | you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line | argument. DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!! | Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box | ====================================================================== 18:33:00.902948 tst mo_mt_sms.py:11: ERR: Error: osmo-nitb_10.42.42.2(pid=26640): Process ended prematurely: osmo-nitb_10.42.42.2(pid=26640) [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=26640)] [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11] 18:33:00.911748 tst mo_mt_sms.py:11: Test FAILED (5.1 sec) 18:33:00.918510 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=26639): Terminating (SIGTERM) 18:33:01.129201 --- ParallelTerminationStrategy: PID 26639 died... 18:33:01.150897 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=26639): Terminated: ok {rc=0} 18:33:01.239280 bus /sierra_3: Setting Powered False 18:33:02.339581 bus /sierra_3: Setting Powered False 18:33:03.459342 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-2064 nitb_sms:sysmo FAIL --------------------------------------------------------------------- --------------------------------------------------------------------- trial-2064 sms:sysmo --------------------------------------------------------------------- 18:33:04.637858 tst sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 18:33:04.641828 tst sms:sysmo: Verifying 1 x bts (candidates: 11) 18:33:04.663298 tst sms:sysmo: Verifying 6 x ip_address (candidates: 9) 18:33:04.698672 tst sms:sysmo: Verifying 2 x modem (candidates: 12) 18:33:04.753423 tst sms:sysmo: Reserving 1 x bts (candidates: 11) 18:33:04.775146 tst sms:sysmo: Reserving 6 x ip_address (candidates: 9) 18:33:04.802770 tst sms:sysmo: Reserving 2 x modem (candidates: 12) 18:33:05.351204 tst sms:sysmo: RESERVED RESOURCES for sms: bts sysmoBTS 1002 ip_address 10.42.42.2 10.42.42.3 10.42.42.4 10.42.42.5 10.42.42.6 10.42.42.7 modem sierra_1st sierra_2nd ---------------------------------------------- trial-2064 sms:sysmo mo_mt_sms.py ---------------------------------------------- 18:33:05.359402 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 18:33:05.370437 tst sms:sysmo: Using 1 x bts (candidates: 1) 18:33:05.405248 tst mo_mt_sms.py: using LAC 3944 18:33:05.529739 tst mo_mt_sms.py: using RAC 119 18:33:05.655003 tst mo_mt_sms.py: using CellId 3944 18:33:05.779286 tst mo_mt_sms.py: using BVCI 3945 18:33:05.887805 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 18:33:05.898529 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 18:33:05.909255 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 18:33:05.919664 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 18:33:05.930257 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 18:33:05.941084 tst sms:sysmo: Using 1 x modem (candidates: 2) 18:33:06.059606 tst sms:sysmo: Using 1 x modem (candidates: 2) 18:33:06.174614 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:33:07.790785 run create_hlr_db(pid=26644): Launched 18:33:07.904797 bus /sierra_2: Setting Powered False 18:33:08.967750 run create_hlr_db(pid=26644): Terminated: ok {rc=0} 18:33:09.396732 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26646): Launched 18:33:09.720587 run osmo-hlr_10.42.42.2(pid=26647): Launched 18:33:09.845902 run osmo-stp_10.42.42.5: Starting osmo-stp 18:33:12.420857 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26648): Launched 18:33:12.741689 run osmo-stp_10.42.42.5(pid=26649): Launched 18:33:12.862089 run osmo-msc_10.42.42.6: Starting osmo-msc 18:33:18.319175 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=26650): Launched 18:33:18.642224 run osmo-msc_10.42.42.6(pid=26651): Launched 18:33:21.820694 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:33:23.546630 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26652): Launched 18:33:23.868579 run osmo-mgw_10.42.42.3(pid=26653): Launched 18:33:23.993165 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:33:24.529749 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26654): Launched 18:33:24.851313 run osmo-mgw_10.42.42.4(pid=26655): Launched 18:33:24.972029 run osmo-bsc_10.42.42.7: Starting osmo-bsc 18:33:29.475994 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=26656): Launched 18:33:29.798586 run osmo-bsc_10.42.42.7(pid=26657): Launched 18:33:36.082359 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 18:33:37.027646 run rm-remote-dir(pid=26658): Launched 18:33:39.219942 run rm-remote-dir(pid=26658): Terminated: ok {rc=0} 18:33:39.572658 run mk-remote-dir(pid=26659): Launched 18:33:40.761198 run mk-remote-dir(pid=26659): Terminated: ok {rc=0} 18:33:41.066341 run scp-inst-to-remote(pid=26660): Launched 18:33:53.272603 run scp-inst-to-remote(pid=26660): Terminated: ok {rc=0} 18:33:53.619887 run mk-remote-dir(pid=26662): Launched 18:33:54.803456 run mk-remote-dir(pid=26662): Terminated: ok {rc=0} 18:33:55.102300 run scp-cfg-to-remote(pid=26663): Launched 18:33:56.285855 run scp-cfg-to-remote(pid=26663): Terminated: ok {rc=0} 18:33:56.607179 run reload-dsp-firmware(pid=26665): Launched 18:33:58.779333 run reload-dsp-firmware(pid=26665): Terminated: ok {rc=0} 18:33:59.094218 run osmo-bts-sysmo(pid=26666): Launched 18:34:03.324933 tst mo_mt_sms.py: using MSISDN 3947 18:34:03.488533 bus /sierra_2: Setting Powered True 18:34:06.957432 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='3947', subscriber_id=1} 18:34:07.088860 tst mo_mt_sms.py: using MSISDN 3948 18:34:07.260917 bus /sierra_3: Setting Powered True 18:34:10.731625 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='3948', subscriber_id=2} 18:34:10.997049 bus /sierra_2: Setting Online False 18:34:12.185982 bus /sierra_2: Setting Powered False 18:34:18.052828 bus /sierra_2: Setting Powered True 18:34:19.386505 bus /sierra_2: Setting Online True 18:34:21.903790 tst /sierra_2: Connect to ('901', '70') 18:34:22.195195 bus /sierra_3: Setting Online False 18:34:23.380896 bus /sierra_3: Setting Powered False 18:34:29.052113 bus /sierra_3: Setting Powered True 18:34:30.397256 bus /sierra_3: Setting Online True 18:34:32.940422 tst /sierra_3: Connect to ('901', '70') 18:34:33.112734 tst /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053' 18:34:33.233944 tst /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028' 18:34:33.347198 tst mo_mt_sms.py:35: waiting for modems to attach... 18:34:36.680742 tst /sierra_2: Already registered with network ('901', '70') 18:34:49.476641 tst /sierra_3: Already registered with network ('901', '70') 18:34:54.852412 tst /sierra_2: sending sms to MSISDN 3948 {sms='message nr. 1, from /sierra_2, to /sierra_3, from 3947, to 3948'} 18:35:01.327379 tst /sierra_3: Incoming SMS: 'message nr. 1, from /sierra_2, to /sierra_3, from 3947, to 3948' 18:35:01.971975 tst /sierra_3: SMS received as expected: 'message nr. 1, from /sierra_2, to /sierra_3, from 3947, to 3948' 18:35:02.137193 tst mo_mt_sms.py: Test passed (116.8 sec) 18:35:02.201277 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26646): Terminating (SIGTERM) 18:35:02.241347 run osmo-hlr_10.42.42.2(pid=26647): Terminating (SIGTERM) 18:35:02.281198 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26648): Terminating (SIGTERM) 18:35:02.322983 run osmo-stp_10.42.42.5(pid=26649): Terminating (SIGTERM) 18:35:02.371438 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=26650): Terminating (SIGTERM) 18:35:02.419434 run osmo-msc_10.42.42.6(pid=26651): Terminating (SIGTERM) 18:35:02.465743 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26652): Terminating (SIGTERM) 18:35:02.511231 run osmo-mgw_10.42.42.3(pid=26653): Terminating (SIGTERM) 18:35:02.562968 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26654): Terminating (SIGTERM) 18:35:02.613458 run osmo-mgw_10.42.42.4(pid=26655): Terminating (SIGTERM) 18:35:02.667012 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=26656): Terminating (SIGTERM) 18:35:02.717952 run osmo-bsc_10.42.42.7(pid=26657): Terminating (SIGTERM) 18:35:02.762008 run osmo-bts-sysmo(pid=26666): Terminating (SIGTERM) 18:35:02.806993 --- ParallelTerminationStrategy: PID 26646 died... 18:35:02.828899 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26646): Terminated: ok {rc=0} 18:35:02.876312 --- ParallelTerminationStrategy: PID 26647 died... 18:35:02.896064 run osmo-hlr_10.42.42.2(pid=26647): Terminated: ok {rc=0} 18:35:02.943192 --- ParallelTerminationStrategy: PID 26648 died... 18:35:02.962919 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26648): Terminated: ok {rc=0} 18:35:03.007740 --- ParallelTerminationStrategy: PID 26649 died... 18:35:03.026792 run osmo-stp_10.42.42.5(pid=26649): Terminated: ok {rc=0} 18:35:03.071827 --- ParallelTerminationStrategy: PID 26650 died... 18:35:03.090596 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=26650): Terminated: ok {rc=0} 18:35:03.135583 --- ParallelTerminationStrategy: PID 26651 died... 18:35:03.154335 run osmo-msc_10.42.42.6(pid=26651): Terminated: ok {rc=0} 18:35:03.198582 --- ParallelTerminationStrategy: PID 26652 died... 18:35:03.217824 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26652): Terminated: ok {rc=0} 18:35:03.262429 --- ParallelTerminationStrategy: PID 26653 died... 18:35:03.281149 run osmo-mgw_10.42.42.3(pid=26653): Terminated {rc=15} 18:35:03.325432 --- ParallelTerminationStrategy: PID 26654 died... 18:35:03.344203 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26654): Terminated: ok {rc=0} 18:35:03.388567 --- ParallelTerminationStrategy: PID 26655 died... 18:35:03.407445 run osmo-mgw_10.42.42.4(pid=26655): Terminated {rc=15} 18:35:03.451788 --- ParallelTerminationStrategy: PID 26656 died... 18:35:03.470585 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=26656): Terminated: ok {rc=0} 18:35:03.515318 --- ParallelTerminationStrategy: PID 26657 died... 18:35:03.534154 run osmo-bsc_10.42.42.7(pid=26657): Terminated: ok {rc=0} 18:35:03.578489 --- ParallelTerminationStrategy: PID 26666 died... 18:35:03.597386 run osmo-bts-sysmo(pid=26666): Terminated: ok {rc=0} 18:35:03.724493 bus /sierra_3: Setting Online False 18:35:04.839909 bus /sierra_3: Setting Powered False 18:35:15.011295 bus /sierra_3: Setting Powered False 18:35:16.108714 bus /sierra_3: Setting Powered False 18:35:17.268015 bus /sierra_2: Setting Online False 18:35:18.419827 bus /sierra_2: Setting Powered False 18:35:26.915106 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-2064 sms:sysmo PASS --------------------------------------------------------------------- --------------------------------------------------------------------- trial-2064 gprs:sysmo --------------------------------------------------------------------- 18:35:28.182892 tst gprs:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 18:35:28.187551 tst gprs:sysmo: Verifying 1 x bts (candidates: 11) 18:35:28.209858 tst gprs:sysmo: Verifying 9 x ip_address (candidates: 9) 18:35:28.249976 tst gprs:sysmo: Verifying 4 x modem (candidates: 12) 18:35:28.337631 tst gprs:sysmo: Reserving 1 x bts (candidates: 11) 18:35:28.359258 tst gprs:sysmo: Reserving 9 x ip_address (candidates: 9) 18:35:28.399139 tst gprs:sysmo: Reserving 4 x modem (candidates: 12) 18:35:28.990568 tst gprs:sysmo: RESERVED RESOURCES for gprs: bts sysmoBTS 1002 ip_address 10.42.42.2 10.42.42.3 10.42.42.4 10.42.42.5 10.42.42.6 10.42.42.7 10.42.42.8 10.42.42.9 10.42.42.10 modem ec20_1 ec20_2 ec20_3 ec20_4 ---------------------------------------------- trial-2064 gprs:sysmo cs_paging_gprs_active.py ---------------------------------------------- 18:35:29.053199 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.064283 tst gprs:sysmo: Using 1 x bts (candidates: 1) 18:35:29.100023 tst cs_paging_gprs_active.py: using LAC 3945 18:35:29.228744 tst cs_paging_gprs_active.py: using RAC 120 18:35:29.355187 tst cs_paging_gprs_active.py: using CellId 3945 18:35:29.477550 tst cs_paging_gprs_active.py: using BVCI 3946 18:35:29.588388 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.599040 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.609540 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.619984 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.630211 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.640539 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.651547 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.661876 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:35:29.672477 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:35:29.791298 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:35:29.911059 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003... 18:35:30.133666 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:35:30.465363 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Launched 18:35:30.756878 run iperf3-srv_10.42.42.10(pid=26682): Launched 18:35:30.880977 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004... 18:35:31.102852 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:35:31.432791 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Launched 18:35:31.718373 run iperf3-srv_10.42.42.10(pid=26684): Launched 18:35:31.842361 tst cs_paging_gprs_active.py:38: start network... 18:35:32.063125 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:35:32.570667 run create_hlr_db(pid=26685): Launched 18:35:32.696672 bus /sierra_2: Setting Powered False 18:35:33.761210 run create_hlr_db(pid=26685): Terminated: ok {rc=0} 18:35:34.199900 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Launched 18:35:34.507072 run osmo-hlr_10.42.42.2(pid=26688): Launched 18:35:34.623423 run osmo-stp_10.42.42.5: Starting osmo-stp 18:35:35.147529 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Launched 18:35:35.453834 run osmo-stp_10.42.42.5(pid=26690): Launched 18:35:35.570249 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 18:35:37.089080 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Launched 18:35:37.206677 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 18:35:37.470046 run patchelf(pid=26692): Launched 18:35:38.645784 run patchelf(pid=26692): Terminated: ok {rc=0} 18:35:38.774375 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 18:35:39.039678 run osmo-gsm-tester_setcap_net_admin.sh(pid=26693): Launched 18:35:40.220718 run osmo-gsm-tester_setcap_net_admin.sh(pid=26693): Terminated: ok {rc=0} 18:35:40.539550 run osmo-ggsn_10.42.42.6(pid=26696): Launched 18:35:40.665713 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 18:35:45.549008 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Launched 18:35:45.855774 run osmo-sgsn_10.42.42.7(pid=26716): Launched 18:35:45.973605 run osmo-msc_10.42.42.8: Starting osmo-msc 18:35:46.576368 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Launched 18:35:46.906062 run osmo-msc_10.42.42.8(pid=26718): Launched 18:35:50.078466 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:35:50.651513 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Launched 18:35:50.990474 run osmo-mgw_10.42.42.3(pid=26720): Launched 18:35:51.117211 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:35:51.674573 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Launched 18:35:52.009110 run osmo-mgw_10.42.42.4(pid=26722): Launched 18:35:52.135863 run osmo-bsc_10.42.42.9: Starting osmo-bsc 18:35:52.815800 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Launched 18:35:53.164683 run osmo-bsc_10.42.42.9(pid=26724): Launched 18:35:59.457431 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 18:36:00.030105 run rm-remote-dir(pid=26725): Launched 18:36:02.225781 run rm-remote-dir(pid=26725): Terminated: ok {rc=0} 18:36:02.587660 run mk-remote-dir(pid=26726): Launched 18:36:03.782132 run mk-remote-dir(pid=26726): Terminated: ok {rc=0} 18:36:04.091239 run scp-inst-to-remote(pid=26727): Launched 18:36:16.299433 run scp-inst-to-remote(pid=26727): Terminated: ok {rc=0} 18:36:16.632145 run mk-remote-dir(pid=26729): Launched 18:36:17.811202 run mk-remote-dir(pid=26729): Terminated: ok {rc=0} 18:36:18.099043 run scp-cfg-to-remote(pid=26730): Launched 18:36:19.278230 run scp-cfg-to-remote(pid=26730): Terminated: ok {rc=0} 18:36:19.607837 run reload-dsp-firmware(pid=26732): Launched 18:36:21.785104 run reload-dsp-firmware(pid=26732): Terminated: ok {rc=0} 18:36:22.107676 run osmo-bts-sysmo(pid=26733): Launched 18:36:26.335161 tst cs_paging_gprs_active.py:38: Waiting for bts to be ready... 18:36:28.358345 run rm-remote-dir(pid=26735): Launched 18:36:30.544612 run rm-remote-dir(pid=26735): Terminated: ok {rc=0} 18:36:30.892086 run mk-remote-dir(pid=26736): Launched 18:36:33.077977 run mk-remote-dir(pid=26736): Terminated: ok {rc=0} 18:36:33.378680 run scp-inst-to-bts(pid=26737): Launched 18:36:46.586554 run scp-inst-to-bts(pid=26737): Terminated: ok {rc=0} 18:36:46.933894 run mk-remote-run-dir(pid=26739): Launched 18:36:49.117396 run mk-remote-run-dir(pid=26739): Terminated: ok {rc=0} 18:36:49.396757 run scp-cfg-to-sysmobts(pid=26740): Launched 18:36:51.572576 run scp-cfg-to-sysmobts(pid=26740): Terminated: ok {rc=0} 18:36:51.898339 run osmo-pcu(pid=26742): Launched 18:36:52.026297 tst cs_paging_gprs_active.py: using MSISDN 3949 18:36:52.202811 bus /gobi_6: Setting Powered True 18:36:55.699584 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3949', subscriber_id=1} 18:36:55.975599 bus /gobi_6: Setting Online False 18:36:57.163713 bus /gobi_6: Setting Powered False 18:37:04.112610 bus /gobi_6: Setting Powered True 18:37:05.505700 bus /gobi_6: Setting Online True 18:37:07.931860 tst /gobi_6: Connect to ('901', '70') 18:37:08.219441 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 18:37:08.345572 tst cs_paging_gprs_active.py: using MSISDN 3950 18:37:08.619084 bus /gobi_0: Setting Powered True 18:37:12.112074 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3950', subscriber_id=2} 18:37:12.385508 bus /gobi_0: Setting Online False 18:37:13.579550 bus /gobi_0: Setting Powered False 18:37:20.323005 bus /gobi_0: Setting Powered True 18:37:21.694558 bus /gobi_0: Setting Online True 18:37:23.400941 tst /gobi_6: Already registered with network ('901', '70') 18:37:24.203602 tst /gobi_0: Connect to ('901', '70') 18:37:24.480896 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873' 18:37:24.596348 tst cs_paging_gprs_active.py:38: waiting for modems to attach... 18:37:38.011549 tst /gobi_0: Already registered with network ('901', '70') 18:37:43.128601 tst cs_paging_gprs_active.py:38: waiting for modems to attach to data services... 18:37:46.281134 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:37:46.398467 tst cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_6/context2'" 18:37:46.842011 run move_netns(pid=26746): Launched 18:37:48.040448 run move_netns(pid=26746): Terminated: ok {rc=0} 18:37:48.370869 run ifup(pid=26749): Launched 18:37:49.548570 run ifup(pid=26749): Terminated: ok {rc=0} 18:37:49.879871 run dhcp(pid=26752): Launched 18:37:51.056449 run dhcp(pid=26752): Terminated: ok {rc=0} 18:37:59.436372 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:37:59.553538 tst cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_0/context3'" 18:37:59.998060 run move_netns(pid=26770): Launched 18:38:01.184894 run move_netns(pid=26770): Terminated: ok {rc=0} 18:38:01.535692 run ifup(pid=26773): Launched 18:38:02.723339 run ifup(pid=26773): Terminated: ok {rc=0} 18:38:03.073258 run dhcp(pid=26776): Launched 18:38:04.257763 run dhcp(pid=26776): Terminated: ok {rc=0} 18:38:04.388039 tst cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'" 18:38:04.602998 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 18:38:04.972753 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Launched 18:38:05.168986 tst cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context3'" 18:38:05.393125 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s) 18:38:05.781897 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Launched 18:38:06.120090 run iperf3-cli_10.42.42.10(pid=26798): Launched 18:38:06.405457 run iperf3-cli_10.42.42.10(pid=26801): Launched 18:38:06.534218 tst cs_paging_gprs_active.py:13: waiting a few secs to make sure iperf3 test is running 18:38:08.773866 tst cs_paging_gprs_active.py:16: iperf3 running, let's place a call 18:38:13.491195 tst cs_paging_gprs_active.py:22: dial success 18:38:16.313230 tst cs_paging_gprs_active.py:28: answer success, call established and ongoing 18:38:22.970525 tst cs_paging_gprs_active.py:35: hangup success 18:39:15.327878 run iperf3-cli_10.42.42.10(pid=26798): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=26798)] 18:39:15.459964 run iperf3-cli_10.42.42.10(pid=26798): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 18:39:15.592854 run iperf3-cli_10.42.42.10(pid=26798): stderr: | (launched: 2022-03-29_18:38:06.104023) 18:39:16.781776 run iperf3-cli_10.42.42.10(pid=26801): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=26801)] 18:39:16.916973 run iperf3-cli_10.42.42.10(pid=26801): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 18:39:17.047789 run iperf3-cli_10.42.42.10(pid=26801): stderr: | (launched: 2022-03-29_18:38:06.389480) 18:39:17.180135 run iperf3-srv_10.42.42.10(pid=26682): Terminating (SIGINT) 18:39:17.516776 run iperf3-srv_10.42.42.10(pid=26682): Terminated {rc=1} 18:39:17.638473 tst cs_paging_gprs_active.py:38: Results for 10.42.42.10:5003 through "'/gobi_6/context2'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42828, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:38:07 GMT', 'timesecs': 1648571887}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'eqam3drdr7ify2k6fvilb6zyzmko7t7hyhnl', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0011892318725586, 'seconds': 1.0011892318725586, 'bytes': 101360, 'bits_per_second': 809916.8211022239, 'retransmits': 4, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0011892318725586, 'seconds': 1.0011892318725586, 'bytes': 101360, 'bits_per_second': 809916.8211022239, 'retransmits': 4, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0011892318725586, 'end': 2.001164197921753, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0011892318725586, 'end': 2.001164197921753, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.001164197921753, 'end': 3.001214027404785, 'seconds': 1.0000498294830322, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.001164197921753, 'end': 3.001214027404785, 'seconds': 1.0000498294830322, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.001214027404785, 'end': 4.00121808052063, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.001214027404785, 'end': 4.00121808052063, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.00121808052063, 'end': 5.001165151596069, 'seconds': 0.9999470710754395, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.00121808052063, 'end': 5.001165151596069, 'seconds': 0.9999470710754395, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001165151596069, 'end': 6.001265048980713, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001165151596069, 'end': 6.001265048980713, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001265048980713, 'end': 7.001306056976318, 'seconds': 1.0000410079956055, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001265048980713, 'end': 7.001306056976318, 'seconds': 1.0000410079956055, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001306056976318, 'end': 8.001264095306396, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001306056976318, 'end': 8.001264095306396, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001264095306396, 'end': 9.001306056976318, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001264095306396, 'end': 9.001306056976318, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001306056976318, 'end': 10.001192092895508, 'seconds': 0.9998860359191895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001306056976318, 'end': 10.001192092895508, 'seconds': 0.9998860359191895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001192092895508, 'end': 11.001074075698853, 'seconds': 0.9998819828033447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 14480, 'rtt': 3171415, 'rttvar': 4761518, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001192092895508, 'end': 11.001074075698853, 'seconds': 0.9998819828033447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001074075698853, 'end': 12.00110912322998, 'seconds': 1.000035047531128, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 4160072, 'rttvar': 5548452, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001074075698853, 'end': 12.00110912322998, 'seconds': 1.000035047531128, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.00110912322998, 'end': 13.001227140426636, 'seconds': 1.0001180171966553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 5177411, 'rttvar': 6196018, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.00110912322998, 'end': 13.001227140426636, 'seconds': 1.0001180171966553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001227140426636, 'end': 14.001306056976318, 'seconds': 1.0000789165496826, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 13032, 'rtt': 7130959, 'rttvar': 6897406, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001227140426636, 'end': 14.001306056976318, 'seconds': 1.0000789165496826, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001306056976318, 'end': 15.001320123672485, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 13032, 'rtt': 8059624, 'rttvar': 7030385, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001306056976318, 'end': 15.001320123672485, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001320123672485, 'end': 16.001228094100952, 'seconds': 0.9999079704284668, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 11584, 'rtt': 8959227, 'rttvar': 7071995, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001320123672485, 'end': 16.001228094100952, 'seconds': 0.9999079704284668, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001228094100952, 'end': 17.001160144805908, 'seconds': 0.999932050704956, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 10136, 'rtt': 10696389, 'rttvar': 7012272, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001228094100952, 'end': 17.001160144805908, 'seconds': 0.999932050704956, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001160144805908, 'end': 18.001296043395996, 'seconds': 1.000135898590088, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 10136, 'rtt': 10967200, 'rttvar': 5800825, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001160144805908, 'end': 18.001296043395996, 'seconds': 1.000135898590088, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001296043395996, 'end': 19.001232147216797, 'seconds': 0.9999361038208008, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 11584, 'rtt': 10210576, 'rttvar': 4576076, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001296043395996, 'end': 19.001232147216797, 'seconds': 0.9999361038208008, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.001232147216797, 'end': 20.00131916999817, 'seconds': 1.000087022781372, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 11584, 'rtt': 9834182, 'rttvar': 4184844, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.001232147216797, 'end': 20.00131916999817, 'seconds': 1.000087022781372, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.00131916999817, 'end': 21.001143217086792, 'seconds': 0.999824047088623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 13032, 'rtt': 9021949, 'rttvar': 3761266, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.00131916999817, 'end': 21.001143217086792, 'seconds': 0.999824047088623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001143217086792, 'end': 22.001182079315186, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 8512041, 'rttvar': 3771203, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001143217086792, 'end': 22.001182079315186, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001182079315186, 'end': 23.00033211708069, 'seconds': 0.9991500377655029, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 8065902, 'rttvar': 3720679, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001182079315186, 'end': 23.00033211708069, 'seconds': 0.9991500377655029, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.00033211708069, 'end': 24.00107502937317, 'seconds': 1.0007429122924805, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 7328549, 'rttvar': 3370981, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.00033211708069, 'end': 24.00107502937317, 'seconds': 1.0007429122924805, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.00107502937317, 'end': 25.001262187957764, 'seconds': 1.0001871585845947, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 7120323, 'rttvar': 2944687, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.00107502937317, 'end': 25.001262187957764, 'seconds': 1.0001871585845947, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001262187957764, 'end': 26.001070022583008, 'seconds': 0.9998078346252441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 6777745, 'rttvar': 2251565, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001262187957764, 'end': 26.001070022583008, 'seconds': 0.9998078346252441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.001070022583008, 'end': 27.00107502937317, 'seconds': 1.0000050067901611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 6723145, 'rttvar': 1797874, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.001070022583008, 'end': 27.00107502937317, 'seconds': 1.0000050067901611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00107502937317, 'end': 28.00122618675232, 'seconds': 1.0001511573791504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 6700290, 'rttvar': 1394116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00107502937317, 'end': 28.00122618675232, 'seconds': 1.0001511573791504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.00122618675232, 'end': 29.00107717514038, 'seconds': 0.9998509883880615, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 6699805, 'rttvar': 1046556, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.00122618675232, 'end': 29.00107717514038, 'seconds': 0.9998509883880615, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.00107717514038, 'end': 30.00109314918518, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 6875469, 'rttvar': 893771, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.00107717514038, 'end': 30.00109314918518, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.00109314918518, 'end': 31.001081228256226, 'seconds': 0.9999880790710449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 6951140, 'rttvar': 821669, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.00109314918518, 'end': 31.001081228256226, 'seconds': 0.9999880790710449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.001081228256226, 'end': 32.00107407569885, 'seconds': 0.999992847442627, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 7249288, 'rttvar': 980299, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.001081228256226, 'end': 32.00107407569885, 'seconds': 0.999992847442627, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00107407569885, 'end': 33.0010941028595, 'seconds': 1.0000200271606445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 7367546, 'rttvar': 971741, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00107407569885, 'end': 33.0010941028595, 'seconds': 1.0000200271606445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.0010941028595, 'end': 34.0011100769043, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 7556482, 'rttvar': 1106678, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.0010941028595, 'end': 34.0011100769043, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.0011100769043, 'end': 35.00132417678833, 'seconds': 1.0002140998840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 7861612, 'rttvar': 1151562, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.0011100769043, 'end': 35.00132417678833, 'seconds': 1.0002140998840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00132417678833, 'end': 36.00116300582886, 'seconds': 0.9998388290405273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 8074202, 'rttvar': 1288852, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00132417678833, 'end': 36.00116300582886, 'seconds': 0.9998388290405273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00116300582886, 'end': 37.00122618675232, 'seconds': 1.000063180923462, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 8397018, 'rttvar': 1277904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00116300582886, 'end': 37.00122618675232, 'seconds': 1.000063180923462, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00122618675232, 'end': 38.00110912322998, 'seconds': 0.9998829364776611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8604892, 'rttvar': 1374175, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00122618675232, 'end': 38.00110912322998, 'seconds': 0.9998829364776611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00110912322998, 'end': 39.0010712146759, 'seconds': 0.9999620914459229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 8903011, 'rttvar': 1288297, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00110912322998, 'end': 39.0010712146759, 'seconds': 0.9999620914459229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0010712146759, 'end': 40.001259088516235, 'seconds': 1.000187873840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9122079, 'rttvar': 1404358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0010712146759, 'end': 40.001259088516235, 'seconds': 1.000187873840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.001259088516235, 'end': 41.001121044158936, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 9311519, 'rttvar': 1432149, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.001259088516235, 'end': 41.001121044158936, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.001121044158936, 'end': 42.0011100769043, 'seconds': 0.9999890327453613, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 9700066, 'rttvar': 1502442, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.001121044158936, 'end': 42.0011100769043, 'seconds': 0.9999890327453613, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.0011100769043, 'end': 43.000755071640015, 'seconds': 0.9996449947357178, 'bytes': 63712, 'bits_per_second': 509877.0090223394, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 9899537, 'rttvar': 1525774, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.0011100769043, 'end': 43.000755071640015, 'seconds': 0.9996449947357178, 'bytes': 63712, 'bits_per_second': 509877.0090223394, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.000755071640015, 'end': 44.00115919113159, 'seconds': 1.0004041194915771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 10307435, 'rttvar': 1589286, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.000755071640015, 'end': 44.00115919113159, 'seconds': 1.0004041194915771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00115919113159, 'end': 45.00115919113159, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 10514235, 'rttvar': 1605563, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00115919113159, 'end': 45.00115919113159, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00115919113159, 'end': 46.00112318992615, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 10696819, 'rttvar': 1569342, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00115919113159, 'end': 46.00112318992615, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00112318992615, 'end': 47.00115919113159, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 11161544, 'rttvar': 1688485, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00112318992615, 'end': 47.00115919113159, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 18:39:17.862752 tst cs_paging_gprs_active.py:38: ERR: KeyError: 'sum_sent' [trial-2064↪gprs:sysmo↪cs_paging_gprs_active.py:38] 18:39:17.875668 tst cs_paging_gprs_active.py:38: Test FAILED (228.9 sec) 18:39:17.883131 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Terminating (SIGTERM) 18:39:17.928872 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Terminating (SIGTERM) 18:39:17.972357 run iperf3-srv_10.42.42.10(pid=26684): Terminating (SIGTERM) 18:39:18.015494 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Terminating (SIGTERM) 18:39:18.058581 run osmo-hlr_10.42.42.2(pid=26688): Terminating (SIGTERM) 18:39:18.102081 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Terminating (SIGTERM) 18:39:18.147511 run osmo-stp_10.42.42.5(pid=26690): Terminating (SIGTERM) 18:39:18.198400 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Terminating (SIGTERM) 18:39:18.246865 run osmo-ggsn_10.42.42.6(pid=26696): Terminating (SIGTERM) 18:39:18.295120 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Terminating (SIGTERM) 18:39:18.348084 run osmo-sgsn_10.42.42.7(pid=26716): Terminating (SIGTERM) 18:39:18.400548 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Terminating (SIGTERM) 18:39:18.471460 run osmo-msc_10.42.42.8(pid=26718): Terminating (SIGTERM) 18:39:18.541263 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Terminating (SIGTERM) 18:39:18.586758 run osmo-mgw_10.42.42.3(pid=26720): Terminating (SIGTERM) 18:39:18.633718 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Terminating (SIGTERM) 18:39:18.678200 run osmo-mgw_10.42.42.4(pid=26722): Terminating (SIGTERM) 18:39:18.721873 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Terminating (SIGTERM) 18:39:18.765687 run osmo-bsc_10.42.42.9(pid=26724): Terminating (SIGTERM) 18:39:18.808811 run osmo-bts-sysmo(pid=26733): Terminating (SIGTERM) 18:39:18.852191 run osmo-pcu(pid=26742): Terminating (SIGTERM) 18:39:18.896577 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGTERM) 18:39:19.041243 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGTERM(pid=26820): Launched 18:39:20.147122 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGTERM(pid=26820): Terminated: ok {rc=0} 18:39:20.213485 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGTERM) 18:39:20.357004 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGTERM(pid=26825): Launched 18:39:21.450010 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGTERM(pid=26825): Terminated: ok {rc=0} 18:39:21.516329 --- ParallelTerminationStrategy: PID 26681 died... 18:39:21.536410 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Terminated: ok {rc=0} 18:39:21.583475 --- ParallelTerminationStrategy: PID 26683 died... 18:39:21.603319 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Terminated: ok {rc=0} 18:39:21.655678 --- ParallelTerminationStrategy: PID 26684 died... 18:39:21.675303 run iperf3-srv_10.42.42.10(pid=26684): Terminated {rc=256} 18:39:21.722589 --- ParallelTerminationStrategy: PID 26687 died... 18:39:21.743029 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Terminated: ok {rc=0} 18:39:21.789911 --- ParallelTerminationStrategy: PID 26688 died... 18:39:21.809718 run osmo-hlr_10.42.42.2(pid=26688): Terminated: ok {rc=0} 18:39:21.856930 --- ParallelTerminationStrategy: PID 26689 died... 18:39:21.876757 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Terminated: ok {rc=0} 18:39:21.923958 --- ParallelTerminationStrategy: PID 26690 died... 18:39:21.944181 run osmo-stp_10.42.42.5(pid=26690): Terminated: ok {rc=0} 18:39:21.990647 --- ParallelTerminationStrategy: PID 26691 died... 18:39:22.010317 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Terminated: ok {rc=0} 18:39:22.056924 --- ParallelTerminationStrategy: PID 26696 died... 18:39:22.077177 run osmo-ggsn_10.42.42.6(pid=26696): Terminated: ok {rc=0} 18:39:22.124009 --- ParallelTerminationStrategy: PID 26715 died... 18:39:22.143710 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Terminated: ok {rc=0} 18:39:22.190268 --- ParallelTerminationStrategy: PID 26716 died... 18:39:22.209909 run osmo-sgsn_10.42.42.7(pid=26716): Terminated: ok {rc=0} 18:39:22.256464 --- ParallelTerminationStrategy: PID 26717 died... 18:39:22.276078 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Terminated: ok {rc=0} 18:39:22.322820 --- ParallelTerminationStrategy: PID 26718 died... 18:39:22.342421 run osmo-msc_10.42.42.8(pid=26718): Terminated: ok {rc=0} 18:39:22.389583 --- ParallelTerminationStrategy: PID 26719 died... 18:39:22.409411 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Terminated: ok {rc=0} 18:39:22.456512 --- ParallelTerminationStrategy: PID 26720 died... 18:39:22.476280 run osmo-mgw_10.42.42.3(pid=26720): Terminated {rc=15} 18:39:22.523405 --- ParallelTerminationStrategy: PID 26721 died... 18:39:22.543318 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Terminated: ok {rc=0} 18:39:22.590518 --- ParallelTerminationStrategy: PID 26722 died... 18:39:22.610815 run osmo-mgw_10.42.42.4(pid=26722): Terminated {rc=15} 18:39:22.657754 --- ParallelTerminationStrategy: PID 26723 died... 18:39:22.677601 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Terminated: ok {rc=0} 18:39:22.724123 --- ParallelTerminationStrategy: PID 26724 died... 18:39:22.744086 run osmo-bsc_10.42.42.9(pid=26724): Terminated: ok {rc=0} 18:39:22.790849 --- ParallelTerminationStrategy: PID 26733 died... 18:39:22.810783 run osmo-bts-sysmo(pid=26733): Terminated: ok {rc=0} 18:39:22.857900 --- ParallelTerminationStrategy: PID 26742 died... 18:39:22.878159 run osmo-pcu(pid=26742): Terminated: ok {rc=0} 18:39:28.747801 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGINT) 18:39:28.895566 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGINT(pid=26828): Launched 18:39:29.981810 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGINT(pid=26828): Terminated: ok {rc=0} 18:39:30.047964 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGINT) 18:39:30.190153 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGINT(pid=26831): Launched 18:39:31.279904 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGINT(pid=26831): Terminated: ok {rc=0} 18:39:37.169022 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGKILL) 18:39:37.317978 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGUSR1(pid=26834): Launched 18:39:38.402893 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGUSR1(pid=26834): Terminated: ok {rc=0} 18:39:38.468708 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGKILL) 18:39:38.613806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGUSR1(pid=26837): Launched 18:39:39.699613 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGUSR1(pid=26837): Terminated: ok {rc=0} 18:39:39.952782 bus /gobi_0: Setting Online False 18:39:41.109901 bus /gobi_0: Setting Powered False 18:39:57.933861 bus /gobi_0: Setting Powered False 18:39:59.031683 bus /gobi_0: Setting Powered False 18:40:00.187625 bus /gobi_6: Setting Online False 18:40:01.324454 bus /gobi_6: Setting Powered False 18:40:09.937679 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2064 gprs:sysmo iperf3.py ---------------------------------------------- 18:40:11.133475 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.144585 tst gprs:sysmo: Using 1 x bts (candidates: 1) 18:40:11.179606 tst iperf3.py: using LAC 3946 18:40:11.296486 tst iperf3.py: using RAC 121 18:40:11.417856 tst iperf3.py: using CellId 3946 18:40:11.534562 tst iperf3.py: using BVCI 3947 18:40:11.652136 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.663475 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.674472 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.685761 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.696994 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.708244 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.719346 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.730187 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:40:11.741327 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:40:11.859782 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003... 18:40:12.076074 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:40:12.401713 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Launched 18:40:12.684914 run iperf3-srv_10.42.42.10(pid=26844): Launched 18:40:12.809071 tst iperf3.py:8: start network... 18:40:13.028324 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:40:13.531050 run create_hlr_db(pid=26845): Launched 18:40:13.655300 bus /gobi_6: Setting Powered False 18:40:14.719347 run create_hlr_db(pid=26845): Terminated: ok {rc=0} 18:40:15.171589 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Launched 18:40:15.495601 run osmo-hlr_10.42.42.2(pid=26848): Launched 18:40:15.620203 run osmo-stp_10.42.42.5: Starting osmo-stp 18:40:16.173360 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Launched 18:40:16.499230 run osmo-stp_10.42.42.5(pid=26850): Launched 18:40:16.622825 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 18:40:17.179654 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Launched 18:40:17.304391 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 18:40:17.580170 run patchelf(pid=26852): Launched 18:40:18.763309 run patchelf(pid=26852): Terminated: ok {rc=0} 18:40:18.900791 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 18:40:19.182582 run osmo-gsm-tester_setcap_net_admin.sh(pid=26853): Launched 18:40:20.368115 run osmo-gsm-tester_setcap_net_admin.sh(pid=26853): Terminated: ok {rc=0} 18:40:20.708483 run osmo-ggsn_10.42.42.6(pid=26856): Launched 18:40:20.833195 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 18:40:21.546002 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Launched 18:40:21.871360 run osmo-sgsn_10.42.42.7(pid=26880): Launched 18:40:21.994607 run osmo-msc_10.42.42.8: Starting osmo-msc 18:40:22.580961 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Launched 18:40:22.905293 run osmo-msc_10.42.42.8(pid=26882): Launched 18:40:26.086848 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:40:26.639561 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Launched 18:40:26.965928 run osmo-mgw_10.42.42.3(pid=26884): Launched 18:40:27.091891 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:40:27.641697 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Launched 18:40:27.970361 run osmo-mgw_10.42.42.4(pid=26886): Launched 18:40:28.095899 run osmo-bsc_10.42.42.9: Starting osmo-bsc 18:40:28.771906 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Launched 18:40:29.102019 run osmo-bsc_10.42.42.9(pid=26888): Launched 18:40:35.392301 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 18:40:35.962825 run rm-remote-dir(pid=26889): Launched 18:40:38.159596 run rm-remote-dir(pid=26889): Terminated: ok {rc=0} 18:40:38.519994 run mk-remote-dir(pid=26890): Launched 18:40:39.711928 run mk-remote-dir(pid=26890): Terminated: ok {rc=0} 18:40:40.019352 run scp-inst-to-remote(pid=26891): Launched 18:40:52.225207 run scp-inst-to-remote(pid=26891): Terminated: ok {rc=0} 18:40:52.557665 run mk-remote-dir(pid=26893): Launched 18:40:53.735584 run mk-remote-dir(pid=26893): Terminated: ok {rc=0} 18:40:54.021235 run scp-cfg-to-remote(pid=26894): Launched 18:40:55.199682 run scp-cfg-to-remote(pid=26894): Terminated: ok {rc=0} 18:40:55.528849 run reload-dsp-firmware(pid=26896): Launched 18:40:57.706484 run reload-dsp-firmware(pid=26896): Terminated: ok {rc=0} 18:40:58.028650 run osmo-bts-sysmo(pid=26897): Launched 18:41:02.260450 tst iperf3.py:8: Waiting for bts to be ready... 18:41:03.982569 run rm-remote-dir(pid=26898): Launched 18:41:06.164315 run rm-remote-dir(pid=26898): Terminated: ok {rc=0} 18:41:06.487881 run mk-remote-dir(pid=26899): Launched 18:41:08.658824 run mk-remote-dir(pid=26899): Terminated: ok {rc=0} 18:41:08.936153 run scp-inst-to-bts(pid=26900): Launched 18:41:22.123036 run scp-inst-to-bts(pid=26900): Terminated: ok {rc=0} 18:41:22.446638 run mk-remote-run-dir(pid=26902): Launched 18:41:24.623632 run mk-remote-run-dir(pid=26902): Terminated: ok {rc=0} 18:41:24.920198 run scp-cfg-to-sysmobts(pid=26903): Launched 18:41:27.107023 run scp-cfg-to-sysmobts(pid=26903): Terminated: ok {rc=0} 18:41:27.452717 run osmo-pcu(pid=26905): Launched 18:41:27.588774 tst iperf3.py: using MSISDN 3951 18:41:27.764385 bus /gobi_6: Setting Powered True 18:41:31.283264 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3951', subscriber_id=1} 18:41:31.558641 bus /gobi_6: Setting Online False 18:41:32.751208 bus /gobi_6: Setting Powered False 18:41:40.133664 bus /gobi_6: Setting Powered True 18:41:41.535326 bus /gobi_6: Setting Online True 18:41:44.084922 tst /gobi_6: Connect to ('901', '70') 18:41:44.346316 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 18:41:44.455164 tst iperf3.py:8: waiting for modems to attach... 18:41:49.961316 tst /gobi_6: Already registered with network ('901', '70') 18:42:01.906359 tst iperf3.py:8: waiting for modems to attach to data services... 18:42:06.570252 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:42:06.688943 tst iperf3.py:8: Setting up data plan for "'/gobi_6/context3'" 18:42:07.139193 run move_netns(pid=26909): Launched 18:42:08.333756 run move_netns(pid=26909): Terminated: ok {rc=0} 18:42:08.696771 run ifup(pid=26912): Launched 18:42:09.889153 run ifup(pid=26912): Terminated: ok {rc=0} 18:42:10.245155 run dhcp(pid=26915): Launched 18:42:11.427619 run dhcp(pid=26915): Terminated: ok {rc=0} 18:42:11.556435 tst iperf3.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context3'" 18:42:11.766122 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 18:42:12.130786 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Launched 18:42:12.455587 run iperf3-cli_10.42.42.10(pid=26934): Launched 18:43:26.776963 run iperf3-cli_10.42.42.10(pid=26934): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=26934)] 18:43:26.915744 run iperf3-cli_10.42.42.10(pid=26934): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 18:43:27.057591 run iperf3-cli_10.42.42.10(pid=26934): stderr: | (launched: 2022-03-29_18:42:12.439797) 18:43:27.196695 run iperf3-srv_10.42.42.10(pid=26844): Terminating (SIGINT) 18:43:27.541012 run iperf3-srv_10.42.42.10(pid=26844): Terminated {rc=1} 18:43:27.667529 tst iperf3.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context3'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42866, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:42:19 GMT', 'timesecs': 1648572139}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'vfzcc6k7krnaudu2qhmwk43mkmmn3xd5enfe', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0008230209350586, 'seconds': 1.0008230209350586, 'bytes': 101360, 'bits_per_second': 810213.1775929807, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 416760, 'rttvar': 179188, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0008230209350586, 'seconds': 1.0008230209350586, 'bytes': 101360, 'bits_per_second': 810213.1775929807, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0008230209350586, 'end': 2.001343011856079, 'seconds': 1.0005199909210205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 504319, 'rttvar': 309509, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0008230209350586, 'end': 2.001343011856079, 'seconds': 1.0005199909210205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.001343011856079, 'end': 3.0013041496276855, 'seconds': 0.9999611377716064, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 18824, 'rtt': 887851, 'rttvar': 861683, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.001343011856079, 'end': 3.0013041496276855, 'seconds': 0.9999611377716064, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0013041496276855, 'end': 4.0011889934539795, 'seconds': 0.999884843826294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1169081, 'rttvar': 1208723, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0013041496276855, 'end': 4.0011889934539795, 'seconds': 0.999884843826294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.0011889934539795, 'end': 5.001459121704102, 'seconds': 1.000270128250122, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 1502871, 'rttvar': 1574122, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.0011889934539795, 'end': 5.001459121704102, 'seconds': 1.000270128250122, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001459121704102, 'end': 6.001329183578491, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 2303976, 'rttvar': 2296626, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001459121704102, 'end': 6.001329183578491, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001329183578491, 'end': 7.001317024230957, 'seconds': 0.9999878406524658, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2758356, 'rttvar': 2631231, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001329183578491, 'end': 7.001317024230957, 'seconds': 0.9999878406524658, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001317024230957, 'end': 8.001265048980713, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 3754779, 'rttvar': 3230560, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001317024230957, 'end': 8.001265048980713, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001265048980713, 'end': 9.001326084136963, 'seconds': 1.00006103515625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 4181887, 'rttvar': 3277135, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001265048980713, 'end': 9.001326084136963, 'seconds': 1.00006103515625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001326084136963, 'end': 10.001069068908691, 'seconds': 0.9997429847717285, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 4614394, 'rttvar': 3322867, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001326084136963, 'end': 10.001069068908691, 'seconds': 0.9997429847717285, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001069068908691, 'end': 11.001177072525024, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 5090389, 'rttvar': 3444140, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001069068908691, 'end': 11.001177072525024, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001177072525024, 'end': 12.001214981079102, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 5594052, 'rttvar': 3590431, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001177072525024, 'end': 12.001214981079102, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001214981079102, 'end': 13.001265048980713, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 6514479, 'rttvar': 3639030, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001214981079102, 'end': 13.001265048980713, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001265048980713, 'end': 14.001188039779663, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 6935277, 'rttvar': 3570868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001265048980713, 'end': 14.001188039779663, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001188039779663, 'end': 15.001317024230957, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 7784605, 'rttvar': 3480777, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001188039779663, 'end': 15.001317024230957, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001317024230957, 'end': 16.001317977905273, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 8221413, 'rttvar': 3484198, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001317024230957, 'end': 16.001317977905273, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001317977905273, 'end': 17.00117516517639, 'seconds': 0.9998571872711182, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9025140, 'rttvar': 3376202, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001317977905273, 'end': 17.00117516517639, 'seconds': 0.9998571872711182, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.00117516517639, 'end': 18.00126600265503, 'seconds': 1.0000908374786377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9394544, 'rttvar': 3270958, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.00117516517639, 'end': 18.00126600265503, 'seconds': 1.0000908374786377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.00126600265503, 'end': 19.00116515159607, 'seconds': 0.99989914894104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 9802550, 'rttvar': 3269232, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.00126600265503, 'end': 19.00116515159607, 'seconds': 0.99989914894104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.00116515159607, 'end': 20.001224040985107, 'seconds': 1.000058889389038, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 10159568, 'rttvar': 3165960, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.00116515159607, 'end': 20.001224040985107, 'seconds': 1.000058889389038, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001224040985107, 'end': 21.001187086105347, 'seconds': 0.9999630451202393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 10992522, 'rttvar': 3229428, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001224040985107, 'end': 21.001187086105347, 'seconds': 0.9999630451202393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001187086105347, 'end': 22.00130820274353, 'seconds': 1.0001211166381836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 11438503, 'rttvar': 3314034, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001187086105347, 'end': 22.00130820274353, 'seconds': 1.0001211166381836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.00130820274353, 'end': 23.001317024230957, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11833439, 'rttvar': 3275397, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.00130820274353, 'end': 23.001317024230957, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001317024230957, 'end': 24.001163005828857, 'seconds': 0.9998459815979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 12276390, 'rttvar': 3342449, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001317024230957, 'end': 24.001163005828857, 'seconds': 0.9998459815979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001163005828857, 'end': 25.001341104507446, 'seconds': 1.0001780986785889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 13150739, 'rttvar': 3424898, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001163005828857, 'end': 25.001341104507446, 'seconds': 1.0001780986785889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001341104507446, 'end': 26.001182079315186, 'seconds': 0.9998409748077393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 13558774, 'rttvar': 3384745, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001341104507446, 'end': 26.001182079315186, 'seconds': 0.9998409748077393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.001182079315186, 'end': 27.000904083251953, 'seconds': 0.9997220039367676, 'bytes': 256296, 'bits_per_second': 2050938.1527323928, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 14016223, 'rttvar': 3453455, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.001182079315186, 'end': 27.000904083251953, 'seconds': 0.9997220039367676, 'bytes': 256296, 'bits_per_second': 2050938.1527323928, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.000904083251953, 'end': 28.001161098480225, 'seconds': 1.0002570152282715, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 14676016, 'rttvar': 3109571, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.000904083251953, 'end': 28.001161098480225, 'seconds': 1.0002570152282715, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001161098480225, 'end': 29.00123620033264, 'seconds': 1.000075101852417, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14991541, 'rttvar': 2963228, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001161098480225, 'end': 29.00123620033264, 'seconds': 1.000075101852417, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.00123620033264, 'end': 30.000699996948242, 'seconds': 0.9994637966156006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 15673002, 'rttvar': 2847857, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.00123620033264, 'end': 30.000699996948242, 'seconds': 0.9994637966156006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.000699996948242, 'end': 31.00126814842224, 'seconds': 1.000568151473999, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 16031214, 'rttvar': 2852316, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.000699996948242, 'end': 31.00126814842224, 'seconds': 1.000568151473999, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00126814842224, 'end': 32.00126504898071, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 16699289, 'rttvar': 2785032, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00126814842224, 'end': 32.00126504898071, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00126504898071, 'end': 33.00132417678833, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 17019166, 'rttvar': 2728527, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00126504898071, 'end': 33.00132417678833, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00132417678833, 'end': 34.00120401382446, 'seconds': 0.9998798370361328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17650620, 'rttvar': 2615336, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00132417678833, 'end': 34.00120401382446, 'seconds': 0.9998798370361328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00120401382446, 'end': 35.00131702423096, 'seconds': 1.0001130104064941, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17964070, 'rttvar': 2588400, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00120401382446, 'end': 35.00131702423096, 'seconds': 1.0001130104064941, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00131702423096, 'end': 36.00126600265503, 'seconds': 0.9999489784240723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 18568273, 'rttvar': 2527256, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00131702423096, 'end': 36.00126600265503, 'seconds': 0.9999489784240723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00126600265503, 'end': 37.00130605697632, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18854070, 'rttvar': 2467036, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00126600265503, 'end': 37.00130605697632, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00130605697632, 'end': 38.00123310089111, 'seconds': 0.9999270439147949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19482133, 'rttvar': 2474958, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00130605697632, 'end': 38.00123310089111, 'seconds': 0.9999270439147949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00123310089111, 'end': 39.00126600265503, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19823876, 'rttvar': 2539705, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00123310089111, 'end': 39.00126600265503, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.00126600265503, 'end': 40.00121712684631, 'seconds': 0.9999511241912842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 20123559, 'rttvar': 2504145, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.00126600265503, 'end': 40.00121712684631, 'seconds': 0.9999511241912842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00121712684631, 'end': 41.00133299827576, 'seconds': 1.0001158714294434, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20649454, 'rttvar': 2320905, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00121712684631, 'end': 41.00133299827576, 'seconds': 1.0001158714294434, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00133299827576, 'end': 42.001216173172, 'seconds': 0.9998831748962402, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20953255, 'rttvar': 2348280, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00133299827576, 'end': 42.001216173172, 'seconds': 0.9998831748962402, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.001216173172, 'end': 43.00133299827576, 'seconds': 1.0001168251037598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 70952, 'rtt': 21538280, 'rttvar': 2358314, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.001216173172, 'end': 43.00133299827576, 'seconds': 1.0001168251037598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00133299827576, 'end': 44.00127410888672, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21783444, 'rttvar': 2259065, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00133299827576, 'end': 44.00127410888672, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00127410888672, 'end': 45.00116300582886, 'seconds': 0.9998888969421387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 22239630, 'rttvar': 2084817, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00127410888672, 'end': 45.00116300582886, 'seconds': 0.9998888969421387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00116300582886, 'end': 46.001182079315186, 'seconds': 1.0000190734863281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 22459443, 'rttvar': 2003239, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00116300582886, 'end': 46.001182079315186, 'seconds': 1.0000190734863281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.001182079315186, 'end': 47.0004141330719, 'seconds': 0.9992320537567139, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22644141, 'rttvar': 1871824, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.001182079315186, 'end': 47.0004141330719, 'seconds': 0.9992320537567139, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.0004141330719, 'end': 48.00126600265503, 'seconds': 1.0008518695831299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 23099025, 'rttvar': 1838000, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.0004141330719, 'end': 48.00126600265503, 'seconds': 1.0008518695831299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.00126600265503, 'end': 49.00107002258301, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 23258668, 'rttvar': 1697785, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.00126600265503, 'end': 49.00107002258301, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.00107002258301, 'end': 50.00130915641785, 'seconds': 1.0002391338348389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 78192, 'rtt': 23660133, 'rttvar': 1644268, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.00107002258301, 'end': 50.00130915641785, 'seconds': 1.0002391338348389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00130915641785, 'end': 51.001320123672485, 'seconds': 1.0000109672546387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23810270, 'rttvar': 1533477, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00130915641785, 'end': 51.001320123672485, 'seconds': 1.0000109672546387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.001320123672485, 'end': 52.00126504898071, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24213998, 'rttvar': 1558192, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.001320123672485, 'end': 52.00126504898071, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00126504898071, 'end': 53.00135803222656, 'seconds': 1.0000929832458496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24374374, 'rttvar': 1489395, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00126504898071, 'end': 53.00135803222656, 'seconds': 1.0000929832458496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00135803222656, 'end': 54.00108003616333, 'seconds': 0.9997220039367676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24799241, 'rttvar': 1573468, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00135803222656, 'end': 54.00108003616333, 'seconds': 0.9997220039367676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00108003616333, 'end': 55.0010781288147, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 24971236, 'rttvar': 1524092, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00108003616333, 'end': 55.0010781288147, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.0010781288147, 'end': 56.0014591217041, 'seconds': 1.0003809928894043, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 25207160, 'rttvar': 1614915, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.0010781288147, 'end': 56.0014591217041, 'seconds': 1.0003809928894043, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.0014591217041, 'end': 57.00111699104309, 'seconds': 0.9996578693389893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 85432, 'rtt': 25593805, 'rttvar': 1578728, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.0014591217041, 'end': 57.00111699104309, 'seconds': 0.9996578693389893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.00111699104309, 'end': 58.001320123672485, 'seconds': 1.0002031326293945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25836838, 'rttvar': 1670112, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.00111699104309, 'end': 58.001320123672485, 'seconds': 1.0002031326293945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001320123672485, 'end': 59.001305103302, 'seconds': 0.9999849796295166, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 88328, 'rtt': 26251788, 'rttvar': 1659292, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001320123672485, 'end': 59.001305103302, 'seconds': 0.9999849796295166, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 18:43:27.909347 tst iperf3.py:8: ERR: KeyError: 'sum_sent' [trial-2064↪gprs:sysmo↪iperf3.py:8] 18:43:27.917072 tst iperf3.py:8: Test FAILED (196.8 sec) 18:43:27.925109 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Terminating (SIGTERM) 18:43:27.968726 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Terminating (SIGTERM) 18:43:28.012435 run osmo-hlr_10.42.42.2(pid=26848): Terminating (SIGTERM) 18:43:28.055458 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Terminating (SIGTERM) 18:43:28.102051 run osmo-stp_10.42.42.5(pid=26850): Terminating (SIGTERM) 18:43:28.150418 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Terminating (SIGTERM) 18:43:28.214286 run osmo-ggsn_10.42.42.6(pid=26856): Terminating (SIGTERM) 18:43:28.262987 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Terminating (SIGTERM) 18:43:28.347167 run osmo-sgsn_10.42.42.7(pid=26880): Terminating (SIGTERM) 18:43:28.394825 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Terminating (SIGTERM) 18:43:28.448858 run osmo-msc_10.42.42.8(pid=26882): Terminating (SIGTERM) 18:43:28.496223 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Terminating (SIGTERM) 18:43:28.541767 run osmo-mgw_10.42.42.3(pid=26884): Terminating (SIGTERM) 18:43:28.588810 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Terminating (SIGTERM) 18:43:28.632853 run osmo-mgw_10.42.42.4(pid=26886): Terminating (SIGTERM) 18:43:28.676571 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Terminating (SIGTERM) 18:43:28.720875 run osmo-bsc_10.42.42.9(pid=26888): Terminating (SIGTERM) 18:43:28.763739 run osmo-bts-sysmo(pid=26897): Terminating (SIGTERM) 18:43:28.808091 run osmo-pcu(pid=26905): Terminating (SIGTERM) 18:43:28.852311 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGTERM) 18:43:28.996987 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGTERM(pid=26955): Launched 18:43:30.090195 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGTERM(pid=26955): Terminated: ok {rc=0} 18:43:30.156370 --- ParallelTerminationStrategy: PID 26843 died... 18:43:30.176371 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Terminated: ok {rc=0} 18:43:30.222871 --- ParallelTerminationStrategy: PID 26847 died... 18:43:30.242907 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Terminated: ok {rc=0} 18:43:30.289766 --- ParallelTerminationStrategy: PID 26848 died... 18:43:30.309613 run osmo-hlr_10.42.42.2(pid=26848): Terminated: ok {rc=0} 18:43:30.356237 --- ParallelTerminationStrategy: PID 26849 died... 18:43:30.375971 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Terminated: ok {rc=0} 18:43:30.422578 --- ParallelTerminationStrategy: PID 26850 died... 18:43:30.442912 run osmo-stp_10.42.42.5(pid=26850): Terminated: ok {rc=0} 18:43:30.489396 --- ParallelTerminationStrategy: PID 26851 died... 18:43:30.509185 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Terminated: ok {rc=0} 18:43:30.555822 --- ParallelTerminationStrategy: PID 26856 died... 18:43:30.575617 run osmo-ggsn_10.42.42.6(pid=26856): Terminated: ok {rc=0} 18:43:30.622206 --- ParallelTerminationStrategy: PID 26879 died... 18:43:30.642338 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Terminated: ok {rc=0} 18:43:30.688871 --- ParallelTerminationStrategy: PID 26880 died... 18:43:30.708755 run osmo-sgsn_10.42.42.7(pid=26880): Terminated: ok {rc=0} 18:43:30.755517 --- ParallelTerminationStrategy: PID 26881 died... 18:43:30.775344 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Terminated: ok {rc=0} 18:43:30.821866 --- ParallelTerminationStrategy: PID 26882 died... 18:43:30.841699 run osmo-msc_10.42.42.8(pid=26882): Terminated: ok {rc=0} 18:43:30.888599 --- ParallelTerminationStrategy: PID 26883 died... 18:43:30.908575 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Terminated: ok {rc=0} 18:43:30.955162 --- ParallelTerminationStrategy: PID 26884 died... 18:43:30.975000 run osmo-mgw_10.42.42.3(pid=26884): Terminated {rc=15} 18:43:31.021719 --- ParallelTerminationStrategy: PID 26885 died... 18:43:31.041588 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Terminated: ok {rc=0} 18:43:31.088246 --- ParallelTerminationStrategy: PID 26886 died... 18:43:31.108220 run osmo-mgw_10.42.42.4(pid=26886): Terminated {rc=15} 18:43:31.154903 --- ParallelTerminationStrategy: PID 26887 died... 18:43:31.174860 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Terminated: ok {rc=0} 18:43:31.221814 --- ParallelTerminationStrategy: PID 26888 died... 18:43:31.241582 run osmo-bsc_10.42.42.9(pid=26888): Terminated: ok {rc=0} 18:43:31.288191 --- ParallelTerminationStrategy: PID 26897 died... 18:43:31.308225 run osmo-bts-sysmo(pid=26897): Terminated: ok {rc=0} 18:43:31.355063 --- ParallelTerminationStrategy: PID 26905 died... 18:43:31.374911 run osmo-pcu(pid=26905): Terminated: ok {rc=0} 18:43:37.243809 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGINT) 18:43:37.382029 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGINT(pid=26960): Launched 18:43:38.472790 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGINT(pid=26960): Terminated: ok {rc=0} 18:43:44.358862 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGKILL) 18:43:44.502121 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGUSR1(pid=26963): Launched 18:43:45.589473 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGUSR1(pid=26963): Terminated: ok {rc=0} 18:43:45.841026 bus /gobi_6: Setting Online False 18:43:48.873285 bus /gobi_6: Setting Powered False 18:44:03.937866 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2064 gprs:sysmo iperf3m4.py ---------------------------------------------- 18:44:05.136907 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.148186 tst gprs:sysmo: Using 1 x bts (candidates: 1) 18:44:05.183216 tst iperf3m4.py: using LAC 3947 18:44:05.311378 tst iperf3m4.py: using RAC 122 18:44:05.438236 tst iperf3m4.py: using CellId 3947 18:44:05.565537 tst iperf3m4.py: using BVCI 3948 18:44:05.685491 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.696587 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.707528 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.718354 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.729132 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.739899 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.750781 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.761644 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:44:05.772694 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:44:05.889967 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:44:06.007375 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:44:06.126900 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:44:06.243958 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003... 18:44:06.449284 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:44:06.759360 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Launched 18:44:07.028978 run iperf3-srv_10.42.42.10(pid=26967): Launched 18:44:07.145656 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004... 18:44:07.352490 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:44:07.670076 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Launched 18:44:07.957811 run iperf3-srv_10.42.42.10(pid=26971): Launched 18:44:08.080823 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005... 18:44:08.287392 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:44:08.595727 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Launched 18:44:08.864073 run iperf3-srv_10.42.42.10(pid=26973): Launched 18:44:08.977215 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006... 18:44:09.180898 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:44:09.489782 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Launched 18:44:09.759377 run iperf3-srv_10.42.42.10(pid=26975): Launched 18:44:09.872449 tst iperf3m4.py:8: start network... 18:44:10.075753 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:44:10.547693 run create_hlr_db(pid=26976): Launched 18:44:10.665746 bus /gobi_6: Setting Powered False 18:44:11.725965 run create_hlr_db(pid=26976): Terminated: ok {rc=0} 18:44:12.173453 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Launched 18:44:12.501801 run osmo-hlr_10.42.42.2(pid=26979): Launched 18:44:12.626842 run osmo-stp_10.42.42.5: Starting osmo-stp 18:44:13.188903 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Launched 18:44:13.515079 run osmo-stp_10.42.42.5(pid=26981): Launched 18:44:13.639823 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 18:44:14.200195 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Launched 18:44:14.323817 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 18:44:14.600687 run patchelf(pid=26983): Launched 18:44:15.785745 run patchelf(pid=26983): Terminated: ok {rc=0} 18:44:15.914425 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 18:44:16.175801 run osmo-gsm-tester_setcap_net_admin.sh(pid=26984): Launched 18:44:17.363964 run osmo-gsm-tester_setcap_net_admin.sh(pid=26984): Terminated: ok {rc=0} 18:44:17.707289 run osmo-ggsn_10.42.42.6(pid=26987): Launched 18:44:17.830611 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 18:44:18.532486 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Launched 18:44:18.859651 run osmo-sgsn_10.42.42.7(pid=27009): Launched 18:44:18.984720 run osmo-msc_10.42.42.8: Starting osmo-msc 18:44:19.579104 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Launched 18:44:19.908672 run osmo-msc_10.42.42.8(pid=27011): Launched 18:44:23.091202 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:44:23.664655 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Launched 18:44:23.993712 run osmo-mgw_10.42.42.3(pid=27013): Launched 18:44:24.119230 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:44:24.668716 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Launched 18:44:24.996797 run osmo-mgw_10.42.42.4(pid=27015): Launched 18:44:25.121290 run osmo-bsc_10.42.42.9: Starting osmo-bsc 18:44:25.796587 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Launched 18:44:26.125430 run osmo-bsc_10.42.42.9(pid=27017): Launched 18:44:32.415553 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 18:44:32.987097 run rm-remote-dir(pid=27018): Launched 18:44:35.185020 run rm-remote-dir(pid=27018): Terminated: ok {rc=0} 18:44:35.546097 run mk-remote-dir(pid=27019): Launched 18:44:36.737476 run mk-remote-dir(pid=27019): Terminated: ok {rc=0} 18:44:37.045347 run scp-inst-to-remote(pid=27020): Launched 18:44:49.256754 run scp-inst-to-remote(pid=27020): Terminated: ok {rc=0} 18:44:49.613206 run mk-remote-dir(pid=27022): Launched 18:44:50.802045 run mk-remote-dir(pid=27022): Terminated: ok {rc=0} 18:44:51.108424 run scp-cfg-to-remote(pid=27023): Launched 18:44:52.298931 run scp-cfg-to-remote(pid=27023): Terminated: ok {rc=0} 18:44:52.652745 run reload-dsp-firmware(pid=27025): Launched 18:44:54.843227 run reload-dsp-firmware(pid=27025): Terminated: ok {rc=0} 18:44:55.189647 run osmo-bts-sysmo(pid=27026): Launched 18:44:59.429304 tst iperf3m4.py:8: Waiting for bts to be ready... 18:45:01.154806 run rm-remote-dir(pid=27027): Launched 18:45:03.341684 run rm-remote-dir(pid=27027): Terminated: ok {rc=0} 18:45:03.691280 run mk-remote-dir(pid=27031): Launched 18:45:05.880125 run mk-remote-dir(pid=27031): Terminated: ok {rc=0} 18:45:06.177538 run scp-inst-to-bts(pid=27032): Launched 18:45:19.377424 run scp-inst-to-bts(pid=27032): Terminated: ok {rc=0} 18:45:19.724486 run mk-remote-run-dir(pid=27034): Launched 18:45:21.912452 run mk-remote-run-dir(pid=27034): Terminated: ok {rc=0} 18:45:22.209247 run scp-cfg-to-sysmobts(pid=27035): Launched 18:45:24.394149 run scp-cfg-to-sysmobts(pid=27035): Terminated: ok {rc=0} 18:45:24.740605 run osmo-pcu(pid=27037): Launched 18:45:24.874454 tst iperf3m4.py: using MSISDN 3952 18:45:25.045636 bus /gobi_6: Setting Powered True 18:45:28.564255 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3952', subscriber_id=1} 18:45:28.837192 bus /gobi_6: Setting Online False 18:45:30.025793 bus /gobi_6: Setting Powered False 18:45:37.109329 bus /gobi_6: Setting Powered True 18:45:38.498647 bus /gobi_6: Setting Online True 18:45:40.959901 tst /gobi_6: Connect to ('901', '70') 18:45:41.236070 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 18:45:41.354649 tst iperf3m4.py: using MSISDN 3953 18:45:41.629393 bus /gobi_0: Setting Powered True 18:45:45.141049 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3953', subscriber_id=2} 18:45:45.417127 bus /gobi_0: Setting Online False 18:45:46.608611 bus /gobi_0: Setting Powered False 18:45:54.109834 bus /gobi_0: Setting Powered True 18:45:55.375490 tst /gobi_6: Already registered with network ('901', '70') 18:45:55.596071 bus /gobi_0: Setting Online True 18:45:58.080877 tst /gobi_0: Connect to ('901', '70') 18:45:58.345558 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873' 18:45:58.470839 tst iperf3m4.py: using MSISDN 3954 18:45:59.030117 bus /gobi_1: Setting Powered True 18:46:02.522178 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3954', subscriber_id=3} 18:46:02.793149 bus /gobi_1: Setting Online False 18:46:03.987042 bus /gobi_1: Setting Powered False 18:46:11.135406 bus /gobi_1: Setting Powered True 18:46:12.516607 bus /gobi_1: Setting Online True 18:46:13.430167 tst /gobi_0: Already registered with network ('901', '70') 18:46:15.240244 tst /gobi_1: Connect to ('901', '70') 18:46:15.517543 tst /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024272103' 18:46:15.643109 tst iperf3m4.py: using MSISDN 3955 18:46:15.820467 bus /gobi_4: Setting Powered True 18:46:19.346144 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3955', subscriber_id=4} 18:46:19.608722 bus /gobi_4: Setting Online False 18:46:20.796686 bus /gobi_4: Setting Powered False 18:46:28.115681 bus /gobi_4: Setting Powered True 18:46:29.508779 bus /gobi_4: Setting Online True 18:46:30.931992 tst /gobi_1: Already registered with network ('901', '70') 18:46:32.176918 tst /gobi_4: Connect to ('901', '70') 18:46:32.464749 tst /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G 1 [Jul 21 2016 10:00:50]', Serial: '868323025161511' 18:46:32.573732 tst iperf3m4.py:8: waiting for modems to attach... 18:46:48.650917 tst /gobi_4: Already registered with network ('901', '70') 18:46:54.201032 tst iperf3m4.py:8: waiting for modems to attach to data services... 18:47:02.637656 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:47:02.756476 tst iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'" 18:47:03.210169 run move_netns(pid=27042): Launched 18:47:04.401302 run move_netns(pid=27042): Terminated: ok {rc=0} 18:47:04.757128 run ifup(pid=27045): Launched 18:47:05.946933 run ifup(pid=27045): Terminated: ok {rc=0} 18:47:06.300682 run dhcp(pid=27048): Launched 18:47:07.487692 run dhcp(pid=27048): Terminated: ok {rc=0} 18:47:15.821796 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:47:15.932128 tst iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'" 18:47:16.350000 run move_netns(pid=27066): Launched 18:47:17.536607 run move_netns(pid=27066): Terminated: ok {rc=0} 18:47:17.867113 run ifup(pid=27069): Launched 18:47:19.051424 run ifup(pid=27069): Terminated: ok {rc=0} 18:47:19.380586 run dhcp(pid=27072): Launched 18:47:20.570818 run dhcp(pid=27072): Terminated: ok {rc=0} 18:47:23.596883 tst /gobi_1: context activated {apn='inet46', path='/gobi_1/context9', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'} 18:47:23.706843 tst iperf3m4.py:8: Setting up data plan for "'/gobi_1/context9'" 18:47:24.123762 run move_netns(pid=27090): Launched 18:47:25.310363 run move_netns(pid=27090): Terminated: ok {rc=0} 18:47:25.641387 run ifup(pid=27093): Launched 18:47:26.832699 run ifup(pid=27093): Terminated: ok {rc=0} 18:47:27.187196 run dhcp(pid=27096): Launched 18:47:28.378114 run dhcp(pid=27096): Terminated: ok {rc=0} 18:47:36.783631 tst /gobi_4: context activated {apn='inet46', path='/gobi_4/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:47:36.894066 tst iperf3m4.py:8: Setting up data plan for "'/gobi_4/context2'" 18:47:37.311648 run move_netns(pid=27114): Launched 18:47:38.506943 run move_netns(pid=27114): Terminated: ok {rc=0} 18:47:38.862571 run ifup(pid=27117): Launched 18:47:40.054669 run ifup(pid=27117): Terminated: ok {rc=0} 18:47:40.408583 run dhcp(pid=27120): Launched 18:47:41.597956 run dhcp(pid=27120): Terminated: ok {rc=0} 18:47:41.737207 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'" 18:47:41.964359 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 18:47:42.355348 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Launched 18:47:42.538390 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'" 18:47:42.765153 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s) 18:47:43.157076 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Launched 18:47:43.338744 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context9'" 18:47:43.563135 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s) 18:47:43.952340 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Launched 18:47:44.132025 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context2'" 18:47:44.356819 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s) 18:47:44.745979 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Launched 18:47:45.105657 run iperf3-cli_10.42.42.10(pid=27148): Launched 18:47:45.382355 run iperf3-cli_10.42.42.10(pid=27151): Launched 18:47:45.659249 run iperf3-cli_10.42.42.10(pid=27154): Launched 18:47:45.929122 run iperf3-cli_10.42.42.10(pid=27157): Launched 18:48:54.251448 run iperf3-cli_10.42.42.10(pid=27148): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27148)] 18:48:54.518913 run iperf3-cli_10.42.42.10(pid=27148): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 18:48:54.656399 run iperf3-cli_10.42.42.10(pid=27148): stderr: | (launched: 2022-03-29_18:47:45.089716) 18:48:55.843780 run iperf3-cli_10.42.42.10(pid=27151): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27151)] 18:48:55.980978 run iperf3-cli_10.42.42.10(pid=27151): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 18:48:56.124272 run iperf3-cli_10.42.42.10(pid=27151): stderr: | (launched: 2022-03-29_18:47:45.365519) 18:48:57.329256 run iperf3-cli_10.42.42.10(pid=27154): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27154)] 18:48:57.471555 run iperf3-cli_10.42.42.10(pid=27154): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 18:48:57.614442 run iperf3-cli_10.42.42.10(pid=27154): stderr: | (launched: 2022-03-29_18:47:45.642079) 18:48:58.810636 run iperf3-cli_10.42.42.10(pid=27157): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27157)] 18:48:58.939380 run iperf3-cli_10.42.42.10(pid=27157): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 18:48:59.067024 run iperf3-cli_10.42.42.10(pid=27157): stderr: | (launched: 2022-03-29_18:47:45.912502) 18:48:59.194618 run iperf3-srv_10.42.42.10(pid=26967): Terminating (SIGINT) 18:48:59.525727 run iperf3-srv_10.42.42.10(pid=26967): Terminated {rc=1} 18:48:59.640844 tst iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42908, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:47:46 GMT', 'timesecs': 1648572466}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'bbwlr3tnbptbxm2dq6maiza55z7ufvcy5did', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0006639957427979, 'seconds': 1.0006639957427979, 'bytes': 101360, 'bits_per_second': 810341.9364040171, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 379730, 'rttvar': 175208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0006639957427979, 'seconds': 1.0006639957427979, 'bytes': 101360, 'bits_per_second': 810341.9364040171, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0006639957427979, 'end': 2.0005221366882324, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 521979, 'rttvar': 415904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0006639957427979, 'end': 2.0005221366882324, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0005221366882324, 'end': 3.001317024230957, 'seconds': 1.0007948875427246, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 4344, 'rtt': 521979, 'rttvar': 415904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0005221366882324, 'end': 3.001317024230957, 'seconds': 1.0007948875427246, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.001317024230957, 'end': 4.001322984695435, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 809107, 'rttvar': 886184, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.001317024230957, 'end': 4.001322984695435, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001322984695435, 'end': 5.001317977905273, 'seconds': 0.9999949932098389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1250143, 'rttvar': 1546710, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001322984695435, 'end': 5.001317977905273, 'seconds': 0.9999949932098389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001317977905273, 'end': 6.000808000564575, 'seconds': 0.9994900226593018, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1250143, 'rttvar': 1546710, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001317977905273, 'end': 6.000808000564575, 'seconds': 0.9994900226593018, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.000808000564575, 'end': 7.001322984695435, 'seconds': 1.0005149841308594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1828734, 'rttvar': 2317215, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.000808000564575, 'end': 7.001322984695435, 'seconds': 1.0005149841308594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001322984695435, 'end': 8.000629186630249, 'seconds': 0.9993062019348145, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2527605, 'rttvar': 3135652, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001322984695435, 'end': 8.000629186630249, 'seconds': 0.9993062019348145, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.000629186630249, 'end': 9.00132417678833, 'seconds': 1.000694990158081, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2527605, 'rttvar': 3135652, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.000629186630249, 'end': 9.00132417678833, 'seconds': 1.000694990158081, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.00132417678833, 'end': 10.001171112060547, 'seconds': 0.9998469352722168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 3326609, 'rttvar': 3949748, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.00132417678833, 'end': 10.001171112060547, 'seconds': 0.9998469352722168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001171112060547, 'end': 11.000813007354736, 'seconds': 0.9996418952941895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4210285, 'rttvar': 4729664, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001171112060547, 'end': 11.000813007354736, 'seconds': 0.9996418952941895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.000813007354736, 'end': 12.001137971878052, 'seconds': 1.0003249645233154, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4210285, 'rttvar': 4729664, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.000813007354736, 'end': 12.001137971878052, 'seconds': 1.0003249645233154, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001137971878052, 'end': 13.001238107681274, 'seconds': 1.0001001358032227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 5170983, 'rttvar': 5468642, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001137971878052, 'end': 13.001238107681274, 'seconds': 1.0001001358032227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001238107681274, 'end': 14.000625133514404, 'seconds': 0.9993870258331299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6209445, 'rttvar': 6178407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001238107681274, 'end': 14.000625133514404, 'seconds': 0.9993870258331299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.000625133514404, 'end': 15.00132417678833, 'seconds': 1.0006990432739258, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6209445, 'rttvar': 6178407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.000625133514404, 'end': 15.00132417678833, 'seconds': 1.0006990432739258, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.00132417678833, 'end': 16.001312971115112, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7307874, 'rttvar': 6830663, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.00132417678833, 'end': 16.001312971115112, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001312971115112, 'end': 17.001317024230957, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7307874, 'rttvar': 6830663, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001312971115112, 'end': 17.001317024230957, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001317024230957, 'end': 18.000580072402954, 'seconds': 0.9992630481719971, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8366236, 'rttvar': 7239721, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001317024230957, 'end': 18.000580072402954, 'seconds': 0.9992630481719971, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.000580072402954, 'end': 19.000797033309937, 'seconds': 1.0002169609069824, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.000580072402954, 'end': 19.000797033309937, 'seconds': 1.0002169609069824, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.000797033309937, 'end': 20.001474142074585, 'seconds': 1.0006771087646484, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.000797033309937, 'end': 20.001474142074585, 'seconds': 1.0006771087646484, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001474142074585, 'end': 21.000576972961426, 'seconds': 0.9991028308868408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001474142074585, 'end': 21.000576972961426, 'seconds': 0.9991028308868408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.000576972961426, 'end': 22.001118183135986, 'seconds': 1.0005412101745605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10496162, 'rttvar': 7820208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.000576972961426, 'end': 22.001118183135986, 'seconds': 1.0005412101745605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001118183135986, 'end': 23.001479148864746, 'seconds': 1.0003609657287598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10496162, 'rttvar': 7820208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001118183135986, 'end': 23.001479148864746, 'seconds': 1.0003609657287598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001479148864746, 'end': 24.001168966293335, 'seconds': 0.9996898174285889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 11651412, 'rttvar': 8175655, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001479148864746, 'end': 24.001168966293335, 'seconds': 0.9996898174285889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001168966293335, 'end': 25.001140117645264, 'seconds': 0.9999711513519287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12672600, 'rttvar': 8174116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001168966293335, 'end': 25.001140117645264, 'seconds': 0.9999711513519287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001140117645264, 'end': 26.000615119934082, 'seconds': 0.9994750022888184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12672600, 'rttvar': 8174116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001140117645264, 'end': 26.000615119934082, 'seconds': 0.9994750022888184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.000615119934082, 'end': 27.00078797340393, 'seconds': 1.0001728534698486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 13753602, 'rttvar': 8292592, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.000615119934082, 'end': 27.00078797340393, 'seconds': 1.0001728534698486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00078797340393, 'end': 28.00065517425537, 'seconds': 0.9998672008514404, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14696598, 'rttvar': 8105436, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00078797340393, 'end': 28.00065517425537, 'seconds': 0.9998672008514404, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.00065517425537, 'end': 29.0013210773468, 'seconds': 1.0006659030914307, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14696598, 'rttvar': 8105436, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.00065517425537, 'end': 29.0013210773468, 'seconds': 1.0006659030914307, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.0013210773468, 'end': 30.000684022903442, 'seconds': 0.9993629455566406, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 15721861, 'rttvar': 8129602, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.0013210773468, 'end': 30.000684022903442, 'seconds': 0.9993629455566406, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.000684022903442, 'end': 31.00068497657776, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16616168, 'rttvar': 7885816, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.000684022903442, 'end': 31.00068497657776, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00068497657776, 'end': 32.00132203102112, 'seconds': 1.0006370544433594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16616168, 'rttvar': 7885816, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00068497657776, 'end': 32.00132203102112, 'seconds': 1.0006370544433594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00132203102112, 'end': 33.00074911117554, 'seconds': 0.999427080154419, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 17588449, 'rttvar': 7858924, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00132203102112, 'end': 33.00074911117554, 'seconds': 0.999427080154419, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00074911117554, 'end': 34.00058197975159, 'seconds': 0.9998328685760498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18444463, 'rttvar': 7606221, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00074911117554, 'end': 34.00058197975159, 'seconds': 0.9998328685760498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00058197975159, 'end': 35.0013210773468, 'seconds': 1.0007390975952148, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18444463, 'rttvar': 7606221, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00058197975159, 'end': 35.0013210773468, 'seconds': 1.0007390975952148, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.0013210773468, 'end': 36.00113916397095, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 19375780, 'rttvar': 7567300, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.0013210773468, 'end': 36.00113916397095, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00113916397095, 'end': 37.00057411193848, 'seconds': 0.9994349479675293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20193508, 'rttvar': 7310931, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00113916397095, 'end': 37.00057411193848, 'seconds': 0.9994349479675293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00057411193848, 'end': 38.00039196014404, 'seconds': 0.9998178482055664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20193508, 'rttvar': 7310931, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00057411193848, 'end': 38.00039196014404, 'seconds': 0.9998178482055664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00039196014404, 'end': 39.0006890296936, 'seconds': 1.0002970695495605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 21094187, 'rttvar': 7284557, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00039196014404, 'end': 39.0006890296936, 'seconds': 1.0002970695495605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0006890296936, 'end': 40.00058197975159, 'seconds': 0.9998929500579834, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21887500, 'rttvar': 7050044, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0006890296936, 'end': 40.00058197975159, 'seconds': 0.9998929500579834, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00058197975159, 'end': 41.00132203102112, 'seconds': 1.0007400512695312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21887500, 'rttvar': 7050044, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00058197975159, 'end': 41.00132203102112, 'seconds': 1.0007400512695312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00132203102112, 'end': 42.000596046447754, 'seconds': 0.9992740154266357, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 22769210, 'rttvar': 7050952, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00132203102112, 'end': 42.000596046447754, 'seconds': 0.9992740154266357, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.000596046447754, 'end': 43.00114297866821, 'seconds': 1.000546932220459, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23532537, 'rttvar': 6814868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.000596046447754, 'end': 43.00114297866821, 'seconds': 1.000546932220459, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00114297866821, 'end': 44.00148797035217, 'seconds': 1.00034499168396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23532537, 'rttvar': 6814868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00114297866821, 'end': 44.00148797035217, 'seconds': 1.00034499168396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00148797035217, 'end': 45.000646114349365, 'seconds': 0.9991581439971924, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 24400576, 'rttvar': 6847229, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00148797035217, 'end': 45.000646114349365, 'seconds': 0.9991581439971924, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.000646114349365, 'end': 46.00114417076111, 'seconds': 1.0004980564117432, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25155044, 'rttvar': 6644358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.000646114349365, 'end': 46.00114417076111, 'seconds': 1.0004980564117432, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00114417076111, 'end': 47.0008499622345, 'seconds': 0.9997057914733887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25155044, 'rttvar': 6644358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00114417076111, 'end': 47.0008499622345, 'seconds': 0.9997057914733887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.0008499622345, 'end': 48.00132203102112, 'seconds': 1.000472068786621, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26022865, 'rttvar': 6718910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.0008499622345, 'end': 48.00132203102112, 'seconds': 1.000472068786621, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.00132203102112, 'end': 49.0011351108551, 'seconds': 0.9998130798339844, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26022865, 'rttvar': 6718910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.00132203102112, 'end': 49.0011351108551, 'seconds': 0.9998130798339844, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.0011351108551, 'end': 50.00085496902466, 'seconds': 0.9997198581695557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 26794956, 'rttvar': 6583365, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.0011351108551, 'end': 50.00085496902466, 'seconds': 0.9997198581695557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00085496902466, 'end': 51.000646114349365, 'seconds': 0.999791145324707, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27670031, 'rttvar': 6687674, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00085496902466, 'end': 51.000646114349365, 'seconds': 0.999791145324707, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.000646114349365, 'end': 52.00117516517639, 'seconds': 1.0005290508270264, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27670031, 'rttvar': 6687674, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.000646114349365, 'end': 52.00117516517639, 'seconds': 1.0005290508270264, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00117516517639, 'end': 53.00131797790527, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 28446091, 'rttvar': 6567875, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00117516517639, 'end': 53.00131797790527, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00131797790527, 'end': 54.00036907196045, 'seconds': 0.9990510940551758, 'bytes': 256296, 'bits_per_second': 2052315.4543352732, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29302817, 'rttvar': 6639359, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00131797790527, 'end': 54.00036907196045, 'seconds': 0.9990510940551758, 'bytes': 256296, 'bits_per_second': 2052315.4543352732, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00036907196045, 'end': 55.00075912475586, 'seconds': 1.0003900527954102, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29302817, 'rttvar': 6639359, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00036907196045, 'end': 55.00075912475586, 'seconds': 1.0003900527954102, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.00075912475586, 'end': 56.00075316429138, 'seconds': 0.9999940395355225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 29869581, 'rttvar': 6113047, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.00075912475586, 'end': 56.00075316429138, 'seconds': 0.9999940395355225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.00075316429138, 'end': 57.000683069229126, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30555344, 'rttvar': 5956311, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.00075316429138, 'end': 57.000683069229126, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.000683069229126, 'end': 58.001219034194946, 'seconds': 1.0005359649658203, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30555344, 'rttvar': 5956311, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.000683069229126, 'end': 58.001219034194946, 'seconds': 1.0005359649658203, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001219034194946, 'end': 59.00129508972168, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 31175458, 'rttvar': 5707460, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001219034194946, 'end': 59.00129508972168, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 18:48:59.859128 tst iperf3m4.py:8: ERR: KeyError: 'sum_sent' [trial-2064↪gprs:sysmo↪iperf3m4.py:8] 18:48:59.866575 tst iperf3m4.py:8: Test FAILED (294.8 sec) 18:48:59.873349 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Terminating (SIGTERM) 18:48:59.916435 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Terminating (SIGTERM) 18:48:59.958484 run iperf3-srv_10.42.42.10(pid=26971): Terminating (SIGTERM) 18:48:59.999372 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Terminating (SIGTERM) 18:49:00.040518 run iperf3-srv_10.42.42.10(pid=26973): Terminating (SIGTERM) 18:49:00.081176 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Terminating (SIGTERM) 18:49:00.122120 run iperf3-srv_10.42.42.10(pid=26975): Terminating (SIGTERM) 18:49:00.162967 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Terminating (SIGTERM) 18:49:00.203950 run osmo-hlr_10.42.42.2(pid=26979): Terminating (SIGTERM) 18:49:00.244856 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Terminating (SIGTERM) 18:49:00.287028 run osmo-stp_10.42.42.5(pid=26981): Terminating (SIGTERM) 18:49:00.328772 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Terminating (SIGTERM) 18:49:00.382622 run osmo-ggsn_10.42.42.6(pid=26987): Terminating (SIGTERM) 18:49:00.424467 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Terminating (SIGTERM) 18:49:00.473819 run osmo-sgsn_10.42.42.7(pid=27009): Terminating (SIGTERM) 18:49:00.529295 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Terminating (SIGTERM) 18:49:00.580227 run osmo-msc_10.42.42.8(pid=27011): Terminating (SIGTERM) 18:49:00.633554 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Terminating (SIGTERM) 18:49:00.686829 run osmo-mgw_10.42.42.3(pid=27013): Terminating (SIGTERM) 18:49:00.750948 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Terminating (SIGTERM) 18:49:00.799371 run osmo-mgw_10.42.42.4(pid=27015): Terminating (SIGTERM) 18:49:00.844649 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Terminating (SIGTERM) 18:49:00.890072 run osmo-bsc_10.42.42.9(pid=27017): Terminating (SIGTERM) 18:49:00.935822 run osmo-bts-sysmo(pid=27026): Terminating (SIGTERM) 18:49:00.983304 run osmo-pcu(pid=27037): Terminating (SIGTERM) 18:49:01.029399 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGTERM) 18:49:01.176449 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGTERM(pid=27181): Launched 18:49:02.277299 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGTERM(pid=27181): Terminated: ok {rc=0} 18:49:02.344389 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGTERM) 18:49:02.488995 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGTERM(pid=27186): Launched 18:49:03.577455 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGTERM(pid=27186): Terminated: ok {rc=0} 18:49:03.643834 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGTERM) 18:49:03.787356 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGTERM(pid=27189): Launched 18:49:04.880339 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGTERM(pid=27189): Terminated: ok {rc=0} 18:49:04.946489 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGTERM) 18:49:05.090521 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGTERM(pid=27192): Launched 18:49:06.177574 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGTERM(pid=27192): Terminated: ok {rc=0} 18:49:06.244170 --- ParallelTerminationStrategy: PID 26966 died... 18:49:06.264257 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Terminated: ok {rc=0} 18:49:06.311419 --- ParallelTerminationStrategy: PID 26969 died... 18:49:06.331213 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Terminated: ok {rc=0} 18:49:06.377416 --- ParallelTerminationStrategy: PID 26971 died... 18:49:06.397009 run iperf3-srv_10.42.42.10(pid=26971): Terminated {rc=256} 18:49:06.443413 --- ParallelTerminationStrategy: PID 26972 died... 18:49:06.463037 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Terminated: ok {rc=0} 18:49:06.509326 --- ParallelTerminationStrategy: PID 26973 died... 18:49:06.528920 run iperf3-srv_10.42.42.10(pid=26973): Terminated {rc=256} 18:49:06.577483 --- ParallelTerminationStrategy: PID 26974 died... 18:49:06.596438 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Terminated: ok {rc=0} 18:49:06.642684 --- ParallelTerminationStrategy: PID 26975 died... 18:49:06.661640 run iperf3-srv_10.42.42.10(pid=26975): Terminated {rc=256} 18:49:06.707945 --- ParallelTerminationStrategy: PID 26978 died... 18:49:06.727174 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Terminated: ok {rc=0} 18:49:06.773390 --- ParallelTerminationStrategy: PID 26979 died... 18:49:06.792396 run osmo-hlr_10.42.42.2(pid=26979): Terminated: ok {rc=0} 18:49:06.838635 --- ParallelTerminationStrategy: PID 26980 died... 18:49:06.857660 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Terminated: ok {rc=0} 18:49:06.903935 --- ParallelTerminationStrategy: PID 26981 died... 18:49:06.922961 run osmo-stp_10.42.42.5(pid=26981): Terminated: ok {rc=0} 18:49:06.968966 --- ParallelTerminationStrategy: PID 26982 died... 18:49:06.988234 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Terminated: ok {rc=0} 18:49:07.034375 --- ParallelTerminationStrategy: PID 26987 died... 18:49:07.053316 run osmo-ggsn_10.42.42.6(pid=26987): Terminated: ok {rc=0} 18:49:07.099491 --- ParallelTerminationStrategy: PID 27008 died... 18:49:07.118433 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Terminated: ok {rc=0} 18:49:07.164729 --- ParallelTerminationStrategy: PID 27009 died... 18:49:07.183676 run osmo-sgsn_10.42.42.7(pid=27009): Terminated: ok {rc=0} 18:49:07.229597 --- ParallelTerminationStrategy: PID 27010 died... 18:49:07.248614 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Terminated: ok {rc=0} 18:49:07.294555 --- ParallelTerminationStrategy: PID 27011 died... 18:49:07.313817 run osmo-msc_10.42.42.8(pid=27011): Terminated: ok {rc=0} 18:49:07.359845 --- ParallelTerminationStrategy: PID 27012 died... 18:49:07.378951 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Terminated: ok {rc=0} 18:49:07.425626 --- ParallelTerminationStrategy: PID 27013 died... 18:49:07.444927 run osmo-mgw_10.42.42.3(pid=27013): Terminated {rc=15} 18:49:07.491404 --- ParallelTerminationStrategy: PID 27014 died... 18:49:07.510826 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Terminated: ok {rc=0} 18:49:07.557304 --- ParallelTerminationStrategy: PID 27015 died... 18:49:07.576806 run osmo-mgw_10.42.42.4(pid=27015): Terminated {rc=15} 18:49:07.627753 --- ParallelTerminationStrategy: PID 27016 died... 18:49:07.647875 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Terminated: ok {rc=0} 18:49:07.694884 --- ParallelTerminationStrategy: PID 27017 died... 18:49:07.714878 run osmo-bsc_10.42.42.9(pid=27017): Terminated: ok {rc=0} 18:49:07.762099 --- ParallelTerminationStrategy: PID 27026 died... 18:49:07.782164 run osmo-bts-sysmo(pid=27026): Terminated: ok {rc=0} 18:49:07.829070 --- ParallelTerminationStrategy: PID 27037 died... 18:49:07.849277 run osmo-pcu(pid=27037): Terminated: ok {rc=0} 18:49:13.719053 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGINT) 18:49:13.862564 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGINT(pid=27199): Launched 18:49:14.951823 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGINT(pid=27199): Terminated: ok {rc=0} 18:49:15.017876 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGINT) 18:49:15.161781 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGINT(pid=27202): Launched 18:49:16.246800 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGINT(pid=27202): Terminated: ok {rc=0} 18:49:16.313204 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGINT) 18:49:16.455282 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGINT(pid=27205): Launched 18:49:17.539859 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGINT(pid=27205): Terminated: ok {rc=0} 18:49:17.606118 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGINT) 18:49:17.749720 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGINT(pid=27208): Launched 18:49:18.842561 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGINT(pid=27208): Terminated: ok {rc=0} 18:49:24.731262 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGKILL) 18:49:24.875297 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGUSR1(pid=27211): Launched 18:49:25.965709 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGUSR1(pid=27211): Terminated: ok {rc=0} 18:49:26.031968 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGKILL) 18:49:26.174568 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGUSR1(pid=27214): Launched 18:49:27.265545 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGUSR1(pid=27214): Terminated: ok {rc=0} 18:49:27.331672 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGKILL) 18:49:27.474564 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGUSR1(pid=27217): Launched 18:49:28.565339 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGUSR1(pid=27217): Terminated: ok {rc=0} 18:49:28.631806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGKILL) 18:49:28.774252 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGUSR1(pid=27220): Launched 18:49:29.859536 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGUSR1(pid=27220): Terminated: ok {rc=0} 18:49:30.105783 bus /gobi_4: Setting Online False 18:49:31.244861 bus /gobi_4: Setting Powered False 18:49:45.944600 bus /gobi_4: Setting Powered False 18:49:47.044501 bus /gobi_4: Setting Powered False 18:49:48.202038 bus /gobi_1: Setting Online False 18:49:49.324698 bus /gobi_1: Setting Powered False 18:49:57.098064 bus /gobi_1: Setting Powered False 18:49:58.197022 bus /gobi_1: Setting Powered False 18:49:59.354516 bus /gobi_0: Setting Online False 18:50:00.491432 bus /gobi_0: Setting Powered False 18:50:08.937999 bus /gobi_0: Setting Powered False 18:50:10.037687 bus /gobi_0: Setting Powered False 18:50:11.196010 bus /gobi_6: Setting Online False 18:50:12.331026 bus /gobi_6: Setting Powered False 18:50:20.944844 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2064 gprs:sysmo ping.py ---------------------------------------------- 18:50:22.103660 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.116667 tst gprs:sysmo: Using 1 x bts (candidates: 1) 18:50:22.150394 tst ping.py: using LAC 3948 18:50:22.271535 tst ping.py: using RAC 123 18:50:22.389731 tst ping.py: using CellId 3948 18:50:22.508184 tst ping.py: using BVCI 3949 18:50:22.619861 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.630352 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.640895 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.651947 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.662828 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.673696 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.684731 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:50:22.695704 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:50:22.813045 tst ping.py:19: start network... 18:50:23.015211 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:50:23.481826 run create_hlr_db(pid=27227): Launched 18:50:23.603372 bus /gobi_6: Setting Powered False 18:50:24.664060 run create_hlr_db(pid=27227): Terminated: ok {rc=0} 18:50:25.105335 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Launched 18:50:25.428549 run osmo-hlr_10.42.42.2(pid=27230): Launched 18:50:25.551650 run osmo-stp_10.42.42.5: Starting osmo-stp 18:50:26.101626 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Launched 18:50:26.426917 run osmo-stp_10.42.42.5(pid=27232): Launched 18:50:26.550984 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 18:50:27.104949 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Launched 18:50:27.228518 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 18:50:27.502204 run patchelf(pid=27234): Launched 18:50:28.685851 run patchelf(pid=27234): Terminated: ok {rc=0} 18:50:28.821713 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 18:50:29.098946 run osmo-gsm-tester_setcap_net_admin.sh(pid=27235): Launched 18:50:30.287309 run osmo-gsm-tester_setcap_net_admin.sh(pid=27235): Terminated: ok {rc=0} 18:50:30.624080 run osmo-ggsn_10.42.42.6(pid=27238): Launched 18:50:30.747704 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 18:50:31.422082 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Launched 18:50:31.725937 run osmo-sgsn_10.42.42.7(pid=27261): Launched 18:50:31.841137 run osmo-msc_10.42.42.8: Starting osmo-msc 18:50:32.403724 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Launched 18:50:32.705078 run osmo-msc_10.42.42.8(pid=27263): Launched 18:50:35.875207 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:50:36.389750 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Launched 18:50:36.692742 run osmo-mgw_10.42.42.3(pid=27265): Launched 18:50:36.808625 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:50:37.312282 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Launched 18:50:37.615382 run osmo-mgw_10.42.42.4(pid=27267): Launched 18:50:37.731142 run osmo-bsc_10.42.42.9: Starting osmo-bsc 18:50:38.355954 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Launched 18:50:38.658582 run osmo-bsc_10.42.42.9(pid=27269): Launched 18:50:44.924840 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 18:50:45.450442 run rm-remote-dir(pid=27270): Launched 18:50:47.630057 run rm-remote-dir(pid=27270): Terminated: ok {rc=0} 18:50:47.962609 run mk-remote-dir(pid=27271): Launched 18:50:49.139843 run mk-remote-dir(pid=27271): Terminated: ok {rc=0} 18:50:49.423231 run scp-inst-to-remote(pid=27272): Launched 18:51:01.614934 run scp-inst-to-remote(pid=27272): Terminated: ok {rc=0} 18:51:01.965540 run mk-remote-dir(pid=27274): Launched 18:51:03.153689 run mk-remote-dir(pid=27274): Terminated: ok {rc=0} 18:51:03.454827 run scp-cfg-to-remote(pid=27275): Launched 18:51:04.643143 run scp-cfg-to-remote(pid=27275): Terminated: ok {rc=0} 18:51:04.990265 run reload-dsp-firmware(pid=27277): Launched 18:51:07.177513 run reload-dsp-firmware(pid=27277): Terminated: ok {rc=0} 18:51:07.517467 run osmo-bts-sysmo(pid=27278): Launched 18:51:11.754838 tst ping.py:31: Waiting for bts to be ready... 18:51:13.470836 run rm-remote-dir(pid=27279): Launched 18:51:15.651499 run rm-remote-dir(pid=27279): Terminated: ok {rc=0} 18:51:15.970610 run mk-remote-dir(pid=27280): Launched 18:51:18.141366 run mk-remote-dir(pid=27280): Terminated: ok {rc=0} 18:51:18.414515 run scp-inst-to-bts(pid=27281): Launched 18:51:31.606348 run scp-inst-to-bts(pid=27281): Terminated: ok {rc=0} 18:51:31.949063 run mk-remote-run-dir(pid=27283): Launched 18:51:34.134158 run mk-remote-run-dir(pid=27283): Terminated: ok {rc=0} 18:51:34.426639 run scp-cfg-to-sysmobts(pid=27284): Launched 18:51:36.603879 run scp-cfg-to-sysmobts(pid=27284): Terminated: ok {rc=0} 18:51:36.922588 run osmo-pcu(pid=27286): Launched 18:51:37.045170 tst ping.py: using MSISDN 3956 18:51:37.217912 bus /gobi_6: Setting Powered True 18:51:40.726402 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3956', subscriber_id=1} 18:51:40.987713 bus /gobi_6: Setting Online False 18:51:42.173276 bus /gobi_6: Setting Powered False 18:51:49.105245 bus /gobi_6: Setting Powered True 18:51:50.463321 bus /gobi_6: Setting Online True 18:51:52.875153 tst /gobi_6: Connect to ('901', '70') 18:51:53.151325 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 18:51:53.268187 tst ping.py:42: waiting for modems to attach... 18:52:07.461869 tst /gobi_6: Already registered with network ('901', '70') 18:52:12.803006 tst ping.py:46: waiting for modems to attach to data services... 18:52:19.982260 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:52:20.089653 tst ping.py:51: Setting up data plan for "'/gobi_6/context3'" 18:52:20.497538 run move_netns(pid=27290): Launched 18:52:21.673417 run move_netns(pid=27290): Terminated: ok {rc=0} 18:52:21.995974 run ifup(pid=27293): Launched 18:52:23.170671 run ifup(pid=27293): Terminated: ok {rc=0} 18:52:23.516847 run dhcp(pid=27296): Launched 18:52:24.701866 run dhcp(pid=27296): Terminated: ok {rc=0} 18:52:24.829168 tst ping.py:53: Running 10 ping requests for "'/gobi_6/context3'" 18:52:25.221334 run ping(pid=27312): Launched 18:52:35.416106 run ping(pid=27312): Terminated: ok {rc=0} 18:52:37.180118 tst /gobi_6: context deactivated {path='/gobi_6/context3'} 18:52:44.456774 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={Interface='r433306bbc2604b'}, Name='Internet', Password='', Protocol='ipv6', Settings={}, Type='internet', Username='ogt'}, user='ogt'} 18:52:52.219378 tst /gobi_6: context deactivated {path='/gobi_6/context3'} 18:52:52.337523 tst ping.py:70: (launched: 2022-03-29_18:52:25.160958) 27313: waiting for 27314 PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. 64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=943 ms 64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=584 ms 64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=585 ms 64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=581 ms 64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=563 ms 64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=581 ms 64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=577 ms 64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=560 ms 64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=578 ms 64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=574 ms --- 10.42.42.6 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 18ms rtt min/avg/max/mdev = 559.738/612.790/943.318/110.469 ms child exited with 0 18:52:52.558240 tst ping.py: Test passed (150.5 sec) 18:52:52.626225 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Terminating (SIGTERM) 18:52:52.669513 run osmo-hlr_10.42.42.2(pid=27230): Terminating (SIGTERM) 18:52:52.713004 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Terminating (SIGTERM) 18:52:52.758484 run osmo-stp_10.42.42.5(pid=27232): Terminating (SIGTERM) 18:52:52.805607 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Terminating (SIGTERM) 18:52:52.871714 run osmo-ggsn_10.42.42.6(pid=27238): Terminating (SIGTERM) 18:52:52.918210 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Terminating (SIGTERM) 18:52:52.992420 run osmo-sgsn_10.42.42.7(pid=27261): Terminating (SIGTERM) 18:52:53.037946 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Terminating (SIGTERM) 18:52:53.083257 run osmo-msc_10.42.42.8(pid=27263): Terminating (SIGTERM) 18:52:53.127270 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Terminating (SIGTERM) 18:52:53.171173 run osmo-mgw_10.42.42.3(pid=27265): Terminating (SIGTERM) 18:52:53.215766 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Terminating (SIGTERM) 18:52:53.259448 run osmo-mgw_10.42.42.4(pid=27267): Terminating (SIGTERM) 18:52:53.301765 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Terminating (SIGTERM) 18:52:53.344995 run osmo-bsc_10.42.42.9(pid=27269): Terminating (SIGTERM) 18:52:53.387124 run osmo-bts-sysmo(pid=27278): Terminating (SIGTERM) 18:52:53.429318 run osmo-pcu(pid=27286): Terminating (SIGTERM) 18:52:53.472420 --- ParallelTerminationStrategy: PID 27229 died... 18:52:53.492815 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Terminated: ok {rc=0} 18:52:53.539802 --- ParallelTerminationStrategy: PID 27230 died... 18:52:53.559815 run osmo-hlr_10.42.42.2(pid=27230): Terminated: ok {rc=0} 18:52:53.606344 --- ParallelTerminationStrategy: PID 27231 died... 18:52:53.626146 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Terminated: ok {rc=0} 18:52:53.672982 --- ParallelTerminationStrategy: PID 27232 died... 18:52:53.692773 run osmo-stp_10.42.42.5(pid=27232): Terminated: ok {rc=0} 18:52:53.739317 --- ParallelTerminationStrategy: PID 27233 died... 18:52:53.759222 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Terminated: ok {rc=0} 18:52:53.806385 --- ParallelTerminationStrategy: PID 27238 died... 18:52:53.827039 run osmo-ggsn_10.42.42.6(pid=27238): Terminated: ok {rc=0} 18:52:53.875159 --- ParallelTerminationStrategy: PID 27260 died... 18:52:53.895859 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Terminated: ok {rc=0} 18:52:53.945206 --- ParallelTerminationStrategy: PID 27262 died... 18:52:53.966123 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Terminated: ok {rc=0} 18:52:54.027165 --- ParallelTerminationStrategy: PID 27264 died... 18:52:54.080419 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Terminated: ok {rc=0} 18:52:54.171259 --- ParallelTerminationStrategy: PID 27265 died... 18:52:54.228114 run osmo-mgw_10.42.42.3(pid=27265): Terminated {rc=15} 18:52:54.276430 --- ParallelTerminationStrategy: PID 27263 died... 18:52:54.297171 run osmo-msc_10.42.42.8(pid=27263): Terminated: ok {rc=0} 18:52:54.345432 --- ParallelTerminationStrategy: PID 27266 died... 18:52:54.366278 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Terminated: ok {rc=0} 18:52:54.414423 --- ParallelTerminationStrategy: PID 27267 died... 18:52:54.435201 run osmo-mgw_10.42.42.4(pid=27267): Terminated {rc=15} 18:52:54.483808 --- ParallelTerminationStrategy: PID 27268 died... 18:52:54.504445 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Terminated: ok {rc=0} 18:52:54.552566 --- ParallelTerminationStrategy: PID 27269 died... 18:52:54.573226 run osmo-bsc_10.42.42.9(pid=27269): Terminated: ok {rc=0} 18:52:54.621660 --- ParallelTerminationStrategy: PID 27278 died... 18:52:54.642676 run osmo-bts-sysmo(pid=27278): Terminated: ok {rc=0} 18:52:54.691239 --- ParallelTerminationStrategy: PID 27286 died... 18:52:54.712056 run osmo-pcu(pid=27286): Terminated: ok {rc=0} 18:52:54.928817 --- ParallelTerminationStrategy: PID 27261 died... 18:52:54.950844 run osmo-sgsn_10.42.42.7(pid=27261): Terminated: ok {rc=0} 18:52:55.176451 bus /gobi_6: Setting Online False 18:52:58.227734 bus /gobi_6: Setting Powered False 18:53:14.935274 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2064 gprs:sysmo ping_idle_ping.py ---------------------------------------------- 18:53:16.079068 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.090959 tst gprs:sysmo: Using 1 x bts (candidates: 1) 18:53:16.126507 tst ping_idle_ping.py: using LAC 3949 18:53:16.254656 tst ping_idle_ping.py: using RAC 124 18:53:16.384775 tst ping_idle_ping.py: using CellId 3949 18:53:16.505449 tst ping_idle_ping.py: using BVCI 3950 18:53:16.616094 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.626804 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.637471 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.648230 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.658742 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.669406 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.680100 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:53:16.690661 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:53:16.808349 tst ping_idle_ping.py:24: start network... 18:53:17.028600 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:53:17.533122 run create_hlr_db(pid=27333): Launched 18:53:17.658259 bus /gobi_6: Setting Powered False 18:53:18.723381 run create_hlr_db(pid=27333): Terminated: ok {rc=0} 18:53:19.159982 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Launched 18:53:19.466114 run osmo-hlr_10.42.42.2(pid=27336): Launched 18:53:19.584068 run osmo-stp_10.42.42.5: Starting osmo-stp 18:53:20.105723 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Launched 18:53:20.411899 run osmo-stp_10.42.42.5(pid=27338): Launched 18:53:20.530549 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 18:53:21.050559 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Launched 18:53:21.167295 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 18:53:21.424726 run patchelf(pid=27340): Launched 18:53:22.597045 run patchelf(pid=27340): Terminated: ok {rc=0} 18:53:22.724490 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 18:53:23.004661 run osmo-gsm-tester_setcap_net_admin.sh(pid=27341): Launched 18:53:24.182487 run osmo-gsm-tester_setcap_net_admin.sh(pid=27341): Terminated: ok {rc=0} 18:53:24.497812 run osmo-ggsn_10.42.42.6(pid=27344): Launched 18:53:24.622209 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 18:53:25.283124 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Launched 18:53:25.587362 run osmo-sgsn_10.42.42.7(pid=27362): Launched 18:53:25.703244 run osmo-msc_10.42.42.8: Starting osmo-msc 18:53:26.254298 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Launched 18:53:26.559736 run osmo-msc_10.42.42.8(pid=27364): Launched 18:53:29.731189 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:53:30.249747 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Launched 18:53:30.553884 run osmo-mgw_10.42.42.3(pid=27366): Launched 18:53:30.670312 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:53:31.176762 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Launched 18:53:31.480988 run osmo-mgw_10.42.42.4(pid=27368): Launched 18:53:31.597684 run osmo-bsc_10.42.42.9: Starting osmo-bsc 18:53:32.224502 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Launched 18:53:32.528231 run osmo-bsc_10.42.42.9(pid=27370): Launched 18:53:38.797577 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 18:53:39.327861 run rm-remote-dir(pid=27371): Launched 18:53:41.511474 run rm-remote-dir(pid=27371): Terminated: ok {rc=0} 18:53:41.872872 run mk-remote-dir(pid=27372): Launched 18:53:43.063839 run mk-remote-dir(pid=27372): Terminated: ok {rc=0} 18:53:43.371396 run scp-inst-to-remote(pid=27373): Launched 18:53:55.581549 run scp-inst-to-remote(pid=27373): Terminated: ok {rc=0} 18:53:55.938539 run mk-remote-dir(pid=27375): Launched 18:53:57.126607 run mk-remote-dir(pid=27375): Terminated: ok {rc=0} 18:53:57.431550 run scp-cfg-to-remote(pid=27376): Launched 18:53:58.619835 run scp-cfg-to-remote(pid=27376): Terminated: ok {rc=0} 18:53:58.970296 run reload-dsp-firmware(pid=27378): Launched 18:54:01.158887 run reload-dsp-firmware(pid=27378): Terminated: ok {rc=0} 18:54:01.502246 run osmo-bts-sysmo(pid=27379): Launched 18:54:05.742859 tst ping_idle_ping.py:36: Waiting for bts to be ready... 18:54:07.463034 run rm-remote-dir(pid=27380): Launched 18:54:09.647121 run rm-remote-dir(pid=27380): Terminated: ok {rc=0} 18:54:09.990246 run mk-remote-dir(pid=27383): Launched 18:54:12.172710 run mk-remote-dir(pid=27383): Terminated: ok {rc=0} 18:54:12.446443 run scp-inst-to-bts(pid=27384): Launched 18:54:25.631617 run scp-inst-to-bts(pid=27384): Terminated: ok {rc=0} 18:54:25.953028 run mk-remote-run-dir(pid=27386): Launched 18:54:28.124103 run mk-remote-run-dir(pid=27386): Terminated: ok {rc=0} 18:54:28.397430 run scp-cfg-to-sysmobts(pid=27387): Launched 18:54:30.569468 run scp-cfg-to-sysmobts(pid=27387): Terminated: ok {rc=0} 18:54:30.912395 run osmo-pcu(pid=27389): Launched 18:54:31.049751 tst ping_idle_ping.py: using MSISDN 3957 18:54:31.223360 bus /gobi_6: Setting Powered True 18:54:34.734287 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3957', subscriber_id=1} 18:54:34.997006 bus /gobi_6: Setting Online False 18:54:36.183822 bus /gobi_6: Setting Powered False 18:54:43.108849 bus /gobi_6: Setting Powered True 18:54:44.475734 bus /gobi_6: Setting Online True 18:54:46.966328 tst /gobi_6: Connect to ('901', '70') 18:54:47.251806 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 18:54:47.368116 tst ping_idle_ping.py:47: waiting for modems to attach... 18:54:54.342296 tst /gobi_6: Already registered with network ('901', '70') 18:55:05.838172 tst ping_idle_ping.py:51: waiting for modems to attach to data services... 18:55:58.827522 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 18:55:58.946065 tst ping_idle_ping.py:56: Setting up data plane for "'/gobi_6/context2'" 18:55:59.392822 run move_netns(pid=27396): Launched 18:56:00.587265 run move_netns(pid=27396): Terminated: ok {rc=0} 18:56:00.941440 run ifup(pid=27399): Launched 18:56:02.123449 run ifup(pid=27399): Terminated: ok {rc=0} 18:56:02.450444 run dhcp(pid=27402): Launched 18:56:03.626231 run dhcp(pid=27402): Terminated: ok {rc=0} 18:56:03.755621 tst ping_idle_ping.py:60: [1] Running 10 ping requests for "'/gobi_6/context2'" 18:56:04.155091 run ping1(pid=27418): Launched 18:56:14.359359 run ping1(pid=27418): Terminated: ok {rc=0} 18:56:14.495867 tst ping_idle_ping.py:64: (launched: 2022-03-29_18:56:04.093106) 27419: waiting for 27420 PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. 64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=679 ms 64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=638 ms 64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=615 ms 64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=596 ms 64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=592 ms 64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=532 ms 64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=533 ms 64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=492 ms 64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=488 ms 64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=487 ms --- 10.42.42.6 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 21ms rtt min/avg/max/mdev = 486.902/565.056/678.590/64.750 ms child exited with 0 18:56:14.718200 tst ping_idle_ping.py:68: Sleeping for 60 seconds 18:57:14.941764 tst ping_idle_ping.py:73: [2] Running 10 ping requests for "'/gobi_6/context2'" 18:57:15.372936 run ping2(pid=27424): Launched 18:57:25.578233 run ping2(pid=27424): Terminated: ok {rc=0} 18:57:25.714790 tst ping_idle_ping.py:77: (launched: 2022-03-29_18:57:15.307744) 27425: waiting for 27426 PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. 64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=536 ms 64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=454 ms 64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=431 ms 64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=413 ms 64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=431 ms 64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=427 ms 64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=408 ms 64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=405 ms 64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=405 ms 64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=381 ms --- 10.42.42.6 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 21ms rtt min/avg/max/mdev = 381.454/429.068/536.304/40.329 ms child exited with 0 18:57:27.707993 tst /gobi_6: context deactivated {path='/gobi_6/context2'} 18:57:27.817824 tst ping_idle_ping.py: Test passed (251.7 sec) 18:57:27.883815 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Terminating (SIGTERM) 18:57:27.925168 run osmo-hlr_10.42.42.2(pid=27336): Terminating (SIGTERM) 18:57:27.966164 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Terminating (SIGTERM) 18:57:28.009810 run osmo-stp_10.42.42.5(pid=27338): Terminating (SIGTERM) 18:57:28.059656 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Terminating (SIGTERM) 18:57:28.105538 run osmo-ggsn_10.42.42.6(pid=27344): Terminating (SIGTERM) 18:57:28.150157 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Terminating (SIGTERM) 18:57:28.210933 run osmo-sgsn_10.42.42.7(pid=27362): Terminating (SIGTERM) 18:57:28.278431 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Terminating (SIGTERM) 18:57:28.325669 run osmo-msc_10.42.42.8(pid=27364): Terminating (SIGTERM) 18:57:28.371429 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Terminating (SIGTERM) 18:57:28.416376 run osmo-mgw_10.42.42.3(pid=27366): Terminating (SIGTERM) 18:57:28.461094 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Terminating (SIGTERM) 18:57:28.507834 run osmo-mgw_10.42.42.4(pid=27368): Terminating (SIGTERM) 18:57:28.551819 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Terminating (SIGTERM) 18:57:28.596373 run osmo-bsc_10.42.42.9(pid=27370): Terminating (SIGTERM) 18:57:28.639641 run osmo-bts-sysmo(pid=27379): Terminating (SIGTERM) 18:57:28.683368 run osmo-pcu(pid=27389): Terminating (SIGTERM) 18:57:28.727835 --- ParallelTerminationStrategy: PID 27335 died... 18:57:28.748547 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Terminated: ok {rc=0} 18:57:28.797671 --- ParallelTerminationStrategy: PID 27336 died... 18:57:28.818327 run osmo-hlr_10.42.42.2(pid=27336): Terminated: ok {rc=0} 18:57:28.867859 --- ParallelTerminationStrategy: PID 27337 died... 18:57:28.888557 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Terminated: ok {rc=0} 18:57:28.938065 --- ParallelTerminationStrategy: PID 27338 died... 18:57:28.958857 run osmo-stp_10.42.42.5(pid=27338): Terminated: ok {rc=0} 18:57:29.008339 --- ParallelTerminationStrategy: PID 27339 died... 18:57:29.028524 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Terminated: ok {rc=0} 18:57:29.076642 --- ParallelTerminationStrategy: PID 27344 died... 18:57:29.097617 run osmo-ggsn_10.42.42.6(pid=27344): Terminated: ok {rc=0} 18:57:29.145956 --- ParallelTerminationStrategy: PID 27361 died... 18:57:29.166800 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Terminated: ok {rc=0} 18:57:29.215559 --- ParallelTerminationStrategy: PID 27363 died... 18:57:29.236338 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Terminated: ok {rc=0} 18:57:29.285039 --- ParallelTerminationStrategy: PID 27364 died... 18:57:29.305812 run osmo-msc_10.42.42.8(pid=27364): Terminated: ok {rc=0} 18:57:29.354191 --- ParallelTerminationStrategy: PID 27365 died... 18:57:29.375437 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Terminated: ok {rc=0} 18:57:29.424266 --- ParallelTerminationStrategy: PID 27366 died... 18:57:29.445247 run osmo-mgw_10.42.42.3(pid=27366): Terminated {rc=15} 18:57:29.493765 --- ParallelTerminationStrategy: PID 27367 died... 18:57:29.514624 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Terminated: ok {rc=0} 18:57:29.563536 --- ParallelTerminationStrategy: PID 27368 died... 18:57:29.584396 run osmo-mgw_10.42.42.4(pid=27368): Terminated {rc=15} 18:57:29.632948 --- ParallelTerminationStrategy: PID 27369 died... 18:57:29.653818 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Terminated: ok {rc=0} 18:57:29.702845 --- ParallelTerminationStrategy: PID 27370 died... 18:57:29.723571 run osmo-bsc_10.42.42.9(pid=27370): Terminated: ok {rc=0} 18:57:29.772222 --- ParallelTerminationStrategy: PID 27379 died... 18:57:29.793199 run osmo-bts-sysmo(pid=27379): Terminated: ok {rc=0} 18:57:29.842106 --- ParallelTerminationStrategy: PID 27389 died... 18:57:29.863251 run osmo-pcu(pid=27389): Terminated: ok {rc=0} 18:57:30.080476 --- ParallelTerminationStrategy: PID 27362 died... 18:57:30.102519 run osmo-sgsn_10.42.42.7(pid=27362): Terminated: ok {rc=0} 18:57:30.325893 bus /gobi_6: Setting Online False 18:57:33.386328 bus /gobi_6: Setting Powered False 18:57:49.001216 bus /gobi_6: Setting Powered False --------------------------------------------------------------------- trial-2064 gprs:sysmo FAIL --------------------------------------------------------------------- 18:57:50.341374 tst trial-2064: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064/run.2022-03-29_18-32-52/trial-2064.xml -------------------------------------------------------------------------------------------- trial-2064 FAIL -------------------------------------------------------------------------------------------- 18:57:50.571122 tst trial-2064: trial-2064: FAIL, 2 suites failed, 1 suites passed FAIL: nitb_sms:sysmo (fail: 1) FAIL: mo_mt_sms.py (5.1 sec) Error: osmo-nitb_10.42.42.2(pid=26640): Process ended prematurely: osmo-nitb_10.42.42.2(pid=26640) [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=26640)] FAIL: gprs:sysmo (fail: 3, pass: 2) FAIL: cs_paging_gprs_active.py (228.9 sec) KeyError: 'sum_sent' FAIL: iperf3.py (196.8 sec) KeyError: 'sum_sent' FAIL: iperf3m4.py (294.8 sec) KeyError: 'sum_sent' pass: ping.py (150.5 sec) pass: ping_idle_ping.py (251.7 sec) PASS: sms:sysmo (pass: 1) pass: mo_mt_sms.py (116.8 sec) + rm -rf trial-2064/inst + cd trial-2064 + command -v journalctl + [ ! -z /bin/journalctl ] + readlink last_run + journalctl -u ofono -o short-precise --since 2022-03-29 18:32:50 + readlink last_run + tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064-run.tgz run.2022-03-29_18-32-52 + tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064-bin.tgz checksums.md5 open5gs.build-8.tgz osmo-bsc.build-15570.tgz osmo-bts-oc2g.build-4604.tgz osmo-bts-sysmo.build-7108.tgz osmo-bts.build-6578.tgz osmo-ggsn.build-5814.tgz osmo-hlr.build-7536.tgz osmo-hnbgw.build-592.tgz osmo-mgw.build-7731.tgz osmo-msc.build-25979.tgz osmo-nitb.build-11307.tgz osmo-pcu-oc2g.build-3308.tgz osmo-pcu-sysmo.build-4589.tgz osmo-pcu.build-4550.tgz osmo-sgsn.build-26313.tgz osmo-stp.build-8062.tgz osmo-trx.build-3840.tgz osmocom-bb.build-4035.tgz srslte.build-19.tgz + exit 1 Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results [Checks API] No suitable checks publisher found. ERROR Gerrit response: ConflictFinished: FAILURE