![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
Triggered by Gerrit: https://gerrit.osmocom.org/c/osmo-gsm-tester/+/29290 Running as SYSTEM Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_gerrit The recommended git tool is: NONE using credential d5eda5e9-b59d-44ba-88d2-43473cb6e42d > git rev-parse --resolve-git-dir /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/.git # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Fetching upstream changes from ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester > git --version # timeout=10 > git --version # 'git version 2.20.1' using GIT_SSH to set credentials > git fetch --tags --force --progress -- ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/90/29290/1 # timeout=10 > git rev-parse 47d5c881147aacf571d8011c7ab3e319981448fa^{commit} # timeout=10 JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://plugins.jenkins.io/git/#remove-git-plugin-buildsbybranch-builddata-script Checking out Revision 47d5c881147aacf571d8011c7ab3e319981448fa (refs/changes/90/29290/1) > git config core.sparsecheckout # timeout=10 > git checkout -f 47d5c881147aacf571d8011c7ab3e319981448fa # timeout=10 Commit message: "WIP: Workaround poky SDK bug building libosmo-netif" > git rev-parse FETCH_HEAD^{commit} # timeout=10 > git rev-list --no-walk 93d3b9df9d07e0fc7934db601370288b7d1b6f1e # timeout=10 [osmo-gsm-tester_gerrit] $ /bin/sh -xe /tmp/jenkins8443448694404525086.sh + rm -f trial-2066-bin.tgz trial-2066-run.tgz *.md5 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bsc" build number 16895 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts" build number 7170 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-ggsn" build number 6368 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hlr" build number 8266 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-mgw" build number 8474 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hnbgw" build number 2029 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-msc" build number 28089 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu" build number 4935 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-sgsn" build number 28698 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-trx" build number 4363 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-nitb" build number 12352 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-stp" build number 8803 Copied 2 artifacts from "osmo-gsm-tester_build-osmocom-bb" build number 4444 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-sysmo" build number 7696 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-sysmo" build number 4973 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-oc2g" build number 5192 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-oc2g" build number 3692 Copied 2 artifacts from "osmo-gsm-tester_build-open5gs" build number 8 Copied 2 artifacts from "osmo-gsm-tester_build-srslte" build number 19 [osmo-gsm-tester_gerrit] $ /bin/sh /tmp/jenkins9132059070059372347.sh + unlink osmo-gsm-tester/sysmocom/resources.conf unlink: cannot unlink 'osmo-gsm-tester/sysmocom/resources.conf': No such file or directory + true + ln -s resources.conf.prod osmo-gsm-tester/sysmocom/resources.conf + export OSMO_GSM_TESTER_OPTS=-s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo + ./osmo-gsm-tester/contrib/jenkins-make-check-and-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit + run_script=osmo-gsm-tester/contrib/jenkins-run.sh + test -x osmo-gsm-tester/contrib/jenkins-run.sh + cd osmo-gsm-tester + make deps ./contrib/check_dependencies.py Skip checking modules: [] Changing workdir dir to /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_ms_driver importing src.osmo_ms_driver.location_update_test importing src.osmo_ms_driver.event_server importing src.osmo_ms_driver.cdf importing src.osmo_ms_driver.__main__ importing src.osmo_ms_driver.__init__ importing src.osmo_ms_driver.test_support importing src.osmo_ms_driver.starter importing src.osmo_ms_driver.simple_loop importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core importing src.osmo_gsm_tester.core.util importing src.osmo_gsm_tester.core.schema importing src.osmo_gsm_tester.core.trial importing src.osmo_gsm_tester.core.template importing src.osmo_gsm_tester.core.config importing src.osmo_gsm_tester.core.event_loop importing src.osmo_gsm_tester.core.log importing src.osmo_gsm_tester.core.scenario importing src.osmo_gsm_tester.core.remote importing src.osmo_gsm_tester.core.__init__ importing src.osmo_gsm_tester.core.process importing src.osmo_gsm_tester.core.report importing src.osmo_gsm_tester.core.test importing src.osmo_gsm_tester.core.resource importing src.osmo_gsm_tester.core.suite importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj importing src.osmo_gsm_tester.obj.android_apn importing src.osmo_gsm_tester.obj.bts importing src.osmo_gsm_tester.obj.mgcpgw_osmo importing src.osmo_gsm_tester.obj.sgsn_osmo importing src.osmo_gsm_tester.obj.epc_srs importing src.osmo_gsm_tester.obj.bts_nanobts importing src.osmo_gsm_tester.obj.enb_srs importing src.osmo_gsm_tester.obj.esme importing src.osmo_gsm_tester.obj.qc_diag importing src.osmo_gsm_tester.obj.stp_osmo importing src.osmo_gsm_tester.obj.ms_ofono importing src.osmo_gsm_tester.obj.ms_android importing src.osmo_gsm_tester.obj.osmo_vty importing src.osmo_gsm_tester.obj.hlr_osmo importing src.osmo_gsm_tester.obj.enb_amarisoft importing src.osmo_gsm_tester.obj.android_host importing src.osmo_gsm_tester.obj.sgwu_open5gs importing src.osmo_gsm_tester.obj.powersupply importing src.osmo_gsm_tester.obj.iperf3 importing src.osmo_gsm_tester.obj.epc importing src.osmo_gsm_tester.obj.rfemu_minicircuits importing src.osmo_gsm_tester.obj.bts_osmo importing src.osmo_gsm_tester.obj.upf_open5gs importing src.osmo_gsm_tester.obj.nitb_osmo importing src.osmo_gsm_tester.obj.android_bitrate_monitor importing src.osmo_gsm_tester.obj.pcu_oc2g importing src.osmo_gsm_tester.obj.ms_srs importing src.osmo_gsm_tester.obj.ms_driver importing src.osmo_gsm_tester.obj.bts_sysmo importing src.osmo_gsm_tester.obj.rfemu_srsenb_stdin importing src.osmo_gsm_tester.obj.pcu importing src.osmo_gsm_tester.obj.osmocon importing src.osmo_gsm_tester.obj.epc_amarisoft importing src.osmo_gsm_tester.obj.ggsn_osmo importing src.osmo_gsm_tester.obj.enb importing src.osmo_gsm_tester.obj.rfemu importing src.osmo_gsm_tester.obj.bts_octphy importing src.osmo_gsm_tester.obj.pcap_recorder importing src.osmo_gsm_tester.obj.sms importing src.osmo_gsm_tester.obj.bts_oc2g importing src.osmo_gsm_tester.obj.stress importing src.osmo_gsm_tester.obj.smsc importing src.osmo_gsm_tester.obj.msc_osmo importing src.osmo_gsm_tester.obj.sgwc_open5gs importing src.osmo_gsm_tester.obj.osmo_ctrl importing src.osmo_gsm_tester.obj.bts_osmotrx importing src.osmo_gsm_tester.obj.rfemu_amarisoftctrl importing src.osmo_gsm_tester.obj.ms importing src.osmo_gsm_tester.obj.powersupply_intellinet importing src.osmo_gsm_tester.obj.ms_amarisoft importing src.osmo_gsm_tester.obj.srslte_common importing src.osmo_gsm_tester.obj.powersupply_sispm importing src.osmo_gsm_tester.obj.gnuradio_zmq_broker importing src.osmo_gsm_tester.obj.pcu_sysmo importing src.osmo_gsm_tester.obj.bts_osmovirtual importing src.osmo_gsm_tester.obj.hss_open5gs importing src.osmo_gsm_tester.obj.mgw_osmo importing src.osmo_gsm_tester.obj.smf_open5gs importing src.osmo_gsm_tester.obj.pcrf_open5gs importing src.osmo_gsm_tester.obj.mme_open5gs importing src.osmo_gsm_tester.obj.bsc_osmo importing src.osmo_gsm_tester.obj.epc_open5gs importing src.osmo_gsm_tester.obj.run_node importing src.osmo_gsm_tester.obj.pcu_osmo importing src.osmo_gsm_tester.obj.rfemu_gnuradio_zmq importing src.osmo_gsm_tester.obj.ms_osmo_mobile importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester importing src.osmo_gsm_tester.testenv importing src.osmo_gsm_tester.__init__ Importing dependencies ok, all installed Retreiving list of imported modules... + make check make -C selftest check make[1]: Entering directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest' echo "export PYTHONPATH=\"/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/../src\"" > set_pythonpath ./all_tests.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/cdf_test/cdf_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/config_test/config_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/lock_test/lock_test.sh /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/log_test/log_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/process_test/process_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/rate_ctrs_test/rate_ctrs_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/report_test/report_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/resource_test/resource_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/scenario_test/scenario_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/schema_test/schema_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/sms_test/sms_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/suite_test/suite_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/template_test/template_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/trial_test/trial_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/util_test/util_test.py 15 tests ok make[1]: Leaving directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest' make check: success + ./contrib/jenkins-build-manuals.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester + export OSMO_GSM_MANUALS_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals ] + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals ] + git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals pull warning: redirecting to https://gitea.osmocom.org/cellular-infrastructure/osmo-gsm-manuals/ Already up to date. + git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals checkout -f HEAD Your branch is up to date with 'origin/master'. + temp=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp ] + cp -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + cd /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + make OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env) ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build build ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common common # a2x can't use a different output file. To support out-of-tree builds, # we create a symlink at /srcfile.adoc pointing at # /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc. osmo-gsm-tester-manual.adoc is the /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc, # is like basename from coreutils, and is WD. if ! [ -f osmo-gsm-tester-manual.adoc ]; then \ ln -s osmo-gsm-tester-manual.adoc osmo-gsm-tester-manual.adoc; \ fi # TEXINPUTS: find LaTeX includes like \includegraphics{./common/images/sysmocom.pdf} # in /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals. TEXINPUTS="/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals" \ a2x -vv -L --asciidoc-opts="-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'" --dblatex-opts="-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0" -a docinfo -a revnumber="DRAFT 0.1-1217-g47d5" -a revdate="2022-Sep-08" osmo-gsm-tester-manual.adoc a2x: args: ['-vv', '-L', "--asciidoc-opts=-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'", '--dblatex-opts=-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0', '-a', 'docinfo', '-a', 'revnumber=DRAFT 0.1-1217-g47d5', '-a', 'revdate=2022-Sep-08', 'osmo-gsm-tester-manual.adoc'] a2x: resource files: [] a2x: resource directories: ['/etc/asciidoc/images', '/etc/asciidoc/stylesheets'] a2x: executing: "/usr/bin/asciidoc" --backend docbook -a "a2x-format=pdf" -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common' --attribute "docinfo" --attribute "revnumber=DRAFT 0.1-1217-g47d5" --attribute "revdate=2022-Sep-08" --verbose --out-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml" "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc" asciidoc: reading: /etc/asciidoc/asciidoc.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf asciidoc: reading: /etc/asciidoc/asciidoc.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc asciidoc: reading: /etc/asciidoc/docbook45.conf asciidoc: reading: /etc/asciidoc/filters/code/code-filter.conf asciidoc: reading: /etc/asciidoc/filters/graphviz/graphviz-filter.conf asciidoc: reading: /etc/asciidoc/filters/music/music-filter.conf asciidoc: reading: /etc/asciidoc/filters/source/source-highlight-filter.conf asciidoc: reading: /etc/asciidoc/filters/latex/latex-filter.conf asciidoc: reading: /etc/asciidoc/lang-en.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf asciidoc: writing: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml asciidoc: osmo-gsm-tester-manual.adoc: line 5: evaluating: {include:/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual-docinfo.xml} asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 13: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc asciidoc: intro.adoc: line 55: evaluating: {counter2:target-number} asciidoc: intro.adoc: line 55: evaluating: {set2:target:osmo-gsm-tester-manual__1.png} asciidoc: intro.adoc: line 103: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 15: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc asciidoc: trial.adoc: line 14: evaluating: {counter2:target-number} asciidoc: trial.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__2.png} asciidoc: trial.adoc: line 25: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 17: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 19: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc asciidoc: resource_pool.adoc: line 14: evaluating: {counter2:target-number} asciidoc: resource_pool.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__3.png} asciidoc: resource_pool.adoc: line 79: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc asciidoc: resource_pool.adoc: line 111: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc asciidoc: config_mangling.adoc: line 165: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1217-g47d5" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Sep-08" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: config_mangling.adoc: line 255: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1217-g47d5" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Sep-08" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: config_mangling.adoc: line 330: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1217-g47d5" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Sep-08" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 21: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 23: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 25: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 27: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 29: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 31: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 33: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc a2x: executing: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0 -V "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml" Build the book set list... xsltproc -o doclist.txt --xinclude --xincludestyle doclist.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml Build the listings... xsltproc -o listings.xml --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' /usr/share/dblatex/xsl/common/mklistings.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml xsltproc -o osmo-gsm-tester-manual.rtex --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' --param listings.xml '/tmp/tmpbSCIbX/listings.xml' /tmp/tmpbSCIbX/custom.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml XSLT stylesheets DocBook - LaTeX 2e (0.3.10) =================================================== WARNING: xref to <simpara id="scenario_suite_params"> has no generated text. Trying its ancestor elements. Image 'dblatex' not found Cannot convert 'osmo-gsm-tester-manual__1.png' to pdf Cannot convert 'osmo-gsm-tester-manual__2.png' to pdf Cannot convert 'osmo-gsm-tester-manual__3.png' to pdf Build osmo-gsm-tester-manual.pdf built-in module pdftex registered no support found for ifthen no support found for ifxetex no support found for fontspec no support found for xltxtra no support found for fontenc no support found for inputenc no support found for fancybox built-in module makeidx registered no support found for custom-dblatex building additional files... checking if compiling is necessary... the output file doesn't exist pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex running post-compilation scripts... [index] the index file /tmp/tmpbSCIbX/osmo-gsm-tester-manual.idx is empty osmo-gsm-tester-manual.toc MD5 checksum changed osmo-gsm-tester-manual.aux MD5 checksum changed the /tmp/tmpbSCIbX/osmo-gsm-tester-manual.toc file has changed pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex running post-compilation scripts... [index] the index file /tmp/tmpbSCIbX/osmo-gsm-tester-manual.idx is empty no new compilation is needed running last-compilation scripts... 'osmo-gsm-tester-manual.pdf' successfully built a2x: deleting /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml + make check OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env) if [ -n "$ASCIIDOC_WARNINGS_CHECK" ]; then \ make osmo-gsm-tester-manual.check; \ fi + [ = 1 ] + rm -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + cd /home/jenkins/workspace/osmo-gsm-tester_gerrit + PATH=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games osmo-gsm-tester/contrib/jenkins-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit + dirname -- osmo-gsm-tester/contrib/jenkins-run.sh + CDPATH= cd -- osmo-gsm-tester/contrib + pwd -P + SCRIPT_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib + OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf + date +%F %T + time_start=2022-09-08 12:00:17 + trial_dir_prefix=trial- + rm -rf trial-2066 + trial_dir=trial-2067 + mkdir -p trial-2067 + mv open5gs.build-8.tgz osmo-bsc.build-16895.tgz osmo-bts-oc2g.build-5192.tgz osmo-bts-sysmo.build-7696.tgz osmo-bts.build-7170.tgz osmo-ggsn.build-6368.tgz osmo-hlr.build-8266.tgz osmo-hnbgw.build-2029.tgz osmo-mgw.build-8474.tgz osmo-msc.build-28089.tgz osmo-nitb.build-12352.tgz osmo-pcu-oc2g.build-3692.tgz osmo-pcu-sysmo.build-4973.tgz osmo-pcu.build-4935.tgz osmo-sgsn.build-28698.tgz osmo-stp.build-8803.tgz osmo-trx.build-4363.tgz osmocom-bb.build-4444.tgz srslte.build-19.tgz trial-2067 + cat open5gs.build-8.md5 osmo-bsc.build-16895.md5 osmo-bts-oc2g.build-5192.md5 osmo-bts-sysmo.build-7696.md5 osmo-bts.build-7170.md5 osmo-ggsn.build-6368.md5 osmo-hlr.build-8266.md5 osmo-hnbgw.build-2029.md5 osmo-mgw.build-8474.md5 osmo-msc.build-28089.md5 osmo-nitb.build-12352.md5 osmo-pcu-oc2g.build-3692.md5 osmo-pcu-sysmo.build-4973.md5 osmo-pcu.build-4935.md5 osmo-sgsn.build-28698.md5 osmo-stp.build-8803.md5 osmo-trx.build-4363.md5 osmocom-bb.build-4444.md5 srslte.build-19.md5 + rm open5gs.build-8.md5 osmo-bsc.build-16895.md5 osmo-bts-oc2g.build-5192.md5 osmo-bts-sysmo.build-7696.md5 osmo-bts.build-7170.md5 osmo-ggsn.build-6368.md5 osmo-hlr.build-8266.md5 osmo-hnbgw.build-2029.md5 osmo-mgw.build-8474.md5 osmo-msc.build-28089.md5 osmo-nitb.build-12352.md5 osmo-pcu-oc2g.build-3692.md5 osmo-pcu-sysmo.build-4973.md5 osmo-pcu.build-4935.md5 osmo-sgsn.build-28698.md5 osmo-stp.build-8803.md5 osmo-trx.build-4363.md5 osmocom-bb.build-4444.md5 srslte.build-19.md5 + exit_code=1 + which osmo-gsm-tester.py + python3 -u /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo-gsm-tester.py -c /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf trial-2067 -s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo combinations: ['nitb_sms:sysmo', 'sms:sysmo', 'gprs:sysmo'] trial: 'trial-2067' tests: None -------------------------------------------------------------------------------------------- trial-2067 -------------------------------------------------------------------------------------------- 12:00:20.133670 tst trial-2067: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2067/run.2022-09-08_12-00-20/log --------------------------------------------------------------------- trial-2067 nitb_sms:sysmo --------------------------------------------------------------------- 12:00:23.074427 tst nitb_sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 12:00:23.079348 tst nitb_sms:sysmo: Verifying 1 x bts (candidates: 11) 12:00:23.100559 tst nitb_sms:sysmo: Verifying 1 x ip_address (candidates: 9) 12:00:23.107790 tst nitb_sms:sysmo: Verifying 2 x modem (candidates: 12) 12:00:23.165866 tst nitb_sms:sysmo: Reserving 1 x bts (candidates: 11) 12:00:23.187030 tst nitb_sms:sysmo: Reserving 1 x ip_address (candidates: 9) 12:00:23.194035 tst nitb_sms:sysmo: Reserving 2 x modem (candidates: 12) 12:00:23.726295 tst nitb_sms:sysmo: RESERVED RESOURCES for nitb_sms: bts sysmoBTS 1002 ip_address 10.42.42.2 modem sierra_1st sierra_2nd ---------------------------------------------- trial-2067 nitb_sms:sysmo mo_mt_sms.py ---------------------------------------------- 12:00:23.733611 tst nitb_sms:sysmo: Using 1 x ip_address (candidates: 1) 12:00:23.744576 tst nitb_sms:sysmo: Using 1 x bts (candidates: 1) 12:00:24.115992 tst mo_mt_sms.py: using LAC 3964 12:00:24.344617 tst mo_mt_sms.py: using RAC 139 12:00:24.455175 tst mo_mt_sms.py: using CellId 3964 12:00:24.566202 tst mo_mt_sms.py: using BVCI 3965 12:00:24.669867 tst nitb_sms:sysmo: Using 1 x modem (candidates: 2) 12:00:24.866265 tst nitb_sms:sysmo: Using 1 x modem (candidates: 2) 12:00:24.981591 tst mo_mt_sms.py:9: start nitb and bts... 12:00:25.177924 run osmo-nitb_10.42.42.2: Starting osmo-nitb 12:00:27.405014 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=20868): Launched 12:00:27.704839 run osmo-nitb_10.42.42.2(pid=20869): Launched 12:00:27.868128 run osmo-nitb_10.42.42.2(pid=20869): ERR: Terminated: ERROR {rc=2} [trial-2067↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=20869)] 12:00:27.990556 run osmo-nitb_10.42.42.2(pid=20869): stdout: | (launched: 2022-09-08_12:00:27.648943) 12:00:28.114997 run osmo-nitb_10.42.42.2(pid=20869): stderr: | (launched: 2022-09-08_12:00:27.693149) | ====================================================================== | ERROR: You should not run osmo-nitb. It is ancient, unmaintained and | obsolete for several years. If you really want to ignore years of hard work on | the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp), | you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line | argument. DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!! | Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box | ====================================================================== 12:00:28.239337 run osmo-nitb_10.42.42.2(pid=20869): stdout: | (launched: 2022-09-08_12:00:27.648943) 12:00:28.357573 run osmo-nitb_10.42.42.2(pid=20869): stderr: | (launched: 2022-09-08_12:00:27.693149) | ====================================================================== | ERROR: You should not run osmo-nitb. It is ancient, unmaintained and | obsolete for several years. If you really want to ignore years of hard work on | the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp), | you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line | argument. DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!! | Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box | ====================================================================== 12:00:28.499694 tst mo_mt_sms.py:11: ERR: Error: osmo-nitb_10.42.42.2(pid=20869): Process ended prematurely: osmo-nitb_10.42.42.2(pid=20869) [trial-2067↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=20869)] [trial-2067↪nitb_sms:sysmo↪mo_mt_sms.py:11] 12:00:28.508069 tst mo_mt_sms.py:11: Test FAILED (4.8 sec) 12:00:28.514528 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=20868): Terminating (SIGTERM) 12:00:28.723390 --- ParallelTerminationStrategy: PID 20868 died... 12:00:28.744547 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=20868): Terminated: ok {rc=0} 12:00:28.830559 bus /sierra_3: Setting Powered False 12:00:29.931717 bus /sierra_3: Setting Powered False 12:00:31.049677 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-2067 nitb_sms:sysmo FAIL --------------------------------------------------------------------- --------------------------------------------------------------------- trial-2067 sms:sysmo --------------------------------------------------------------------- 12:00:32.229255 tst sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 12:00:32.233313 tst sms:sysmo: Verifying 1 x bts (candidates: 11) 12:00:32.254321 tst sms:sysmo: Verifying 6 x ip_address (candidates: 9) 12:00:32.282458 tst sms:sysmo: Verifying 2 x modem (candidates: 12) 12:00:32.334422 tst sms:sysmo: Reserving 1 x bts (candidates: 11) 12:00:32.355512 tst sms:sysmo: Reserving 6 x ip_address (candidates: 9) 12:00:32.382426 tst sms:sysmo: Reserving 2 x modem (candidates: 12) 12:00:32.933279 tst sms:sysmo: RESERVED RESOURCES for sms: bts sysmoBTS 1002 ip_address 10.42.42.2 10.42.42.3 10.42.42.4 10.42.42.5 10.42.42.6 10.42.42.7 modem sierra_1st sierra_2nd ---------------------------------------------- trial-2067 sms:sysmo mo_mt_sms.py ---------------------------------------------- 12:00:32.941021 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 12:00:32.952601 tst sms:sysmo: Using 1 x bts (candidates: 1) 12:00:32.988410 tst mo_mt_sms.py: using LAC 3965 12:00:33.112241 tst mo_mt_sms.py: using RAC 140 12:00:33.234904 tst mo_mt_sms.py: using CellId 3965 12:00:33.362484 tst mo_mt_sms.py: using BVCI 3966 12:00:33.476919 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 12:00:33.488123 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 12:00:33.499305 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 12:00:33.510424 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 12:00:33.521790 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 12:00:33.533112 tst sms:sysmo: Using 1 x modem (candidates: 2) 12:00:33.646151 tst sms:sysmo: Using 1 x modem (candidates: 2) 12:00:33.762022 run osmo-hlr_10.42.42.2: Starting osmo-hlr 12:00:35.444933 run create_hlr_db(pid=20873): Launched 12:00:35.570396 bus /sierra_2: Setting Powered False 12:00:36.628892 run create_hlr_db(pid=20873): Terminated: ok {rc=0} 12:00:37.045135 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20875): Launched 12:00:37.344775 run osmo-hlr_10.42.42.2(pid=20876): Launched 12:00:37.457762 run osmo-stp_10.42.42.5: Starting osmo-stp 12:00:39.960131 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20878): Launched 12:00:40.259176 run osmo-stp_10.42.42.5(pid=20879): Launched 12:00:40.370818 run osmo-msc_10.42.42.6: Starting osmo-msc 12:00:45.632440 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=20880): Launched 12:00:45.931839 run osmo-msc_10.42.42.6(pid=20881): Launched 12:00:49.099628 run osmo-mgw_10.42.42.3: Starting osmo-mgw 12:00:50.790239 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=20882): Launched 12:00:51.089178 run osmo-mgw_10.42.42.3(pid=20883): Launched 12:00:51.206928 run osmo-mgw_10.42.42.4: Starting osmo-mgw 12:00:51.708250 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=20884): Launched 12:00:52.006051 run osmo-mgw_10.42.42.4(pid=20885): Launched 12:00:52.121059 run osmo-bsc_10.42.42.7: Starting osmo-bsc 12:00:56.653982 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=20886): Launched 12:00:56.956849 run osmo-bsc_10.42.42.7(pid=20887): Launched 12:01:03.234412 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 12:01:04.174172 run rm-remote-dir(pid=20888): Launched 12:01:06.362876 run rm-remote-dir(pid=20888): Terminated: ok {rc=0} 12:01:06.712430 run mk-remote-dir(pid=20889): Launched 12:01:07.897437 run mk-remote-dir(pid=20889): Terminated: ok {rc=0} 12:01:08.199168 run scp-inst-to-remote(pid=20890): Launched 12:01:20.403650 run scp-inst-to-remote(pid=20890): Terminated: ok {rc=0} 12:01:20.748416 run mk-remote-dir(pid=20892): Launched 12:01:21.931539 run mk-remote-dir(pid=20892): Terminated: ok {rc=0} 12:01:22.228389 run scp-cfg-to-remote(pid=20893): Launched 12:01:23.408987 run scp-cfg-to-remote(pid=20893): Terminated: ok {rc=0} 12:01:23.730024 run reload-dsp-firmware(pid=20895): Launched 12:01:25.900923 run reload-dsp-firmware(pid=20895): Terminated: ok {rc=0} 12:01:26.214159 run osmo-bts-sysmo(pid=20896): Launched 12:01:30.454283 tst mo_mt_sms.py: using MSISDN 3980 12:01:30.627870 bus /sierra_2: Setting Powered True 12:01:34.081969 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='3980', subscriber_id=1} 12:01:34.211951 tst mo_mt_sms.py: using MSISDN 3981 12:01:34.382028 bus /sierra_3: Setting Powered True 12:01:37.872662 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='3981', subscriber_id=2} 12:01:38.137095 bus /sierra_2: Setting Online False 12:01:39.316632 bus /sierra_2: Setting Powered False 12:01:45.068523 bus /sierra_2: Setting Powered True 12:01:46.425480 bus /sierra_2: Setting Online True 12:01:48.885941 tst /sierra_2: Connect to ('901', '70') 12:01:49.192718 bus /sierra_3: Setting Online False 12:01:50.370239 bus /sierra_3: Setting Powered False 12:01:56.066119 bus /sierra_3: Setting Powered True 12:01:57.404847 bus /sierra_3: Setting Online True 12:01:59.931457 tst /sierra_3: Connect to ('901', '70') 12:02:00.087697 tst /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053' 12:02:00.201386 tst /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028' 12:02:00.315116 tst mo_mt_sms.py:35: waiting for modems to attach... 12:02:04.134639 tst /sierra_2: Already registered with network ('901', '70') 12:02:15.684486 tst /sierra_3: Already registered with network ('901', '70') 12:02:20.292305 tst /sierra_2: sending sms to MSISDN 3981 {sms='message nr. 1, from /sierra_2, to /sierra_3, from 3980, to 3981'} 12:02:26.092618 tst /sierra_3: Incoming SMS: 'message nr. 1, from /sierra_2, to /sierra_3, from 3980, to 3981' 12:02:26.419092 tst /sierra_3: SMS received as expected: 'message nr. 1, from /sierra_2, to /sierra_3, from 3980, to 3981' 12:02:26.592534 tst mo_mt_sms.py: Test passed (113.7 sec) 12:02:26.660948 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20875): Terminating (SIGTERM) 12:02:26.703919 run osmo-hlr_10.42.42.2(pid=20876): Terminating (SIGTERM) 12:02:26.746818 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20878): Terminating (SIGTERM) 12:02:26.791175 run osmo-stp_10.42.42.5(pid=20879): Terminating (SIGTERM) 12:02:26.834885 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=20880): Terminating (SIGTERM) 12:02:26.899830 run osmo-msc_10.42.42.6(pid=20881): Terminating (SIGTERM) 12:02:26.944857 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=20882): Terminating (SIGTERM) 12:02:26.988195 run osmo-mgw_10.42.42.3(pid=20883): Terminating (SIGTERM) 12:02:27.032053 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=20884): Terminating (SIGTERM) 12:02:27.075912 run osmo-mgw_10.42.42.4(pid=20885): Terminating (SIGTERM) 12:02:27.120588 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=20886): Terminating (SIGTERM) 12:02:27.176847 run osmo-bsc_10.42.42.7(pid=20887): Terminating (SIGTERM) 12:02:27.221781 run osmo-bts-sysmo(pid=20896): Terminating (SIGTERM) 12:02:27.264900 --- ParallelTerminationStrategy: PID 20875 died... 12:02:27.285574 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20875): Terminated: ok {rc=0} 12:02:27.334204 --- ParallelTerminationStrategy: PID 20876 died... 12:02:27.354903 run osmo-hlr_10.42.42.2(pid=20876): Terminated: ok {rc=0} 12:02:27.403250 --- ParallelTerminationStrategy: PID 20878 died... 12:02:27.423758 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20878): Terminated: ok {rc=0} 12:02:27.472084 --- ParallelTerminationStrategy: PID 20879 died... 12:02:27.492755 run osmo-stp_10.42.42.5(pid=20879): Terminated: ok {rc=0} 12:02:27.542499 --- ParallelTerminationStrategy: PID 20880 died... 12:02:27.562353 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=20880): Terminated: ok {rc=0} 12:02:27.608906 --- ParallelTerminationStrategy: PID 20881 died... 12:02:27.628834 run osmo-msc_10.42.42.6(pid=20881): Terminated: ok {rc=0} 12:02:27.675349 --- ParallelTerminationStrategy: PID 20882 died... 12:02:27.695420 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=20882): Terminated: ok {rc=0} 12:02:27.741987 --- ParallelTerminationStrategy: PID 20883 died... 12:02:27.761648 run osmo-mgw_10.42.42.3(pid=20883): Terminated {rc=15} 12:02:27.808671 --- ParallelTerminationStrategy: PID 20884 died... 12:02:27.828607 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=20884): Terminated: ok {rc=0} 12:02:27.875184 --- ParallelTerminationStrategy: PID 20885 died... 12:02:27.895129 run osmo-mgw_10.42.42.4(pid=20885): Terminated {rc=15} 12:02:27.942340 --- ParallelTerminationStrategy: PID 20886 died... 12:02:27.962397 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=20886): Terminated: ok {rc=0} 12:02:28.008997 --- ParallelTerminationStrategy: PID 20887 died... 12:02:28.028822 run osmo-bsc_10.42.42.7(pid=20887): Terminated: ok {rc=0} 12:02:28.075446 --- ParallelTerminationStrategy: PID 20896 died... 12:02:28.095231 run osmo-bts-sysmo(pid=20896): Terminated: ok {rc=0} 12:02:28.223796 bus /sierra_3: Setting Online False 12:02:29.346425 bus /sierra_3: Setting Powered False 12:02:39.918369 bus /sierra_3: Setting Powered False 12:02:41.019261 bus /sierra_3: Setting Powered False 12:02:42.180240 bus /sierra_2: Setting Online False 12:02:43.300720 bus /sierra_2: Setting Powered False 12:02:49.918473 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-2067 sms:sysmo PASS --------------------------------------------------------------------- --------------------------------------------------------------------- trial-2067 gprs:sysmo --------------------------------------------------------------------- 12:02:51.194249 tst gprs:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 12:02:51.198665 tst gprs:sysmo: Verifying 1 x bts (candidates: 11) 12:02:51.219583 tst gprs:sysmo: Verifying 9 x ip_address (candidates: 9) 12:02:51.259540 tst gprs:sysmo: Verifying 4 x modem (candidates: 12) 12:02:51.346754 tst gprs:sysmo: Reserving 1 x bts (candidates: 11) 12:02:51.367429 tst gprs:sysmo: Reserving 9 x ip_address (candidates: 9) 12:02:51.406023 tst gprs:sysmo: Reserving 4 x modem (candidates: 12) 12:02:51.998394 tst gprs:sysmo: RESERVED RESOURCES for gprs: bts sysmoBTS 1002 ip_address 10.42.42.2 10.42.42.3 10.42.42.4 10.42.42.5 10.42.42.6 10.42.42.7 10.42.42.8 10.42.42.9 10.42.42.10 modem ec20_1 ec20_2 ec20_3 ec20_4 ---------------------------------------------- trial-2067 gprs:sysmo cs_paging_gprs_active.py ---------------------------------------------- 12:02:52.060781 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.072219 tst gprs:sysmo: Using 1 x bts (candidates: 1) 12:02:52.108764 tst cs_paging_gprs_active.py: using LAC 3966 12:02:52.235899 tst cs_paging_gprs_active.py: using RAC 141 12:02:52.363888 tst cs_paging_gprs_active.py: using CellId 3966 12:02:52.489043 tst cs_paging_gprs_active.py: using BVCI 3967 12:02:52.608165 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.619314 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.630311 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.641371 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.652469 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.663598 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.674773 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.685821 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:02:52.697183 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:02:52.817111 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:02:52.932591 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003... 12:02:53.136289 run iperf3-srv_10.42.42.10: Starting iperf3-srv 12:02:53.443978 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20907): Launched 12:02:53.714420 run iperf3-srv_10.42.42.10(pid=20908): Launched 12:02:53.828730 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004... 12:02:54.032050 run iperf3-srv_10.42.42.10: Starting iperf3-srv 12:02:54.338909 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20909): Launched 12:02:54.605350 run iperf3-srv_10.42.42.10(pid=20910): Launched 12:02:54.721261 tst cs_paging_gprs_active.py:38: start network... 12:02:54.924817 run osmo-hlr_10.42.42.2: Starting osmo-hlr 12:02:55.396621 run create_hlr_db(pid=20911): Launched 12:02:55.522482 bus /sierra_2: Setting Powered False 12:02:56.583404 run create_hlr_db(pid=20911): Terminated: ok {rc=0} 12:02:57.008659 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20913): Launched 12:02:57.315744 run osmo-hlr_10.42.42.2(pid=20914): Launched 12:02:57.433081 run osmo-stp_10.42.42.5: Starting osmo-stp 12:02:57.955697 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20916): Launched 12:02:58.261779 run osmo-stp_10.42.42.5(pid=20917): Launched 12:02:58.377138 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 12:02:59.899340 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20918): Launched 12:03:00.017075 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 12:03:00.275912 run patchelf(pid=20919): Launched 12:03:01.449008 run patchelf(pid=20919): Terminated: ok {rc=0} 12:03:01.577208 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 12:03:01.840758 run osmo-gsm-tester_setcap_net_admin.sh(pid=20920): Launched 12:03:03.015926 run osmo-gsm-tester_setcap_net_admin.sh(pid=20920): Terminated: ok {rc=0} 12:03:03.334966 run osmo-ggsn_10.42.42.6(pid=20923): Launched 12:03:03.451194 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 12:03:08.557115 run pcap-recorder_any(filters='host 10.42.42.7')(pid=20950): Launched 12:03:08.865641 run osmo-sgsn_10.42.42.7(pid=20951): Launched 12:03:08.982546 run osmo-msc_10.42.42.8: Starting osmo-msc 12:03:09.556532 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=20952): Launched 12:03:09.865947 run osmo-msc_10.42.42.8(pid=20953): Launched 12:03:13.036887 run osmo-mgw_10.42.42.3: Starting osmo-mgw 12:03:13.561102 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=20954): Launched 12:03:13.869316 run osmo-mgw_10.42.42.3(pid=20955): Launched 12:03:13.986431 run osmo-mgw_10.42.42.4: Starting osmo-mgw 12:03:14.509947 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=20956): Launched 12:03:14.819078 run osmo-mgw_10.42.42.4(pid=20957): Launched 12:03:14.935087 run osmo-bsc_10.42.42.9: Starting osmo-bsc 12:03:15.616011 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=20958): Launched 12:03:15.944948 run osmo-bsc_10.42.42.9(pid=20959): Launched 12:03:22.234170 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 12:03:22.803126 run rm-remote-dir(pid=20960): Launched 12:03:24.998327 run rm-remote-dir(pid=20960): Terminated: ok {rc=0} 12:03:25.356314 run mk-remote-dir(pid=20961): Launched 12:03:26.549079 run mk-remote-dir(pid=20961): Terminated: ok {rc=0} 12:03:26.853968 run scp-inst-to-remote(pid=20962): Launched 12:03:39.064081 run scp-inst-to-remote(pid=20962): Terminated: ok {rc=0} 12:03:39.418419 run mk-remote-dir(pid=20964): Launched 12:03:40.607799 run mk-remote-dir(pid=20964): Terminated: ok {rc=0} 12:03:40.910872 run scp-cfg-to-remote(pid=20965): Launched 12:03:42.095234 run scp-cfg-to-remote(pid=20965): Terminated: ok {rc=0} 12:03:42.423552 run reload-dsp-firmware(pid=20967): Launched 12:03:44.597999 run reload-dsp-firmware(pid=20967): Terminated: ok {rc=0} 12:03:44.921434 run osmo-bts-sysmo(pid=20968): Launched 12:03:49.152451 tst cs_paging_gprs_active.py:38: Waiting for bts to be ready... 12:03:51.239654 run rm-remote-dir(pid=20969): Launched 12:03:53.425424 run rm-remote-dir(pid=20969): Terminated: ok {rc=0} 12:03:53.769550 run mk-remote-dir(pid=20970): Launched 12:03:55.955658 run mk-remote-dir(pid=20970): Terminated: ok {rc=0} 12:03:56.250668 run scp-inst-to-bts(pid=20971): Launched 12:04:09.446020 run scp-inst-to-bts(pid=20971): Terminated: ok {rc=0} 12:04:09.768945 run mk-remote-run-dir(pid=20973): Launched 12:04:11.943899 run mk-remote-run-dir(pid=20973): Terminated: ok {rc=0} 12:04:12.220857 run scp-cfg-to-sysmobts(pid=20974): Launched 12:04:14.398515 run scp-cfg-to-sysmobts(pid=20974): Terminated: ok {rc=0} 12:04:14.742855 run osmo-pcu(pid=20976): Launched 12:04:14.878097 tst cs_paging_gprs_active.py: using MSISDN 3982 12:04:15.050981 bus /gobi_6: Setting Powered True 12:04:18.576200 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3982', subscriber_id=1} 12:04:18.857997 bus /gobi_6: Setting Online False 12:04:20.049747 bus /gobi_6: Setting Powered False 12:04:27.129938 bus /gobi_6: Setting Powered True 12:04:28.503190 bus /gobi_6: Setting Online True 12:04:30.945885 tst /gobi_6: Connect to ('901', '70') 12:04:31.210458 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 12:04:31.333193 tst cs_paging_gprs_active.py: using MSISDN 3983 12:04:31.557516 bus /gobi_0: Setting Powered True 12:04:35.046998 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3983', subscriber_id=2} 12:04:35.307351 bus /gobi_0: Setting Online False 12:04:36.494467 bus /gobi_0: Setting Powered False 12:04:43.262360 bus /gobi_0: Setting Powered True 12:04:44.667716 bus /gobi_0: Setting Online True 12:04:46.367655 tst /gobi_6: Already registered with network ('901', '70') 12:04:47.153610 tst /gobi_0: Connect to ('901', '70') 12:04:47.429629 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873' 12:04:47.545779 tst cs_paging_gprs_active.py:38: waiting for modems to attach... 12:05:01.793933 tst /gobi_0: Already registered with network ('901', '70') 12:05:07.153100 tst cs_paging_gprs_active.py:38: waiting for modems to attach to data services... 12:05:10.277329 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:05:10.394811 tst cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_6/context2'" 12:05:10.842396 run move_netns(pid=20985): Launched 12:05:12.038364 run move_netns(pid=20985): Terminated: ok {rc=0} 12:05:12.397287 run ifup(pid=20988): Launched 12:05:13.595901 run ifup(pid=20988): Terminated: ok {rc=0} 12:05:13.947936 run dhcp(pid=20991): Launched 12:05:15.136330 run dhcp(pid=20991): Terminated: ok {rc=0} 12:05:18.213760 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:05:18.331343 tst cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_0/context3'" 12:05:18.775720 run move_netns(pid=21009): Launched 12:05:19.954553 run move_netns(pid=21009): Terminated: ok {rc=0} 12:05:20.282109 run ifup(pid=21012): Launched 12:05:21.457171 run ifup(pid=21012): Terminated: ok {rc=0} 12:05:21.785131 run dhcp(pid=21015): Launched 12:05:22.962508 run dhcp(pid=21015): Terminated: ok {rc=0} 12:05:23.091414 tst cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'" 12:05:23.298500 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 12:05:23.662877 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031): Launched 12:05:23.845811 tst cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context3'" 12:05:24.068188 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s) 12:05:24.455665 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034): Launched 12:05:24.794813 run iperf3-cli_10.42.42.10(pid=21037): Launched 12:05:25.076693 run iperf3-cli_10.42.42.10(pid=21040): Launched 12:05:25.202338 tst cs_paging_gprs_active.py:13: waiting a few secs to make sure iperf3 test is running 12:05:27.454015 tst cs_paging_gprs_active.py:16: iperf3 running, let's place a call 12:05:32.297556 tst cs_paging_gprs_active.py:22: dial success 12:05:35.103549 tst cs_paging_gprs_active.py:28: answer success, call established and ongoing 12:05:41.707626 tst cs_paging_gprs_active.py:35: hangup success 12:06:34.068477 run iperf3-cli_10.42.42.10(pid=21037): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=21037)] 12:06:34.209645 run iperf3-cli_10.42.42.10(pid=21037): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 12:06:34.351512 run iperf3-cli_10.42.42.10(pid=21037): stderr: | (launched: 2022-09-08_12:05:24.778295) 12:06:35.548359 run iperf3-cli_10.42.42.10(pid=21040): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=21040)] 12:06:35.675657 run iperf3-cli_10.42.42.10(pid=21040): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 12:06:35.804547 run iperf3-cli_10.42.42.10(pid=21040): stderr: | (launched: 2022-09-08_12:05:25.061346) 12:06:35.932736 run iperf3-srv_10.42.42.10(pid=20908): Terminating (SIGINT) 12:06:36.262802 run iperf3-srv_10.42.42.10(pid=20908): Terminated {rc=1} 12:06:36.376438 tst cs_paging_gprs_active.py:38: Results for 10.42.42.10:5003 through "'/gobi_6/context2'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 47382, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Thu, 08 Sep 2022 10:05:26 GMT', 'timesecs': 1662631526}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'dlztxdpb62bd3uuyr5ktbmr5w4rqyjrhpna6', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0012640953063965, 'seconds': 1.0012640953063965, 'bytes': 101360, 'bits_per_second': 809856.2644971933, 'retransmits': 4, 'snd_cwnd': 15928, 'rtt': 1969880, 'rttvar': 3111235, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0012640953063965, 'seconds': 1.0012640953063965, 'bytes': 101360, 'bits_per_second': 809856.2644971933, 'retransmits': 4, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0012640953063965, 'end': 2.001099109649658, 'seconds': 0.9998350143432617, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1846438, 'rttvar': 1948827, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0012640953063965, 'end': 2.001099109649658, 'seconds': 0.9998350143432617, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.001099109649658, 'end': 3.0013179779052734, 'seconds': 1.0002188682556152, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 1960599, 'rttvar': 1308694, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.001099109649658, 'end': 3.0013179779052734, 'seconds': 1.0002188682556152, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0013179779052734, 'end': 4.001054048538208, 'seconds': 0.9997360706329346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2460540, 'rttvar': 1353507, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0013179779052734, 'end': 4.001054048538208, 'seconds': 0.9997360706329346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001054048538208, 'end': 5.001269102096558, 'seconds': 1.0002150535583496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 2952943, 'rttvar': 1630066, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001054048538208, 'end': 5.001269102096558, 'seconds': 1.0002150535583496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001269102096558, 'end': 6.001161098480225, 'seconds': 0.999891996383667, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 3358158, 'rttvar': 1633296, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001269102096558, 'end': 6.001161098480225, 'seconds': 0.999891996383667, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001161098480225, 'end': 7.001321077346802, 'seconds': 1.0001599788665771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 3976328, 'rttvar': 1641618, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001161098480225, 'end': 7.001321077346802, 'seconds': 1.0001599788665771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001321077346802, 'end': 8.00126600265503, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 4408929, 'rttvar': 1675705, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001321077346802, 'end': 8.00126600265503, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.00126600265503, 'end': 9.001164197921753, 'seconds': 0.9998981952667236, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 4866691, 'rttvar': 1738790, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.00126600265503, 'end': 9.001164197921753, 'seconds': 0.9998981952667236, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001164197921753, 'end': 10.001183986663818, 'seconds': 1.0000197887420654, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 5143199, 'rttvar': 1857108, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001164197921753, 'end': 10.001183986663818, 'seconds': 1.0000197887420654, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001183986663818, 'end': 11.00134015083313, 'seconds': 1.0001561641693115, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 5395028, 'rttvar': 1896489, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001183986663818, 'end': 11.00134015083313, 'seconds': 1.0001561641693115, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.00134015083313, 'end': 12.000159978866577, 'seconds': 0.9988198280334473, 'bytes': 204168, 'bits_per_second': 1635273.9044196312, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 5968752, 'rttvar': 2075250, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.00134015083313, 'end': 12.000159978866577, 'seconds': 0.9988198280334473, 'bytes': 204168, 'bits_per_second': 1635273.9044196312, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.000159978866577, 'end': 13.001100063323975, 'seconds': 1.0009400844573975, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 6282451, 'rttvar': 2183837, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.000159978866577, 'end': 13.001100063323975, 'seconds': 1.0009400844573975, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001100063323975, 'end': 14.001145124435425, 'seconds': 1.0000450611114502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 6607148, 'rttvar': 2287271, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001100063323975, 'end': 14.001145124435425, 'seconds': 1.0000450611114502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001145124435425, 'end': 15.001073122024536, 'seconds': 0.9999279975891113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 7460864, 'rttvar': 2274154, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001145124435425, 'end': 15.001073122024536, 'seconds': 0.9999279975891113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001073122024536, 'end': 16.001265048980713, 'seconds': 1.0001919269561768, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 8011395, 'rttvar': 2234269, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001073122024536, 'end': 16.001265048980713, 'seconds': 1.0001919269561768, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001265048980713, 'end': 17.001079082489014, 'seconds': 0.9998140335083008, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 8528767, 'rttvar': 2152136, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001265048980713, 'end': 17.001079082489014, 'seconds': 0.9998140335083008, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001079082489014, 'end': 18.001317977905273, 'seconds': 1.0002388954162598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 9028223, 'rttvar': 2077681, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001079082489014, 'end': 18.001317977905273, 'seconds': 1.0002388954162598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001317977905273, 'end': 19.001159191131592, 'seconds': 0.9998412132263184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 9773117, 'rttvar': 2023080, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001317977905273, 'end': 19.001159191131592, 'seconds': 0.9998412132263184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.001159191131592, 'end': 20.001160144805908, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 10232558, 'rttvar': 1948957, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.001159191131592, 'end': 20.001160144805908, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001160144805908, 'end': 21.001103162765503, 'seconds': 0.9999430179595947, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 10689966, 'rttvar': 1904520, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001160144805908, 'end': 21.001103162765503, 'seconds': 0.9999430179595947, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001103162765503, 'end': 22.001068115234375, 'seconds': 0.9999649524688721, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 11121829, 'rttvar': 1834493, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001103162765503, 'end': 22.001068115234375, 'seconds': 0.9999649524688721, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 12:06:36.586250 tst cs_paging_gprs_active.py:38: ERR: KeyError: 'sum_sent' [trial-2067↪gprs:sysmo↪cs_paging_gprs_active.py:38] 12:06:36.594049 tst cs_paging_gprs_active.py:38: Test FAILED (224.6 sec) 12:06:36.600336 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20907): Terminating (SIGTERM) 12:06:36.643028 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20909): Terminating (SIGTERM) 12:06:36.683781 run iperf3-srv_10.42.42.10(pid=20910): Terminating (SIGTERM) 12:06:36.724650 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20913): Terminating (SIGTERM) 12:06:36.765500 run osmo-hlr_10.42.42.2(pid=20914): Terminating (SIGTERM) 12:06:36.807047 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20916): Terminating (SIGTERM) 12:06:36.850537 run osmo-stp_10.42.42.5(pid=20917): Terminating (SIGTERM) 12:06:36.895024 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20918): Terminating (SIGTERM) 12:06:36.948975 run osmo-ggsn_10.42.42.6(pid=20923): Terminating (SIGTERM) 12:06:36.990473 run pcap-recorder_any(filters='host 10.42.42.7')(pid=20950): Terminating (SIGTERM) 12:06:37.039160 run osmo-sgsn_10.42.42.7(pid=20951): Terminating (SIGTERM) 12:06:37.085610 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=20952): Terminating (SIGTERM) 12:06:37.129236 run osmo-msc_10.42.42.8(pid=20953): Terminating (SIGTERM) 12:06:37.174475 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=20954): Terminating (SIGTERM) 12:06:37.218327 run osmo-mgw_10.42.42.3(pid=20955): Terminating (SIGTERM) 12:06:37.273043 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=20956): Terminating (SIGTERM) 12:06:37.327722 run osmo-mgw_10.42.42.4(pid=20957): Terminating (SIGTERM) 12:06:37.368605 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=20958): Terminating (SIGTERM) 12:06:37.409974 run osmo-bsc_10.42.42.9(pid=20959): Terminating (SIGTERM) 12:06:37.450628 run osmo-bts-sysmo(pid=20968): Terminating (SIGTERM) 12:06:37.494187 run osmo-pcu(pid=20976): Terminating (SIGTERM) 12:06:37.538406 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031): Terminating (SIGTERM) 12:06:37.678030 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031)-killSignals.SIGTERM(pid=21060): Launched 12:06:38.763542 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031)-killSignals.SIGTERM(pid=21060): Terminated: ok {rc=0} 12:06:38.834655 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034): Terminating (SIGTERM) 12:06:38.987437 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034)-killSignals.SIGTERM(pid=21065): Launched 12:06:40.096851 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034)-killSignals.SIGTERM(pid=21065): Terminated: ok {rc=0} 12:06:40.160225 --- ParallelTerminationStrategy: PID 20907 died... 12:06:40.179443 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20907): Terminated: ok {rc=0} 12:06:40.225134 --- ParallelTerminationStrategy: PID 20909 died... 12:06:40.244023 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20909): Terminated: ok {rc=0} 12:06:40.290090 --- ParallelTerminationStrategy: PID 20910 died... 12:06:40.308929 run iperf3-srv_10.42.42.10(pid=20910): Terminated {rc=256} 12:06:40.353537 --- ParallelTerminationStrategy: PID 20913 died... 12:06:40.372409 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20913): Terminated: ok {rc=0} 12:06:40.417221 --- ParallelTerminationStrategy: PID 20914 died... 12:06:40.436091 run osmo-hlr_10.42.42.2(pid=20914): Terminated: ok {rc=0} 12:06:40.480982 --- ParallelTerminationStrategy: PID 20916 died... 12:06:40.499858 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20916): Terminated: ok {rc=0} 12:06:40.544804 --- ParallelTerminationStrategy: PID 20917 died... 12:06:40.563699 run osmo-stp_10.42.42.5(pid=20917): Terminated: ok {rc=0} 12:06:40.608741 --- ParallelTerminationStrategy: PID 20918 died... 12:06:40.628243 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20918): Terminated: ok {rc=0} 12:06:40.673539 --- ParallelTerminationStrategy: PID 20923 died... 12:06:40.693043 run osmo-ggsn_10.42.42.6(pid=20923): Terminated: ok {rc=0} 12:06:40.738166 --- ParallelTerminationStrategy: PID 20950 died... 12:06:40.757787 run pcap-recorder_any(filters='host 10.42.42.7')(pid=20950): Terminated: ok {rc=0} 12:06:40.802645 --- ParallelTerminationStrategy: PID 20951 died... 12:06:40.821950 run osmo-sgsn_10.42.42.7(pid=20951): Terminated: ok {rc=0} 12:06:40.866917 --- ParallelTerminationStrategy: PID 20952 died... 12:06:40.885754 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=20952): Terminated: ok {rc=0} 12:06:40.930883 --- ParallelTerminationStrategy: PID 20953 died... 12:06:40.949876 run osmo-msc_10.42.42.8(pid=20953): Terminated: ok {rc=0} 12:06:40.995053 --- ParallelTerminationStrategy: PID 20954 died... 12:06:41.014281 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=20954): Terminated: ok {rc=0} 12:06:41.059524 --- ParallelTerminationStrategy: PID 20955 died... 12:06:41.078453 run osmo-mgw_10.42.42.3(pid=20955): Terminated {rc=15} 12:06:41.123430 --- ParallelTerminationStrategy: PID 20956 died... 12:06:41.142159 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=20956): Terminated: ok {rc=0} 12:06:41.187076 --- ParallelTerminationStrategy: PID 20957 died... 12:06:41.205726 run osmo-mgw_10.42.42.4(pid=20957): Terminated {rc=15} 12:06:41.250663 --- ParallelTerminationStrategy: PID 20958 died... 12:06:41.269929 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=20958): Terminated: ok {rc=0} 12:06:41.315131 --- ParallelTerminationStrategy: PID 20959 died... 12:06:41.334062 run osmo-bsc_10.42.42.9(pid=20959): Terminated: ok {rc=0} 12:06:41.379098 --- ParallelTerminationStrategy: PID 20968 died... 12:06:41.397944 run osmo-bts-sysmo(pid=20968): Terminated: ok {rc=0} 12:06:41.442969 --- ParallelTerminationStrategy: PID 20976 died... 12:06:41.461888 run osmo-pcu(pid=20976): Terminated: ok {rc=0} 12:06:47.329637 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031): Terminating (SIGINT) 12:06:47.467776 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031)-killSignals.SIGINT(pid=21068): Launched 12:06:48.550348 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031)-killSignals.SIGINT(pid=21068): Terminated: ok {rc=0} 12:06:48.613484 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034): Terminating (SIGINT) 12:06:48.749691 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034)-killSignals.SIGINT(pid=21071): Launched 12:06:49.833647 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034)-killSignals.SIGINT(pid=21071): Terminated: ok {rc=0} 12:06:55.719737 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031): Terminating (SIGKILL) 12:06:55.857615 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031)-killSignals.SIGUSR1(pid=21074): Launched 12:06:56.945193 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21031)-killSignals.SIGUSR1(pid=21074): Terminated: ok {rc=0} 12:06:57.008102 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034): Terminating (SIGKILL) 12:06:57.144337 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034)-killSignals.SIGUSR1(pid=21077): Launched 12:06:58.234046 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21034)-killSignals.SIGUSR1(pid=21077): Terminated: ok {rc=0} 12:06:58.525884 bus /gobi_0: Setting Online False 12:07:01.310117 bus /gobi_0: Setting Powered False 12:07:15.991656 bus /gobi_0: Setting Powered False 12:07:17.090273 bus /gobi_0: Setting Powered False 12:07:18.249186 bus /gobi_6: Setting Online False 12:07:19.389107 bus /gobi_6: Setting Powered False 12:07:27.939223 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2067 gprs:sysmo iperf3.py ---------------------------------------------- 12:07:29.145052 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.156939 tst gprs:sysmo: Using 1 x bts (candidates: 1) 12:07:29.200997 tst iperf3.py: using LAC 3967 12:07:29.338183 tst iperf3.py: using RAC 142 12:07:29.466834 tst iperf3.py: using CellId 3967 12:07:29.594001 tst iperf3.py: using BVCI 3968 12:07:29.711709 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.722968 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.733979 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.745158 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.756333 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.767516 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.778630 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.790439 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:07:29.801793 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:07:29.932960 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003... 12:07:30.154590 run iperf3-srv_10.42.42.10: Starting iperf3-srv 12:07:30.484787 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21080): Launched 12:07:30.772077 run iperf3-srv_10.42.42.10(pid=21081): Launched 12:07:30.894571 tst iperf3.py:8: start network... 12:07:31.113859 run osmo-hlr_10.42.42.2: Starting osmo-hlr 12:07:31.621700 run create_hlr_db(pid=21082): Launched 12:07:31.747457 bus /gobi_6: Setting Powered False 12:07:32.812492 run create_hlr_db(pid=21082): Terminated: ok {rc=0} 12:07:33.251183 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21084): Launched 12:07:33.561120 run osmo-hlr_10.42.42.2(pid=21085): Launched 12:07:33.681147 run osmo-stp_10.42.42.5: Starting osmo-stp 12:07:34.206610 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21086): Launched 12:07:34.514439 run osmo-stp_10.42.42.5(pid=21087): Launched 12:07:34.631784 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 12:07:35.157327 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21088): Launched 12:07:35.274313 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 12:07:35.533105 run patchelf(pid=21089): Launched 12:07:36.707624 run patchelf(pid=21089): Terminated: ok {rc=0} 12:07:36.836724 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 12:07:37.099343 run osmo-gsm-tester_setcap_net_admin.sh(pid=21090): Launched 12:07:38.279604 run osmo-gsm-tester_setcap_net_admin.sh(pid=21090): Terminated: ok {rc=0} 12:07:38.598441 run osmo-ggsn_10.42.42.6(pid=21093): Launched 12:07:38.714516 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 12:07:39.415359 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21112): Launched 12:07:39.746048 run osmo-sgsn_10.42.42.7(pid=21113): Launched 12:07:39.873079 run osmo-msc_10.42.42.8: Starting osmo-msc 12:07:40.468569 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21114): Launched 12:07:40.797882 run osmo-msc_10.42.42.8(pid=21115): Launched 12:07:43.981365 run osmo-mgw_10.42.42.3: Starting osmo-mgw 12:07:44.542415 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21116): Launched 12:07:44.872524 run osmo-mgw_10.42.42.3(pid=21117): Launched 12:07:44.999110 run osmo-mgw_10.42.42.4: Starting osmo-mgw 12:07:45.550956 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21118): Launched 12:07:45.880859 run osmo-mgw_10.42.42.4(pid=21119): Launched 12:07:46.007401 run osmo-bsc_10.42.42.9: Starting osmo-bsc 12:07:46.683856 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21120): Launched 12:07:47.014503 run osmo-bsc_10.42.42.9(pid=21121): Launched 12:07:53.303768 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 12:07:53.871738 run rm-remote-dir(pid=21122): Launched 12:07:56.064558 run rm-remote-dir(pid=21122): Terminated: ok {rc=0} 12:07:56.423578 run mk-remote-dir(pid=21123): Launched 12:07:57.613885 run mk-remote-dir(pid=21123): Terminated: ok {rc=0} 12:07:57.921083 run scp-inst-to-remote(pid=21124): Launched 12:08:10.128460 run scp-inst-to-remote(pid=21124): Terminated: ok {rc=0} 12:08:10.482547 run mk-remote-dir(pid=21126): Launched 12:08:11.670890 run mk-remote-dir(pid=21126): Terminated: ok {rc=0} 12:08:11.975778 run scp-cfg-to-remote(pid=21127): Launched 12:08:13.163088 run scp-cfg-to-remote(pid=21127): Terminated: ok {rc=0} 12:08:13.514168 run reload-dsp-firmware(pid=21129): Launched 12:08:15.699208 run reload-dsp-firmware(pid=21129): Terminated: ok {rc=0} 12:08:16.042370 run osmo-bts-sysmo(pid=21130): Launched 12:08:20.288868 tst iperf3.py:8: Waiting for bts to be ready... 12:08:21.965967 run rm-remote-dir(pid=21131): Launched 12:08:24.142230 run rm-remote-dir(pid=21131): Terminated: ok {rc=0} 12:08:24.486529 run mk-remote-dir(pid=21132): Launched 12:08:26.673076 run mk-remote-dir(pid=21132): Terminated: ok {rc=0} 12:08:26.968629 run scp-inst-to-bts(pid=21133): Launched 12:08:40.167166 run scp-inst-to-bts(pid=21133): Terminated: ok {rc=0} 12:08:40.512068 run mk-remote-run-dir(pid=21135): Launched 12:08:42.696171 run mk-remote-run-dir(pid=21135): Terminated: ok {rc=0} 12:08:42.992856 run scp-cfg-to-sysmobts(pid=21136): Launched 12:08:45.176615 run scp-cfg-to-sysmobts(pid=21136): Terminated: ok {rc=0} 12:08:45.521120 run osmo-pcu(pid=21138): Launched 12:08:45.655416 tst iperf3.py: using MSISDN 3984 12:08:45.830390 bus /gobi_6: Setting Powered True 12:08:49.342543 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3984', subscriber_id=1} 12:08:49.617898 bus /gobi_6: Setting Online False 12:08:50.812709 bus /gobi_6: Setting Powered False 12:08:58.111847 bus /gobi_6: Setting Powered True 12:08:59.497357 bus /gobi_6: Setting Online True 12:09:01.986314 tst /gobi_6: Connect to ('901', '70') 12:09:02.252695 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 12:09:02.361968 tst iperf3.py:8: waiting for modems to attach... 12:09:09.461099 tst /gobi_6: Already registered with network ('901', '70') 12:09:19.823079 tst iperf3.py:8: waiting for modems to attach to data services... 12:09:29.784299 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:09:29.901857 tst iperf3.py:8: Setting up data plan for "'/gobi_6/context3'" 12:09:30.346272 run move_netns(pid=21141): Launched 12:09:31.530182 run move_netns(pid=21141): Terminated: ok {rc=0} 12:09:31.865499 run ifup(pid=21144): Launched 12:09:33.041679 run ifup(pid=21144): Terminated: ok {rc=0} 12:09:33.372032 run dhcp(pid=21147): Launched 12:09:34.548463 run dhcp(pid=21147): Terminated: ok {rc=0} 12:09:34.677155 tst iperf3.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context3'" 12:09:34.887493 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 12:09:35.253249 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163): Launched 12:09:35.597025 run iperf3-cli_10.42.42.10(pid=21166): Launched 12:10:49.924311 run iperf3-cli_10.42.42.10(pid=21166): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=21166)] 12:10:50.055442 run iperf3-cli_10.42.42.10(pid=21166): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 12:10:50.185818 run iperf3-cli_10.42.42.10(pid=21166): stderr: | (launched: 2022-09-08_12:09:35.580850) 12:10:50.317839 run iperf3-srv_10.42.42.10(pid=21081): Terminating (SIGINT) 12:10:50.652455 run iperf3-srv_10.42.42.10(pid=21081): Terminated {rc=1} 12:10:50.770250 tst iperf3.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context3'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 47420, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Thu, 08 Sep 2022 10:09:42 GMT', 'timesecs': 1662631782}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': '6utatofvupnwo76ieefiao5ouwjplf72isn2', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0006608963012695, 'seconds': 1.0006608963012695, 'bytes': 101360, 'bits_per_second': 810344.4463526512, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 409720, 'rttvar': 175280, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0006608963012695, 'seconds': 1.0006608963012695, 'bytes': 101360, 'bits_per_second': 810344.4463526512, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0006608963012695, 'end': 2.0013020038604736, 'seconds': 1.000641107559204, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 501092, 'rttvar': 314205, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0006608963012695, 'end': 2.0013020038604736, 'seconds': 1.000641107559204, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0013020038604736, 'end': 3.0012640953063965, 'seconds': 0.9999620914459229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 18824, 'rtt': 888183, 'rttvar': 870927, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0013020038604736, 'end': 3.0012640953063965, 'seconds': 0.9999620914459229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0012640953063965, 'end': 4.001322031021118, 'seconds': 1.0000579357147217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1174652, 'rttvar': 1226134, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0012640953063965, 'end': 4.001322031021118, 'seconds': 1.0000579357147217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001322031021118, 'end': 5.001301050186157, 'seconds': 0.9999790191650391, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 1512946, 'rttvar': 1596188, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001322031021118, 'end': 5.001301050186157, 'seconds': 0.9999790191650391, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001301050186157, 'end': 6.001262903213501, 'seconds': 0.9999618530273438, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 2326721, 'rttvar': 2332369, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001301050186157, 'end': 6.001262903213501, 'seconds': 0.9999618530273438, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001262903213501, 'end': 7.001219987869263, 'seconds': 0.9999570846557617, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2783463, 'rttvar': 2662759, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001262903213501, 'end': 7.001219987869263, 'seconds': 0.9999570846557617, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001219987869263, 'end': 8.00131893157959, 'seconds': 1.0000989437103271, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 3783796, 'rttvar': 3255077, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001219987869263, 'end': 8.00131893157959, 'seconds': 1.0000989437103271, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.00131893157959, 'end': 9.001285076141357, 'seconds': 0.9999661445617676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 4216495, 'rttvar': 3306706, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.00131893157959, 'end': 9.001285076141357, 'seconds': 0.9999661445617676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001285076141357, 'end': 10.001281023025513, 'seconds': 0.9999959468841553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 4646970, 'rttvar': 3340978, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001285076141357, 'end': 10.001281023025513, 'seconds': 0.9999959468841553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001281023025513, 'end': 11.001210927963257, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 5116004, 'rttvar': 3443802, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001281023025513, 'end': 11.001210927963257, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001210927963257, 'end': 12.001169919967651, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 5616428, 'rttvar': 3583700, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001210927963257, 'end': 12.001169919967651, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001169919967651, 'end': 13.001281976699829, 'seconds': 1.0001120567321777, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 6523208, 'rttvar': 3611564, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001169919967651, 'end': 13.001281976699829, 'seconds': 1.0001120567321777, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001281976699829, 'end': 14.001282930374146, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 6937662, 'rttvar': 3537581, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001281976699829, 'end': 14.001282930374146, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001282930374146, 'end': 15.001264095306396, 'seconds': 0.999981164932251, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 7772336, 'rttvar': 3436666, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001282930374146, 'end': 15.001264095306396, 'seconds': 0.999981164932251, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001264095306396, 'end': 16.001312017440796, 'seconds': 1.0000479221343994, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 8203166, 'rttvar': 3439159, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001264095306396, 'end': 16.001312017440796, 'seconds': 1.0000479221343994, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001312017440796, 'end': 17.0012629032135, 'seconds': 0.9999508857727051, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9004020, 'rttvar': 3345450, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001312017440796, 'end': 17.0012629032135, 'seconds': 0.9999508857727051, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.0012629032135, 'end': 18.0012629032135, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9365691, 'rttvar': 3232430, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.0012629032135, 'end': 18.0012629032135, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.0012629032135, 'end': 19.001161098480225, 'seconds': 0.9998981952667236, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 9769849, 'rttvar': 3232638, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.0012629032135, 'end': 19.001161098480225, 'seconds': 0.9998981952667236, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.001161098480225, 'end': 20.001161098480225, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 10529202, 'rttvar': 3157956, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.001161098480225, 'end': 20.001161098480225, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001161098480225, 'end': 21.001210927963257, 'seconds': 1.0000498294830322, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 10885377, 'rttvar': 3080817, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001161098480225, 'end': 21.001210927963257, 'seconds': 1.0000498294830322, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001210927963257, 'end': 22.001219987869263, 'seconds': 1.0000090599060059, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11640844, 'rttvar': 3043094, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001210927963257, 'end': 22.001219987869263, 'seconds': 1.0000090599060059, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001219987869263, 'end': 23.0013210773468, 'seconds': 1.000101089477539, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 12047872, 'rttvar': 3096377, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001219987869263, 'end': 23.0013210773468, 'seconds': 1.000101089477539, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.0013210773468, 'end': 24.001322984695435, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 12428836, 'rttvar': 3084210, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.0013210773468, 'end': 24.001322984695435, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001322984695435, 'end': 25.001312017440796, 'seconds': 0.9999890327453613, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 12864918, 'rttvar': 3185321, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001322984695435, 'end': 25.001312017440796, 'seconds': 0.9999890327453613, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001312017440796, 'end': 26.00046396255493, 'seconds': 0.9991519451141357, 'bytes': 256296, 'bits_per_second': 2052108.3004705366, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 13655891, 'rttvar': 3185545, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001312017440796, 'end': 26.00046396255493, 'seconds': 0.9991519451141357, 'bytes': 256296, 'bits_per_second': 2052108.3004705366, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.00046396255493, 'end': 27.001264095306396, 'seconds': 1.0008001327514648, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 13933357, 'rttvar': 2944091, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.00046396255493, 'end': 27.001264095306396, 'seconds': 1.0008001327514648, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.001264095306396, 'end': 28.001178979873657, 'seconds': 0.9999148845672607, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14560238, 'rttvar': 2743411, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.001264095306396, 'end': 28.001178979873657, 'seconds': 0.9999148845672607, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001178979873657, 'end': 29.001301050186157, 'seconds': 1.0001220703125, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14904689, 'rttvar': 2746462, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001178979873657, 'end': 29.001301050186157, 'seconds': 1.0001220703125, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.001301050186157, 'end': 30.00116205215454, 'seconds': 0.9998610019683838, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 15211148, 'rttvar': 2672764, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.001301050186157, 'end': 30.00116205215454, 'seconds': 0.9998610019683838, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.00116205215454, 'end': 31.00128197669983, 'seconds': 1.000119924545288, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 15866390, 'rttvar': 2638554, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.00116205215454, 'end': 31.00128197669983, 'seconds': 1.000119924545288, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00128197669983, 'end': 32.00128102302551, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 16220574, 'rttvar': 2687283, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00128197669983, 'end': 32.00128102302551, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00128102302551, 'end': 33.0012629032135, 'seconds': 0.9999818801879883, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 16890940, 'rttvar': 2696187, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00128102302551, 'end': 33.0012629032135, 'seconds': 0.9999818801879883, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.0012629032135, 'end': 34.001319885253906, 'seconds': 1.0000569820404053, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17464856, 'rttvar': 2532612, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.0012629032135, 'end': 34.001319885253906, 'seconds': 1.0000569820404053, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.001319885253906, 'end': 35.001161098480225, 'seconds': 0.9998412132263184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 17734059, 'rttvar': 2437864, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.001319885253906, 'end': 35.001161098480225, 'seconds': 0.9998412132263184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.001161098480225, 'end': 36.001261949539185, 'seconds': 1.00010085105896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18334671, 'rttvar': 2412377, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.001161098480225, 'end': 36.001261949539185, 'seconds': 1.00010085105896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.001261949539185, 'end': 37.0013210773468, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18667629, 'rttvar': 2475197, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.001261949539185, 'end': 37.0013210773468, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.0013210773468, 'end': 38.001161098480225, 'seconds': 0.9998400211334229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19283110, 'rttvar': 2482498, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.0013210773468, 'end': 38.001161098480225, 'seconds': 0.9998400211334229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.001161098480225, 'end': 39.0012149810791, 'seconds': 1.000053882598877, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 19577101, 'rttvar': 2449854, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.001161098480225, 'end': 39.0012149810791, 'seconds': 1.000053882598877, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0012149810791, 'end': 40.001312017440796, 'seconds': 1.0000970363616943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 19839640, 'rttvar': 2362470, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0012149810791, 'end': 40.001312017440796, 'seconds': 1.0000970363616943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.001312017440796, 'end': 41.001319885253906, 'seconds': 1.0000078678131104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20357868, 'rttvar': 2250542, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.001312017440796, 'end': 41.001319885253906, 'seconds': 1.0000078678131104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.001319885253906, 'end': 42.001322984695435, 'seconds': 1.0000030994415283, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 70952, 'rtt': 20608083, 'rttvar': 2188336, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.001319885253906, 'end': 42.001322984695435, 'seconds': 1.0000030994415283, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.001322984695435, 'end': 43.00132203102112, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21187466, 'rttvar': 2235194, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.001322984695435, 'end': 43.00132203102112, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00132203102112, 'end': 44.0011510848999, 'seconds': 0.9998290538787842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21418832, 'rttvar': 2139127, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00132203102112, 'end': 44.0011510848999, 'seconds': 0.9998290538787842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.0011510848999, 'end': 45.00106501579285, 'seconds': 0.9999139308929443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 21957501, 'rttvar': 2135588, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.0011510848999, 'end': 45.00106501579285, 'seconds': 0.9999139308929443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00106501579285, 'end': 46.0012640953064, 'seconds': 1.0001990795135498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22172680, 'rttvar': 2032049, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00106501579285, 'end': 46.0012640953064, 'seconds': 1.0001990795135498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.0012640953064, 'end': 47.00126099586487, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22448688, 'rttvar': 2076054, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.0012640953064, 'end': 47.00126099586487, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.00126099586487, 'end': 48.00113797187805, 'seconds': 0.9998769760131836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 22862237, 'rttvar': 1877800, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.00126099586487, 'end': 48.00113797187805, 'seconds': 0.9998769760131836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.00113797187805, 'end': 49.00115394592285, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 78192, 'rtt': 23106857, 'rttvar': 1897590, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.00113797187805, 'end': 49.00115394592285, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.00115394592285, 'end': 50.001261949539185, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23480229, 'rttvar': 1714587, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.00115394592285, 'end': 50.001261949539185, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.001261949539185, 'end': 51.00108003616333, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23720331, 'rttvar': 1766145, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.001261949539185, 'end': 51.00108003616333, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.00108003616333, 'end': 52.001302003860474, 'seconds': 1.0002219676971436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24122848, 'rttvar': 1691161, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.00108003616333, 'end': 52.001302003860474, 'seconds': 1.0002219676971436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.001302003860474, 'end': 53.001070976257324, 'seconds': 0.9997689723968506, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24374904, 'rttvar': 1772482, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.001302003860474, 'end': 53.001070976257324, 'seconds': 0.9997689723968506, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.001070976257324, 'end': 54.00107288360596, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24590238, 'rttvar': 1760029, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.001070976257324, 'end': 54.00107288360596, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00107288360596, 'end': 55.00128197669983, 'seconds': 1.000209093093872, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 25023740, 'rttvar': 1765415, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00107288360596, 'end': 55.00128197669983, 'seconds': 1.000209093093872, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.00128197669983, 'end': 56.0013210773468, 'seconds': 1.0000391006469727, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 85432, 'rtt': 25243335, 'rttvar': 1763251, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.00128197669983, 'end': 56.0013210773468, 'seconds': 1.0000391006469727, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.0013210773468, 'end': 57.00132489204407, 'seconds': 1.0000038146972656, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25683687, 'rttvar': 1777874, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.0013210773468, 'end': 57.00132489204407, 'seconds': 1.0000038146972656, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.00132489204407, 'end': 58.001261949539185, 'seconds': 0.9999370574951172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25918239, 'rttvar': 1802509, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.00132489204407, 'end': 58.001261949539185, 'seconds': 0.9999370574951172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001261949539185, 'end': 59.001261949539185, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 88328, 'rtt': 26120669, 'rttvar': 1756742, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001261949539185, 'end': 59.001261949539185, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 12:10:50.993226 tst iperf3.py:8: ERR: KeyError: 'sum_sent' [trial-2067↪gprs:sysmo↪iperf3.py:8] 12:10:51.001208 tst iperf3.py:8: Test FAILED (201.9 sec) 12:10:51.007949 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21080): Terminating (SIGTERM) 12:10:51.050050 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21084): Terminating (SIGTERM) 12:10:51.092303 run osmo-hlr_10.42.42.2(pid=21085): Terminating (SIGTERM) 12:10:51.133849 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21086): Terminating (SIGTERM) 12:10:51.176774 run osmo-stp_10.42.42.5(pid=21087): Terminating (SIGTERM) 12:10:51.220406 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21088): Terminating (SIGTERM) 12:10:51.263463 run osmo-ggsn_10.42.42.6(pid=21093): Terminating (SIGTERM) 12:10:51.306208 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21112): Terminating (SIGTERM) 12:10:51.357186 run osmo-sgsn_10.42.42.7(pid=21113): Terminating (SIGTERM) 12:10:51.409497 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21114): Terminating (SIGTERM) 12:10:51.453713 run osmo-msc_10.42.42.8(pid=21115): Terminating (SIGTERM) 12:10:51.510645 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21116): Terminating (SIGTERM) 12:10:51.555035 run osmo-mgw_10.42.42.3(pid=21117): Terminating (SIGTERM) 12:10:51.599617 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21118): Terminating (SIGTERM) 12:10:51.642052 run osmo-mgw_10.42.42.4(pid=21119): Terminating (SIGTERM) 12:10:51.682982 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21120): Terminating (SIGTERM) 12:10:51.724086 run osmo-bsc_10.42.42.9(pid=21121): Terminating (SIGTERM) 12:10:51.765579 run osmo-bts-sysmo(pid=21130): Terminating (SIGTERM) 12:10:51.806125 run osmo-pcu(pid=21138): Terminating (SIGTERM) 12:10:51.847069 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163): Terminating (SIGTERM) 12:10:51.985626 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163)-killSignals.SIGTERM(pid=21189): Launched 12:10:53.066557 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163)-killSignals.SIGTERM(pid=21189): Terminated: ok {rc=0} 12:10:53.130249 --- ParallelTerminationStrategy: PID 21080 died... 12:10:53.149639 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21080): Terminated: ok {rc=0} 12:10:53.194511 --- ParallelTerminationStrategy: PID 21084 died... 12:10:53.213734 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21084): Terminated: ok {rc=0} 12:10:53.258364 --- ParallelTerminationStrategy: PID 21085 died... 12:10:53.277549 run osmo-hlr_10.42.42.2(pid=21085): Terminated: ok {rc=0} 12:10:53.322549 --- ParallelTerminationStrategy: PID 21086 died... 12:10:53.342035 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21086): Terminated: ok {rc=0} 12:10:53.386923 --- ParallelTerminationStrategy: PID 21087 died... 12:10:53.405754 run osmo-stp_10.42.42.5(pid=21087): Terminated: ok {rc=0} 12:10:53.450128 --- ParallelTerminationStrategy: PID 21088 died... 12:10:53.469159 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21088): Terminated: ok {rc=0} 12:10:53.513632 --- ParallelTerminationStrategy: PID 21093 died... 12:10:53.532584 run osmo-ggsn_10.42.42.6(pid=21093): Terminated: ok {rc=0} 12:10:53.577273 --- ParallelTerminationStrategy: PID 21112 died... 12:10:53.596432 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21112): Terminated: ok {rc=0} 12:10:53.641516 --- ParallelTerminationStrategy: PID 21113 died... 12:10:53.660431 run osmo-sgsn_10.42.42.7(pid=21113): Terminated: ok {rc=0} 12:10:53.705016 --- ParallelTerminationStrategy: PID 21114 died... 12:10:53.724187 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21114): Terminated: ok {rc=0} 12:10:53.768897 --- ParallelTerminationStrategy: PID 21115 died... 12:10:53.787839 run osmo-msc_10.42.42.8(pid=21115): Terminated: ok {rc=0} 12:10:53.832403 --- ParallelTerminationStrategy: PID 21116 died... 12:10:53.851434 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21116): Terminated: ok {rc=0} 12:10:53.896321 --- ParallelTerminationStrategy: PID 21117 died... 12:10:53.915307 run osmo-mgw_10.42.42.3(pid=21117): Terminated {rc=15} 12:10:53.960637 --- ParallelTerminationStrategy: PID 21118 died... 12:10:53.979801 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21118): Terminated: ok {rc=0} 12:10:54.024437 --- ParallelTerminationStrategy: PID 21119 died... 12:10:54.043456 run osmo-mgw_10.42.42.4(pid=21119): Terminated {rc=15} 12:10:54.088072 --- ParallelTerminationStrategy: PID 21120 died... 12:10:54.107141 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21120): Terminated: ok {rc=0} 12:10:54.151787 --- ParallelTerminationStrategy: PID 21121 died... 12:10:54.170940 run osmo-bsc_10.42.42.9(pid=21121): Terminated: ok {rc=0} 12:10:54.215853 --- ParallelTerminationStrategy: PID 21130 died... 12:10:54.235028 run osmo-bts-sysmo(pid=21130): Terminated: ok {rc=0} 12:10:54.279958 --- ParallelTerminationStrategy: PID 21138 died... 12:10:54.299435 run osmo-pcu(pid=21138): Terminated: ok {rc=0} 12:11:00.166869 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163): Terminating (SIGINT) 12:11:00.305838 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163)-killSignals.SIGINT(pid=21195): Launched 12:11:01.393543 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163)-killSignals.SIGINT(pid=21195): Terminated: ok {rc=0} 12:11:07.279070 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163): Terminating (SIGKILL) 12:11:07.417628 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163)-killSignals.SIGUSR1(pid=21199): Launched 12:11:08.508161 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21163)-killSignals.SIGUSR1(pid=21199): Terminated: ok {rc=0} 12:11:08.764530 bus /gobi_6: Setting Online False 12:11:11.787353 bus /gobi_6: Setting Powered False 12:11:27.183988 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2067 gprs:sysmo iperf3m4.py ---------------------------------------------- 12:11:28.384003 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.395446 tst gprs:sysmo: Using 1 x bts (candidates: 1) 12:11:28.431056 tst iperf3m4.py: using LAC 3968 12:11:28.550276 tst iperf3m4.py: using RAC 143 12:11:28.668751 tst iperf3m4.py: using CellId 3968 12:11:28.787890 tst iperf3m4.py: using BVCI 3969 12:11:28.900390 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.911149 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.921645 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.932208 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.942826 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.953241 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.963905 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.974434 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:11:28.985081 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:11:29.103531 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:11:29.224807 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:11:29.342410 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:11:29.459596 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003... 12:11:29.680058 run iperf3-srv_10.42.42.10: Starting iperf3-srv 12:11:30.009646 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21202): Launched 12:11:30.296529 run iperf3-srv_10.42.42.10(pid=21203): Launched 12:11:30.419486 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004... 12:11:30.639186 run iperf3-srv_10.42.42.10: Starting iperf3-srv 12:11:30.968116 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21204): Launched 12:11:31.253544 run iperf3-srv_10.42.42.10(pid=21205): Launched 12:11:31.377229 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005... 12:11:31.595585 run iperf3-srv_10.42.42.10: Starting iperf3-srv 12:11:31.923683 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21206): Launched 12:11:32.208897 run iperf3-srv_10.42.42.10(pid=21207): Launched 12:11:32.332208 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006... 12:11:32.550589 run iperf3-srv_10.42.42.10: Starting iperf3-srv 12:11:32.878653 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21208): Launched 12:11:33.164342 run iperf3-srv_10.42.42.10(pid=21209): Launched 12:11:33.285889 tst iperf3m4.py:8: start network... 12:11:33.503672 run osmo-hlr_10.42.42.2: Starting osmo-hlr 12:11:34.012087 run create_hlr_db(pid=21210): Launched 12:11:34.144639 bus /gobi_6: Setting Powered False 12:11:35.209213 run create_hlr_db(pid=21210): Terminated: ok {rc=0} 12:11:35.647541 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21212): Launched 12:11:35.955109 run osmo-hlr_10.42.42.2(pid=21213): Launched 12:11:36.072243 run osmo-stp_10.42.42.5: Starting osmo-stp 12:11:36.596938 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21214): Launched 12:11:36.905571 run osmo-stp_10.42.42.5(pid=21215): Launched 12:11:37.023602 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 12:11:37.546954 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21216): Launched 12:11:37.664764 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 12:11:37.923089 run patchelf(pid=21217): Launched 12:11:39.096651 run patchelf(pid=21217): Terminated: ok {rc=0} 12:11:39.224613 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 12:11:39.486000 run osmo-gsm-tester_setcap_net_admin.sh(pid=21218): Launched 12:11:40.666497 run osmo-gsm-tester_setcap_net_admin.sh(pid=21218): Terminated: ok {rc=0} 12:11:40.984569 run osmo-ggsn_10.42.42.6(pid=21221): Launched 12:11:41.106036 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 12:11:41.779554 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21238): Launched 12:11:42.087089 run osmo-sgsn_10.42.42.7(pid=21239): Launched 12:11:42.203454 run osmo-msc_10.42.42.8: Starting osmo-msc 12:11:42.761266 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21240): Launched 12:11:43.071582 run osmo-msc_10.42.42.8(pid=21241): Launched 12:11:46.242287 run osmo-mgw_10.42.42.3: Starting osmo-mgw 12:11:46.780984 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21242): Launched 12:11:47.089717 run osmo-mgw_10.42.42.3(pid=21243): Launched 12:11:47.205741 run osmo-mgw_10.42.42.4: Starting osmo-mgw 12:11:47.717901 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21244): Launched 12:11:48.024814 run osmo-mgw_10.42.42.4(pid=21245): Launched 12:11:48.142544 run osmo-bsc_10.42.42.9: Starting osmo-bsc 12:11:48.773949 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21246): Launched 12:11:49.082553 run osmo-bsc_10.42.42.9(pid=21247): Launched 12:11:55.353567 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 12:11:55.884557 run rm-remote-dir(pid=21248): Launched 12:11:58.068952 run rm-remote-dir(pid=21248): Terminated: ok {rc=0} 12:11:58.406187 run mk-remote-dir(pid=21249): Launched 12:11:59.584378 run mk-remote-dir(pid=21249): Terminated: ok {rc=0} 12:11:59.872330 run scp-inst-to-remote(pid=21250): Launched 12:12:12.069405 run scp-inst-to-remote(pid=21250): Terminated: ok {rc=0} 12:12:12.424698 run mk-remote-dir(pid=21252): Launched 12:12:13.613724 run mk-remote-dir(pid=21252): Terminated: ok {rc=0} 12:12:13.917714 run scp-cfg-to-remote(pid=21253): Launched 12:12:15.105122 run scp-cfg-to-remote(pid=21253): Terminated: ok {rc=0} 12:12:15.456355 run reload-dsp-firmware(pid=21255): Launched 12:12:17.645098 run reload-dsp-firmware(pid=21255): Terminated: ok {rc=0} 12:12:17.988437 run osmo-bts-sysmo(pid=21256): Launched 12:12:22.224269 tst iperf3m4.py:8: Waiting for bts to be ready... 12:12:23.945400 run rm-remote-dir(pid=21257): Launched 12:12:26.133495 run rm-remote-dir(pid=21257): Terminated: ok {rc=0} 12:12:26.478836 run mk-remote-dir(pid=21258): Launched 12:12:28.661419 run mk-remote-dir(pid=21258): Terminated: ok {rc=0} 12:12:28.957983 run scp-inst-to-bts(pid=21259): Launched 12:12:42.163024 run scp-inst-to-bts(pid=21259): Terminated: ok {rc=0} 12:12:42.486836 run mk-remote-run-dir(pid=21261): Launched 12:12:44.660848 run mk-remote-run-dir(pid=21261): Terminated: ok {rc=0} 12:12:44.939910 run scp-cfg-to-sysmobts(pid=21262): Launched 12:12:47.114560 run scp-cfg-to-sysmobts(pid=21262): Terminated: ok {rc=0} 12:12:47.438562 run osmo-pcu(pid=21264): Launched 12:12:47.563668 tst iperf3m4.py: using MSISDN 3985 12:12:47.739112 bus /gobi_6: Setting Powered True 12:12:51.248761 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3985', subscriber_id=1} 12:12:51.515085 bus /gobi_6: Setting Online False 12:12:52.701763 bus /gobi_6: Setting Powered False 12:13:00.111592 bus /gobi_6: Setting Powered True 12:13:01.468638 bus /gobi_6: Setting Online True 12:13:03.911468 tst /gobi_6: Connect to ('901', '70') 12:13:04.188571 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 12:13:04.310566 tst iperf3m4.py: using MSISDN 3986 12:13:04.586416 bus /gobi_0: Setting Powered True 12:13:08.046551 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3986', subscriber_id=2} 12:13:08.319244 bus /gobi_0: Setting Online False 12:13:09.513095 bus /gobi_0: Setting Powered False 12:13:16.291138 bus /gobi_0: Setting Powered True 12:13:17.664118 bus /gobi_0: Setting Online True 12:13:19.490609 tst /gobi_6: Already registered with network ('901', '70') 12:13:20.227886 tst /gobi_0: Connect to ('901', '70') 12:13:20.510456 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873' 12:13:20.634269 tst iperf3m4.py: using MSISDN 3987 12:13:20.865790 bus /gobi_1: Setting Powered True 12:13:24.374020 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3987', subscriber_id=3} 12:13:24.641941 bus /gobi_1: Setting Online False 12:13:25.828852 bus /gobi_1: Setting Powered False 12:13:33.125594 bus /gobi_1: Setting Powered True 12:13:34.494953 bus /gobi_1: Setting Online True 12:13:35.553379 tst /gobi_0: Already registered with network ('901', '70') 12:13:37.188906 tst /gobi_1: Connect to ('901', '70') 12:13:37.455900 tst /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024272103' 12:13:37.574543 tst iperf3m4.py: using MSISDN 3988 12:13:37.743031 bus /gobi_4: Setting Powered True 12:13:41.397692 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3988', subscriber_id=4} 12:13:41.670192 bus /gobi_4: Setting Online False 12:13:42.859108 bus /gobi_4: Setting Powered False 12:13:50.128454 bus /gobi_4: Setting Powered True 12:13:51.520078 bus /gobi_4: Setting Online True 12:13:54.001494 tst /gobi_4: Connect to ('901', '70') 12:13:54.288455 tst /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G 1 [Jul 21 2016 10:00:50]', Serial: '868323025161511' 12:13:54.405304 tst iperf3m4.py:8: waiting for modems to attach... 12:13:55.046687 tst /gobi_1: Already registered with network ('901', '70') 12:14:09.960730 tst /gobi_4: Already registered with network ('901', '70') 12:14:15.035163 tst iperf3m4.py:8: waiting for modems to attach to data services... 12:14:18.146259 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:14:18.264777 tst iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'" 12:14:18.711233 run move_netns(pid=21268): Launched 12:14:19.903227 run move_netns(pid=21268): Terminated: ok {rc=0} 12:14:20.255338 run ifup(pid=21271): Launched 12:14:21.447544 run ifup(pid=21271): Terminated: ok {rc=0} 12:14:21.777096 run dhcp(pid=21274): Launched 12:14:22.970070 run dhcp(pid=21274): Terminated: ok {rc=0} 12:14:26.121945 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:14:26.233462 tst iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'" 12:14:26.655066 run move_netns(pid=21292): Launched 12:14:27.848406 run move_netns(pid=21292): Terminated: ok {rc=0} 12:14:28.200391 run ifup(pid=21295): Launched 12:14:29.389588 run ifup(pid=21295): Terminated: ok {rc=0} 12:14:29.741867 run dhcp(pid=21298): Launched 12:14:30.932556 run dhcp(pid=21298): Terminated: ok {rc=0} 12:14:39.335976 tst /gobi_1: context activated {apn='inet46', path='/gobi_1/context12', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'} 12:14:39.454178 tst iperf3m4.py:8: Setting up data plan for "'/gobi_1/context12'" 12:14:39.897590 run move_netns(pid=21316): Launched 12:14:41.093505 run move_netns(pid=21316): Terminated: ok {rc=0} 12:14:41.444972 run ifup(pid=21319): Launched 12:14:42.640326 run ifup(pid=21319): Terminated: ok {rc=0} 12:14:42.992193 run dhcp(pid=21322): Launched 12:14:44.177569 run dhcp(pid=21322): Terminated: ok {rc=0} 12:14:47.303839 tst /gobi_4: context activated {apn='inet46', path='/gobi_4/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:14:47.421760 tst iperf3m4.py:8: Setting up data plan for "'/gobi_4/context2'" 12:14:47.866923 run move_netns(pid=21344): Launched 12:14:49.057395 run move_netns(pid=21344): Terminated: ok {rc=0} 12:14:49.409027 run ifup(pid=21347): Launched 12:14:50.591685 run ifup(pid=21347): Terminated: ok {rc=0} 12:14:50.921944 run dhcp(pid=21350): Launched 12:14:52.101072 run dhcp(pid=21350): Terminated: ok {rc=0} 12:14:52.230967 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'" 12:14:52.440242 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 12:14:52.804921 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366): Launched 12:14:52.988920 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'" 12:14:53.213137 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s) 12:14:53.602018 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369): Launched 12:14:53.783997 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context12'" 12:14:54.007789 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s) 12:14:54.397364 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372): Launched 12:14:54.579788 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context2'" 12:14:54.803333 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s) 12:14:55.191443 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375): Launched 12:14:55.532646 run iperf3-cli_10.42.42.10(pid=21378): Launched 12:14:55.817872 run iperf3-cli_10.42.42.10(pid=21381): Launched 12:14:56.104486 run iperf3-cli_10.42.42.10(pid=21384): Launched 12:14:56.394604 run iperf3-cli_10.42.42.10(pid=21387): Launched 12:16:19.737364 run iperf3-cli_10.42.42.10(pid=21378): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=21378)] 12:16:19.872153 run iperf3-cli_10.42.42.10(pid=21378): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 12:16:20.007173 run iperf3-cli_10.42.42.10(pid=21378): stderr: | (launched: 2022-09-08_12:14:55.515951) 12:16:21.197603 run iperf3-cli_10.42.42.10(pid=21381): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=21381)] 12:16:21.330423 run iperf3-cli_10.42.42.10(pid=21381): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 12:16:21.463686 run iperf3-cli_10.42.42.10(pid=21381): stderr: | (launched: 2022-09-08_12:14:55.801521) 12:16:22.657322 run iperf3-cli_10.42.42.10(pid=21384): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=21384)] 12:16:22.795647 run iperf3-cli_10.42.42.10(pid=21384): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 12:16:22.932723 run iperf3-cli_10.42.42.10(pid=21384): stderr: | (launched: 2022-09-08_12:14:56.087095) 12:16:24.130997 run iperf3-cli_10.42.42.10(pid=21387): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=21387)] 12:16:24.265284 run iperf3-cli_10.42.42.10(pid=21387): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 12:16:24.399499 run iperf3-cli_10.42.42.10(pid=21387): stderr: | (launched: 2022-09-08_12:14:56.377632) 12:16:24.533153 run iperf3-srv_10.42.42.10(pid=21203): Terminating (SIGINT) 12:16:24.870598 run iperf3-srv_10.42.42.10(pid=21203): Terminated {rc=1} 12:16:24.991261 tst iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 47468, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Thu, 08 Sep 2022 10:15:11 GMT', 'timesecs': 1662632111}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'nv347cizjtmgswwq3u3gartd2nu7dlx7uv27', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0007679462432861, 'seconds': 1.0007679462432861, 'bytes': 101360, 'bits_per_second': 810257.7655928195, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 507991, 'rttvar': 267499, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0007679462432861, 'seconds': 1.0007679462432861, 'bytes': 101360, 'bits_per_second': 810257.7655928195, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0007679462432861, 'end': 2.000662088394165, 'seconds': 0.9998941421508789, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 507991, 'rttvar': 267499, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0007679462432861, 'end': 2.000662088394165, 'seconds': 0.9998941421508789, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.000662088394165, 'end': 3.0012669563293457, 'seconds': 1.0006048679351807, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 692039, 'rttvar': 568720, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.000662088394165, 'end': 3.0012669563293457, 'seconds': 1.0006048679351807, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0012669563293457, 'end': 4.00131893157959, 'seconds': 1.0000519752502441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 17376, 'rtt': 1035284, 'rttvar': 1113030, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0012669563293457, 'end': 4.00131893157959, 'seconds': 1.0000519752502441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.00131893157959, 'end': 5.001167058944702, 'seconds': 0.9998481273651123, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 1035284, 'rttvar': 1113030, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.00131893157959, 'end': 5.001167058944702, 'seconds': 0.9998481273651123, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001167058944702, 'end': 6.001317977905273, 'seconds': 1.0001509189605713, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1530575, 'rttvar': 1825353, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001167058944702, 'end': 6.001317977905273, 'seconds': 1.0001509189605713, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001317977905273, 'end': 7.001166105270386, 'seconds': 0.9998481273651123, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 2151465, 'rttvar': 2610796, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001317977905273, 'end': 7.001166105270386, 'seconds': 0.9998481273651123, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001166105270386, 'end': 8.001166105270386, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 2151465, 'rttvar': 2610796, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001166105270386, 'end': 8.001166105270386, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001166105270386, 'end': 9.00116515159607, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2885095, 'rttvar': 3425357, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001166105270386, 'end': 9.00116515159607, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.00116515159607, 'end': 10.001085042953491, 'seconds': 0.9999198913574219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2885095, 'rttvar': 3425357, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.00116515159607, 'end': 10.001085042953491, 'seconds': 0.9999198913574219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001085042953491, 'end': 11.001328945159912, 'seconds': 1.000243902206421, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 3716796, 'rttvar': 4232419, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001085042953491, 'end': 11.001328945159912, 'seconds': 1.000243902206421, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001328945159912, 'end': 12.001320123672485, 'seconds': 0.9999911785125732, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4629685, 'rttvar': 5000092, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001328945159912, 'end': 12.001320123672485, 'seconds': 0.9999911785125732, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001320123672485, 'end': 13.001320123672485, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4629685, 'rttvar': 5000092, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001320123672485, 'end': 13.001320123672485, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001320123672485, 'end': 14.001325130462646, 'seconds': 1.0000050067901611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 5628060, 'rttvar': 5746819, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001320123672485, 'end': 14.001325130462646, 'seconds': 1.0000050067901611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001325130462646, 'end': 15.001151084899902, 'seconds': 0.9998259544372559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6691993, 'rttvar': 6437980, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001325130462646, 'end': 15.001151084899902, 'seconds': 0.9998259544372559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001151084899902, 'end': 16.001330137252808, 'seconds': 1.0001790523529053, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6691993, 'rttvar': 6437980, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001151084899902, 'end': 16.001330137252808, 'seconds': 1.0001790523529053, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001330137252808, 'end': 17.00131893157959, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7814996, 'rttvar': 7074492, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001330137252808, 'end': 17.00131893157959, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.00131893157959, 'end': 18.00114607810974, 'seconds': 0.9998271465301514, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8854569, 'rttvar': 7385014, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.00131893157959, 'end': 18.00114607810974, 'seconds': 0.9998271465301514, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.00114607810974, 'end': 19.001327991485596, 'seconds': 1.0001819133758545, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8854569, 'rttvar': 7385014, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.00114607810974, 'end': 19.001327991485596, 'seconds': 1.0001819133758545, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.001327991485596, 'end': 20.00126600265503, 'seconds': 0.9999380111694336, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9837713, 'rttvar': 7505050, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.001327991485596, 'end': 20.00126600265503, 'seconds': 0.9999380111694336, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.00126600265503, 'end': 21.001070976257324, 'seconds': 0.9998049736022949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9837713, 'rttvar': 7505050, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.00126600265503, 'end': 21.001070976257324, 'seconds': 0.9998049736022949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001070976257324, 'end': 22.001305103302002, 'seconds': 1.0002341270446777, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9837713, 'rttvar': 7505050, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001070976257324, 'end': 22.001305103302002, 'seconds': 1.0002341270446777, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001305103302002, 'end': 23.001161098480225, 'seconds': 0.9998559951782227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10895330, 'rttvar': 7744021, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001305103302002, 'end': 23.001161098480225, 'seconds': 0.9998559951782227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001161098480225, 'end': 24.00130605697632, 'seconds': 1.0001449584960938, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 12013358, 'rttvar': 8044070, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001161098480225, 'end': 24.00130605697632, 'seconds': 1.0001449584960938, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.00130605697632, 'end': 25.001070976257324, 'seconds': 0.9997649192810059, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 12013358, 'rttvar': 8044070, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.00130605697632, 'end': 25.001070976257324, 'seconds': 0.9997649192810059, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001070976257324, 'end': 26.00126600265503, 'seconds': 1.000195026397705, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12981903, 'rttvar': 7970143, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001070976257324, 'end': 26.00126600265503, 'seconds': 1.000195026397705, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.00126600265503, 'end': 27.00131893157959, 'seconds': 1.0000529289245605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 14019142, 'rttvar': 8052085, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.00126600265503, 'end': 27.00131893157959, 'seconds': 1.0000529289245605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00131893157959, 'end': 28.001266956329346, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 14019142, 'rttvar': 8052085, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00131893157959, 'end': 28.001266956329346, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001266956329346, 'end': 29.001308917999268, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14934241, 'rttvar': 7869263, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001266956329346, 'end': 29.001308917999268, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.001308917999268, 'end': 30.001138925552368, 'seconds': 0.9998300075531006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 15917279, 'rttvar': 7868024, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.001308917999268, 'end': 30.001138925552368, 'seconds': 0.9998300075531006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.001138925552368, 'end': 31.001156091690063, 'seconds': 1.0000171661376953, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 15917279, 'rttvar': 7868024, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.001138925552368, 'end': 31.001156091690063, 'seconds': 1.0000171661376953, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.001156091690063, 'end': 32.001073122024536, 'seconds': 0.9999170303344727, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16772137, 'rttvar': 7610734, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.001156091690063, 'end': 32.001073122024536, 'seconds': 0.9999170303344727, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.001073122024536, 'end': 33.00126600265503, 'seconds': 1.0001928806304932, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 17712816, 'rttvar': 7589407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.001073122024536, 'end': 33.00126600265503, 'seconds': 1.0001928806304932, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00126600265503, 'end': 34.0011420249939, 'seconds': 0.9998760223388672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 17712816, 'rttvar': 7589407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00126600265503, 'end': 34.0011420249939, 'seconds': 0.9998760223388672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.0011420249939, 'end': 35.001166105270386, 'seconds': 1.0000240802764893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18538249, 'rttvar': 7342922, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.0011420249939, 'end': 35.001166105270386, 'seconds': 1.0000240802764893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.001166105270386, 'end': 36.00111794471741, 'seconds': 0.9999518394470215, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 19453154, 'rttvar': 7337001, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.001166105270386, 'end': 36.00111794471741, 'seconds': 0.9999518394470215, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00111794471741, 'end': 37.00128698348999, 'seconds': 1.000169038772583, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 19453154, 'rttvar': 7337001, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00111794471741, 'end': 37.00128698348999, 'seconds': 1.000169038772583, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00128698348999, 'end': 38.00131702423096, 'seconds': 1.0000300407409668, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20250812, 'rttvar': 7098067, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00128698348999, 'end': 38.00131702423096, 'seconds': 1.0000300407409668, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00131702423096, 'end': 39.00122094154358, 'seconds': 0.9999039173126221, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 21136253, 'rttvar': 7094432, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00131702423096, 'end': 39.00122094154358, 'seconds': 0.9999039173126221, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.00122094154358, 'end': 40.00131893157959, 'seconds': 1.0000979900360107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 21136253, 'rttvar': 7094432, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.00122094154358, 'end': 40.00131893157959, 'seconds': 1.0000979900360107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00131893157959, 'end': 41.00126910209656, 'seconds': 0.9999501705169678, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21904109, 'rttvar': 6856536, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00131893157959, 'end': 41.00126910209656, 'seconds': 0.9999501705169678, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00126910209656, 'end': 42.001307010650635, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 22763479, 'rttvar': 6861143, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00126910209656, 'end': 42.001307010650635, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.001307010650635, 'end': 43.001307010650635, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 22763479, 'rttvar': 6861143, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.001307010650635, 'end': 43.001307010650635, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.001307010650635, 'end': 44.00131607055664, 'seconds': 1.0000090599060059, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23525223, 'rttvar': 6669344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.001307010650635, 'end': 44.00131607055664, 'seconds': 1.0000090599060059, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00131607055664, 'end': 45.00116801261902, 'seconds': 0.9998519420623779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23525223, 'rttvar': 6669344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00131607055664, 'end': 45.00116801261902, 'seconds': 0.9998519420623779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00116801261902, 'end': 46.00107002258301, 'seconds': 0.9999020099639893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 24386695, 'rttvar': 6724952, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00116801261902, 'end': 46.00107002258301, 'seconds': 0.9999020099639893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00107002258301, 'end': 47.00105810165405, 'seconds': 0.9999880790710449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25133094, 'rttvar': 6536512, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00107002258301, 'end': 47.00105810165405, 'seconds': 0.9999880790710449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.00105810165405, 'end': 48.001057147979736, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25133094, 'rttvar': 6536512, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.00105810165405, 'end': 48.001057147979736, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.001057147979736, 'end': 49.001057147979736, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25988586, 'rttvar': 6613368, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.001057147979736, 'end': 49.001057147979736, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.001057147979736, 'end': 50.001304149627686, 'seconds': 1.0002470016479492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 26764938, 'rttvar': 6512731, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.001057147979736, 'end': 50.001304149627686, 'seconds': 1.0002470016479492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.001304149627686, 'end': 51.00120306015015, 'seconds': 0.9998989105224609, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 26764938, 'rttvar': 6512731, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.001304149627686, 'end': 51.00120306015015, 'seconds': 0.9998989105224609, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.00120306015015, 'end': 52.00122809410095, 'seconds': 1.0000250339508057, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27661740, 'rttvar': 6678153, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.00120306015015, 'end': 52.00122809410095, 'seconds': 1.0000250339508057, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00122809410095, 'end': 53.00126814842224, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27661740, 'rttvar': 6678153, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00122809410095, 'end': 53.00126814842224, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00126814842224, 'end': 54.00116801261902, 'seconds': 0.9998998641967773, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 28448716, 'rttvar': 6582565, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00126814842224, 'end': 54.00116801261902, 'seconds': 0.9998998641967773, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00116801261902, 'end': 55.000683069229126, 'seconds': 0.9995150566101074, 'bytes': 256296, 'bits_per_second': 2051362.7948276233, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29321833, 'rttvar': 6683158, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00116801261902, 'end': 55.000683069229126, 'seconds': 0.9995150566101074, 'bytes': 256296, 'bits_per_second': 2051362.7948276233, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.000683069229126, 'end': 56.00126814842224, 'seconds': 1.0005850791931152, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29321833, 'rttvar': 6683158, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.000683069229126, 'end': 56.00126814842224, 'seconds': 1.0005850791931152, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.00126814842224, 'end': 57.0011670589447, 'seconds': 0.9998989105224609, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 29896515, 'rttvar': 6161733, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.00126814842224, 'end': 57.0011670589447, 'seconds': 0.9998989105224609, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.0011670589447, 'end': 58.00122404098511, 'seconds': 1.0000569820404053, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30594352, 'rttvar': 6016973, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.0011670589447, 'end': 58.00122404098511, 'seconds': 1.0000569820404053, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.00122404098511, 'end': 59.00116515159607, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30594352, 'rttvar': 6016973, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.00122404098511, 'end': 59.00116515159607, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 12:16:25.220774 tst iperf3m4.py:8: ERR: KeyError: 'sum_sent' [trial-2067↪gprs:sysmo↪iperf3m4.py:8] 12:16:25.228332 tst iperf3m4.py:8: Test FAILED (296.9 sec) 12:16:25.235379 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21202): Terminating (SIGTERM) 12:16:25.278066 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21204): Terminating (SIGTERM) 12:16:25.320404 run iperf3-srv_10.42.42.10(pid=21205): Terminating (SIGTERM) 12:16:25.362500 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21206): Terminating (SIGTERM) 12:16:25.405066 run iperf3-srv_10.42.42.10(pid=21207): Terminating (SIGTERM) 12:16:25.447263 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21208): Terminating (SIGTERM) 12:16:25.489519 run iperf3-srv_10.42.42.10(pid=21209): Terminating (SIGTERM) 12:16:25.531838 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21212): Terminating (SIGTERM) 12:16:25.574175 run osmo-hlr_10.42.42.2(pid=21213): Terminating (SIGTERM) 12:16:25.616500 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21214): Terminating (SIGTERM) 12:16:25.660002 run osmo-stp_10.42.42.5(pid=21215): Terminating (SIGTERM) 12:16:25.703559 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21216): Terminating (SIGTERM) 12:16:25.766511 run osmo-ggsn_10.42.42.6(pid=21221): Terminating (SIGTERM) 12:16:25.809609 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21238): Terminating (SIGTERM) 12:16:25.874115 run osmo-sgsn_10.42.42.7(pid=21239): Terminating (SIGTERM) 12:16:25.937925 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21240): Terminating (SIGTERM) 12:16:26.010230 run osmo-msc_10.42.42.8(pid=21241): Terminating (SIGTERM) 12:16:26.061108 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21242): Terminating (SIGTERM) 12:16:26.108416 run osmo-mgw_10.42.42.3(pid=21243): Terminating (SIGTERM) 12:16:26.152172 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21244): Terminating (SIGTERM) 12:16:26.196699 run osmo-mgw_10.42.42.4(pid=21245): Terminating (SIGTERM) 12:16:26.240199 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21246): Terminating (SIGTERM) 12:16:26.284120 run osmo-bsc_10.42.42.9(pid=21247): Terminating (SIGTERM) 12:16:26.330611 run osmo-bts-sysmo(pid=21256): Terminating (SIGTERM) 12:16:26.377397 run osmo-pcu(pid=21264): Terminating (SIGTERM) 12:16:26.421520 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366): Terminating (SIGTERM) 12:16:26.561361 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366)-killSignals.SIGTERM(pid=21422): Launched 12:16:27.654258 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366)-killSignals.SIGTERM(pid=21422): Terminated: ok {rc=0} 12:16:27.717981 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369): Terminating (SIGTERM) 12:16:27.855473 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369)-killSignals.SIGTERM(pid=21427): Launched 12:16:28.946489 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369)-killSignals.SIGTERM(pid=21427): Terminated: ok {rc=0} 12:16:29.013419 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372): Terminating (SIGTERM) 12:16:29.158021 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372)-killSignals.SIGTERM(pid=21430): Launched 12:16:30.249996 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372)-killSignals.SIGTERM(pid=21430): Terminated: ok {rc=0} 12:16:30.317052 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375): Terminating (SIGTERM) 12:16:30.462136 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375)-killSignals.SIGTERM(pid=21433): Launched 12:16:31.555542 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375)-killSignals.SIGTERM(pid=21433): Terminated: ok {rc=0} 12:16:31.622585 --- ParallelTerminationStrategy: PID 21202 died... 12:16:31.642845 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21202): Terminated: ok {rc=0} 12:16:31.690104 --- ParallelTerminationStrategy: PID 21204 died... 12:16:31.709853 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21204): Terminated: ok {rc=0} 12:16:31.756907 --- ParallelTerminationStrategy: PID 21205 died... 12:16:31.776632 run iperf3-srv_10.42.42.10(pid=21205): Terminated {rc=256} 12:16:31.823671 --- ParallelTerminationStrategy: PID 21206 died... 12:16:31.843334 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21206): Terminated: ok {rc=0} 12:16:31.890167 --- ParallelTerminationStrategy: PID 21207 died... 12:16:31.909939 run iperf3-srv_10.42.42.10(pid=21207): Terminated {rc=256} 12:16:31.957008 --- ParallelTerminationStrategy: PID 21208 died... 12:16:31.976776 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21208): Terminated: ok {rc=0} 12:16:32.023716 --- ParallelTerminationStrategy: PID 21209 died... 12:16:32.043412 run iperf3-srv_10.42.42.10(pid=21209): Terminated {rc=256} 12:16:32.090085 --- ParallelTerminationStrategy: PID 21212 died... 12:16:32.109920 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21212): Terminated: ok {rc=0} 12:16:32.156748 --- ParallelTerminationStrategy: PID 21213 died... 12:16:32.176656 run osmo-hlr_10.42.42.2(pid=21213): Terminated: ok {rc=0} 12:16:32.223775 --- ParallelTerminationStrategy: PID 21214 died... 12:16:32.243521 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21214): Terminated: ok {rc=0} 12:16:32.290343 --- ParallelTerminationStrategy: PID 21215 died... 12:16:32.310073 run osmo-stp_10.42.42.5(pid=21215): Terminated: ok {rc=0} 12:16:32.356992 --- ParallelTerminationStrategy: PID 21216 died... 12:16:32.376806 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21216): Terminated: ok {rc=0} 12:16:32.423956 --- ParallelTerminationStrategy: PID 21221 died... 12:16:32.443632 run osmo-ggsn_10.42.42.6(pid=21221): Terminated: ok {rc=0} 12:16:32.490672 --- ParallelTerminationStrategy: PID 21238 died... 12:16:32.510682 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21238): Terminated: ok {rc=0} 12:16:32.557675 --- ParallelTerminationStrategy: PID 21239 died... 12:16:32.577412 run osmo-sgsn_10.42.42.7(pid=21239): Terminated: ok {rc=0} 12:16:32.624607 --- ParallelTerminationStrategy: PID 21240 died... 12:16:32.644464 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21240): Terminated: ok {rc=0} 12:16:32.691656 --- ParallelTerminationStrategy: PID 21241 died... 12:16:32.711344 run osmo-msc_10.42.42.8(pid=21241): Terminated: ok {rc=0} 12:16:32.758271 --- ParallelTerminationStrategy: PID 21242 died... 12:16:32.778055 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21242): Terminated: ok {rc=0} 12:16:32.825129 --- ParallelTerminationStrategy: PID 21243 died... 12:16:32.845043 run osmo-mgw_10.42.42.3(pid=21243): Terminated {rc=15} 12:16:32.892114 --- ParallelTerminationStrategy: PID 21244 died... 12:16:32.911927 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21244): Terminated: ok {rc=0} 12:16:32.959034 --- ParallelTerminationStrategy: PID 21245 died... 12:16:32.979047 run osmo-mgw_10.42.42.4(pid=21245): Terminated {rc=15} 12:16:33.026348 --- ParallelTerminationStrategy: PID 21246 died... 12:16:33.046116 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21246): Terminated: ok {rc=0} 12:16:33.093579 --- ParallelTerminationStrategy: PID 21247 died... 12:16:33.113446 run osmo-bsc_10.42.42.9(pid=21247): Terminated: ok {rc=0} 12:16:33.160501 --- ParallelTerminationStrategy: PID 21256 died... 12:16:33.180373 run osmo-bts-sysmo(pid=21256): Terminated: ok {rc=0} 12:16:33.227416 --- ParallelTerminationStrategy: PID 21264 died... 12:16:33.247424 run osmo-pcu(pid=21264): Terminated: ok {rc=0} 12:16:39.117951 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366): Terminating (SIGINT) 12:16:39.264422 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366)-killSignals.SIGINT(pid=21436): Launched 12:16:40.351273 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366)-killSignals.SIGINT(pid=21436): Terminated: ok {rc=0} 12:16:40.418583 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369): Terminating (SIGINT) 12:16:40.563765 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369)-killSignals.SIGINT(pid=21439): Launched 12:16:41.655299 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369)-killSignals.SIGINT(pid=21439): Terminated: ok {rc=0} 12:16:41.722445 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372): Terminating (SIGINT) 12:16:41.867252 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372)-killSignals.SIGINT(pid=21442): Launched 12:16:42.957759 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372)-killSignals.SIGINT(pid=21442): Terminated: ok {rc=0} 12:16:43.024837 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375): Terminating (SIGINT) 12:16:43.169622 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375)-killSignals.SIGINT(pid=21445): Launched 12:16:44.255153 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375)-killSignals.SIGINT(pid=21445): Terminated: ok {rc=0} 12:16:50.141528 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366): Terminating (SIGKILL) 12:16:50.287454 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366)-killSignals.SIGUSR1(pid=21448): Launched 12:16:51.383032 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21366)-killSignals.SIGUSR1(pid=21448): Terminated: ok {rc=0} 12:16:51.447184 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369): Terminating (SIGKILL) 12:16:51.585330 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369)-killSignals.SIGUSR1(pid=21451): Launched 12:16:52.672745 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21369)-killSignals.SIGUSR1(pid=21451): Terminated: ok {rc=0} 12:16:52.737068 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372): Terminating (SIGKILL) 12:16:52.875360 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372)-killSignals.SIGUSR1(pid=21454): Launched 12:16:53.962014 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21372)-killSignals.SIGUSR1(pid=21454): Terminated: ok {rc=0} 12:16:54.025489 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375): Terminating (SIGKILL) 12:16:54.164474 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375)-killSignals.SIGUSR1(pid=21457): Launched 12:16:55.249989 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=21375)-killSignals.SIGUSR1(pid=21457): Terminated: ok {rc=0} 12:16:55.499633 bus /gobi_4: Setting Online False 12:16:56.640050 bus /gobi_4: Setting Powered False 12:17:06.942104 bus /gobi_4: Setting Powered False 12:17:08.041874 bus /gobi_4: Setting Powered False 12:17:09.200900 bus /gobi_1: Setting Online False 12:17:10.336516 bus /gobi_1: Setting Powered False 12:17:18.047015 bus /gobi_1: Setting Powered False 12:17:19.147116 bus /gobi_1: Setting Powered False 12:17:20.305538 bus /gobi_0: Setting Online False 12:17:21.438623 bus /gobi_0: Setting Powered False 12:17:29.941755 bus /gobi_0: Setting Powered False 12:17:31.040233 bus /gobi_0: Setting Powered False 12:17:32.198462 bus /gobi_6: Setting Online False 12:17:33.309492 bus /gobi_6: Setting Powered False 12:17:41.015174 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2067 gprs:sysmo ping.py ---------------------------------------------- 12:17:42.175169 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.188562 tst gprs:sysmo: Using 1 x bts (candidates: 1) 12:17:42.224575 tst ping.py: using LAC 3969 12:17:42.355536 tst ping.py: using RAC 144 12:17:42.479640 tst ping.py: using CellId 3969 12:17:42.606062 tst ping.py: using BVCI 3970 12:17:42.718632 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.729645 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.740554 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.752146 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.763633 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.774962 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.786423 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:17:42.797861 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:17:42.912641 tst ping.py:19: start network... 12:17:43.112735 run osmo-hlr_10.42.42.2: Starting osmo-hlr 12:17:43.577843 run create_hlr_db(pid=21463): Launched 12:17:43.701812 bus /gobi_6: Setting Powered False 12:17:44.762501 run create_hlr_db(pid=21463): Terminated: ok {rc=0} 12:17:45.204091 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21465): Launched 12:17:45.527140 run osmo-hlr_10.42.42.2(pid=21466): Launched 12:17:45.649907 run osmo-stp_10.42.42.5: Starting osmo-stp 12:17:46.200905 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21467): Launched 12:17:46.527772 run osmo-stp_10.42.42.5(pid=21468): Launched 12:17:46.651691 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 12:17:47.204898 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21469): Launched 12:17:47.328558 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 12:17:47.602118 run patchelf(pid=21470): Launched 12:17:48.784841 run patchelf(pid=21470): Terminated: ok {rc=0} 12:17:48.920129 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 12:17:49.196573 run osmo-gsm-tester_setcap_net_admin.sh(pid=21471): Launched 12:17:50.383840 run osmo-gsm-tester_setcap_net_admin.sh(pid=21471): Terminated: ok {rc=0} 12:17:50.719505 run osmo-ggsn_10.42.42.6(pid=21474): Launched 12:17:50.842016 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 12:17:51.523828 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21491): Launched 12:17:51.825238 run osmo-sgsn_10.42.42.7(pid=21492): Launched 12:17:51.942294 run osmo-msc_10.42.42.8: Starting osmo-msc 12:17:52.489744 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21493): Launched 12:17:52.790805 run osmo-msc_10.42.42.8(pid=21494): Launched 12:17:55.957393 run osmo-mgw_10.42.42.3: Starting osmo-mgw 12:17:56.472686 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21495): Launched 12:17:56.775521 run osmo-mgw_10.42.42.3(pid=21496): Launched 12:17:56.891001 run osmo-mgw_10.42.42.4: Starting osmo-mgw 12:17:57.396935 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21497): Launched 12:17:57.699785 run osmo-mgw_10.42.42.4(pid=21498): Launched 12:17:57.813684 run osmo-bsc_10.42.42.9: Starting osmo-bsc 12:17:58.438063 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21499): Launched 12:17:58.740624 run osmo-bsc_10.42.42.9(pid=21500): Launched 12:18:05.006140 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 12:18:05.529573 run rm-remote-dir(pid=21501): Launched 12:18:07.709326 run rm-remote-dir(pid=21501): Terminated: ok {rc=0} 12:18:08.040283 run mk-remote-dir(pid=21502): Launched 12:18:09.216145 run mk-remote-dir(pid=21502): Terminated: ok {rc=0} 12:18:09.497953 run scp-inst-to-remote(pid=21503): Launched 12:18:21.687106 run scp-inst-to-remote(pid=21503): Terminated: ok {rc=0} 12:18:22.015355 run mk-remote-dir(pid=21505): Launched 12:18:23.189353 run mk-remote-dir(pid=21505): Terminated: ok {rc=0} 12:18:23.469864 run scp-cfg-to-remote(pid=21506): Launched 12:18:24.645108 run scp-cfg-to-remote(pid=21506): Terminated: ok {rc=0} 12:18:24.967837 run reload-dsp-firmware(pid=21508): Launched 12:18:27.140748 run reload-dsp-firmware(pid=21508): Terminated: ok {rc=0} 12:18:27.457181 run osmo-bts-sysmo(pid=21513): Launched 12:18:31.679327 tst ping.py:31: Waiting for bts to be ready... 12:18:33.343106 run rm-remote-dir(pid=21514): Launched 12:18:35.513467 run rm-remote-dir(pid=21514): Terminated: ok {rc=0} 12:18:35.831138 run mk-remote-dir(pid=21515): Launched 12:18:38.005047 run mk-remote-dir(pid=21515): Terminated: ok {rc=0} 12:18:38.279568 run scp-inst-to-bts(pid=21516): Launched 12:18:51.464951 run scp-inst-to-bts(pid=21516): Terminated: ok {rc=0} 12:18:51.782334 run mk-remote-run-dir(pid=21518): Launched 12:18:53.952829 run mk-remote-run-dir(pid=21518): Terminated: ok {rc=0} 12:18:54.227777 run scp-cfg-to-sysmobts(pid=21519): Launched 12:18:56.401757 run scp-cfg-to-sysmobts(pid=21519): Terminated: ok {rc=0} 12:18:56.741898 run osmo-pcu(pid=21521): Launched 12:18:56.874309 tst ping.py: using MSISDN 3989 12:18:57.048643 bus /gobi_6: Setting Powered True 12:19:00.558056 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3989', subscriber_id=1} 12:19:00.838243 bus /gobi_6: Setting Online False 12:19:02.025788 bus /gobi_6: Setting Powered False 12:19:09.127061 bus /gobi_6: Setting Powered True 12:19:10.509718 bus /gobi_6: Setting Online True 12:19:12.944654 tst /gobi_6: Connect to ('901', '70') 12:19:13.204235 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 12:19:13.311976 tst ping.py:42: waiting for modems to attach... 12:19:28.055880 tst /gobi_6: Already registered with network ('901', '70') 12:19:33.760765 tst ping.py:46: waiting for modems to attach to data services... 12:19:36.860190 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:19:36.969779 tst ping.py:51: Setting up data plan for "'/gobi_6/context3'" 12:19:37.397437 run move_netns(pid=21525): Launched 12:19:38.577893 run move_netns(pid=21525): Terminated: ok {rc=0} 12:19:38.906380 run ifup(pid=21528): Launched 12:19:40.083722 run ifup(pid=21528): Terminated: ok {rc=0} 12:19:40.411116 run dhcp(pid=21531): Launched 12:19:41.584443 run dhcp(pid=21531): Terminated: ok {rc=0} 12:19:41.712943 tst ping.py:53: Running 10 ping requests for "'/gobi_6/context3'" 12:19:42.140861 run ping(pid=21547): Launched 12:19:52.344683 run ping(pid=21547): Terminated: ok {rc=0} 12:19:53.992670 tst /gobi_6: context deactivated {path='/gobi_6/context3'} 12:20:01.107624 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={Interface='r433306bbc2604b'}, Name='Internet', Password='', Protocol='ipv6', Settings={}, Type='internet', Username='ogt'}, user='ogt'} 12:20:08.648434 tst /gobi_6: context deactivated {path='/gobi_6/context3'} 12:20:08.756995 tst ping.py:70: (launched: 2022-09-08_12:19:42.075900) 21548: waiting for 21549 PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. 64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=614 ms 64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=532 ms 64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=509 ms 64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=492 ms 64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=511 ms 64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=507 ms 64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=488 ms 64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=484 ms 64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=485 ms 64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=462 ms --- 10.42.42.6 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 18ms rtt min/avg/max/mdev = 461.661/508.363/613.770/39.585 ms child exited with 0 12:20:08.960428 tst ping.py: Test passed (146.8 sec) 12:20:09.025434 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21465): Terminating (SIGTERM) 12:20:09.066241 run osmo-hlr_10.42.42.2(pid=21466): Terminating (SIGTERM) 12:20:09.106928 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21467): Terminating (SIGTERM) 12:20:09.150149 run osmo-stp_10.42.42.5(pid=21468): Terminating (SIGTERM) 12:20:09.194808 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21469): Terminating (SIGTERM) 12:20:09.238422 run osmo-ggsn_10.42.42.6(pid=21474): Terminating (SIGTERM) 12:20:09.281864 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21491): Terminating (SIGTERM) 12:20:09.329969 run osmo-sgsn_10.42.42.7(pid=21492): Terminating (SIGTERM) 12:20:09.393358 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21493): Terminating (SIGTERM) 12:20:09.439174 run osmo-msc_10.42.42.8(pid=21494): Terminating (SIGTERM) 12:20:09.483613 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21495): Terminating (SIGTERM) 12:20:09.528905 run osmo-mgw_10.42.42.3(pid=21496): Terminating (SIGTERM) 12:20:09.573021 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21497): Terminating (SIGTERM) 12:20:09.616872 run osmo-mgw_10.42.42.4(pid=21498): Terminating (SIGTERM) 12:20:09.659882 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21499): Terminating (SIGTERM) 12:20:09.703535 run osmo-bsc_10.42.42.9(pid=21500): Terminating (SIGTERM) 12:20:09.746415 run osmo-bts-sysmo(pid=21513): Terminating (SIGTERM) 12:20:09.789823 run osmo-pcu(pid=21521): Terminating (SIGTERM) 12:20:09.832742 --- ParallelTerminationStrategy: PID 21465 died... 12:20:09.852867 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21465): Terminated: ok {rc=0} 12:20:09.899802 --- ParallelTerminationStrategy: PID 21466 died... 12:20:09.919482 run osmo-hlr_10.42.42.2(pid=21466): Terminated: ok {rc=0} 12:20:09.966353 --- ParallelTerminationStrategy: PID 21467 died... 12:20:09.986952 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21467): Terminated: ok {rc=0} 12:20:10.065492 --- ParallelTerminationStrategy: PID 21468 died... 12:20:10.090348 run osmo-stp_10.42.42.5(pid=21468): Terminated: ok {rc=0} 12:20:10.138451 --- ParallelTerminationStrategy: PID 21469 died... 12:20:10.176019 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21469): Terminated: ok {rc=0} 12:20:10.263991 --- ParallelTerminationStrategy: PID 21474 died... 12:20:10.284387 run osmo-ggsn_10.42.42.6(pid=21474): Terminated: ok {rc=0} 12:20:10.331494 --- ParallelTerminationStrategy: PID 21491 died... 12:20:10.351215 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21491): Terminated: ok {rc=0} 12:20:10.398076 --- ParallelTerminationStrategy: PID 21493 died... 12:20:10.418798 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21493): Terminated: ok {rc=0} 12:20:10.466452 --- ParallelTerminationStrategy: PID 21494 died... 12:20:10.487058 run osmo-msc_10.42.42.8(pid=21494): Terminated: ok {rc=0} 12:20:10.534989 --- ParallelTerminationStrategy: PID 21495 died... 12:20:10.555441 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21495): Terminated: ok {rc=0} 12:20:10.603525 --- ParallelTerminationStrategy: PID 21496 died... 12:20:10.624093 run osmo-mgw_10.42.42.3(pid=21496): Terminated {rc=15} 12:20:10.671989 --- ParallelTerminationStrategy: PID 21497 died... 12:20:10.692551 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21497): Terminated: ok {rc=0} 12:20:10.742365 --- ParallelTerminationStrategy: PID 21498 died... 12:20:10.763287 run osmo-mgw_10.42.42.4(pid=21498): Terminated {rc=15} 12:20:10.812256 --- ParallelTerminationStrategy: PID 21499 died... 12:20:10.833178 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21499): Terminated: ok {rc=0} 12:20:10.882417 --- ParallelTerminationStrategy: PID 21500 died... 12:20:10.903488 run osmo-bsc_10.42.42.9(pid=21500): Terminated: ok {rc=0} 12:20:10.952975 --- ParallelTerminationStrategy: PID 21513 died... 12:20:10.973894 run osmo-bts-sysmo(pid=21513): Terminated: ok {rc=0} 12:20:11.023390 --- ParallelTerminationStrategy: PID 21521 died... 12:20:11.044427 run osmo-pcu(pid=21521): Terminated: ok {rc=0} 12:20:11.261940 --- ParallelTerminationStrategy: PID 21492 died... 12:20:11.283597 run osmo-sgsn_10.42.42.7(pid=21492): Terminated: ok {rc=0} 12:20:11.512050 bus /gobi_6: Setting Online False 12:20:14.548554 bus /gobi_6: Setting Powered False 12:20:28.938313 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2067 gprs:sysmo ping_idle_ping.py ---------------------------------------------- 12:20:30.081086 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.093109 tst gprs:sysmo: Using 1 x bts (candidates: 1) 12:20:30.129548 tst ping_idle_ping.py: using LAC 3970 12:20:30.257386 tst ping_idle_ping.py: using RAC 145 12:20:30.386056 tst ping_idle_ping.py: using CellId 3970 12:20:30.510484 tst ping_idle_ping.py: using BVCI 3971 12:20:30.627375 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.638433 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.649582 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.660686 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.671661 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.682861 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.694624 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 12:20:30.705681 tst gprs:sysmo: Using 1 x modem (candidates: 4) 12:20:30.823984 tst ping_idle_ping.py:24: start network... 12:20:31.041894 run osmo-hlr_10.42.42.2: Starting osmo-hlr 12:20:31.542239 run create_hlr_db(pid=21574): Launched 12:20:31.672453 bus /gobi_6: Setting Powered False 12:20:32.735954 run create_hlr_db(pid=21574): Terminated: ok {rc=0} 12:20:33.171991 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21576): Launched 12:20:33.476490 run osmo-hlr_10.42.42.2(pid=21577): Launched 12:20:33.592100 run osmo-stp_10.42.42.5: Starting osmo-stp 12:20:34.109534 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21578): Launched 12:20:34.414143 run osmo-stp_10.42.42.5(pid=21579): Launched 12:20:34.530054 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 12:20:35.048264 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21580): Launched 12:20:35.164846 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 12:20:35.421766 run patchelf(pid=21581): Launched 12:20:36.594015 run patchelf(pid=21581): Terminated: ok {rc=0} 12:20:36.720707 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 12:20:36.979622 run osmo-gsm-tester_setcap_net_admin.sh(pid=21582): Launched 12:20:38.158111 run osmo-gsm-tester_setcap_net_admin.sh(pid=21582): Terminated: ok {rc=0} 12:20:38.493213 run osmo-ggsn_10.42.42.6(pid=21585): Launched 12:20:38.617266 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 12:20:39.307287 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21607): Launched 12:20:39.626679 run osmo-sgsn_10.42.42.7(pid=21608): Launched 12:20:39.744538 run osmo-msc_10.42.42.8: Starting osmo-msc 12:20:40.300267 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21609): Launched 12:20:40.606792 run osmo-msc_10.42.42.8(pid=21610): Launched 12:20:43.779103 run osmo-mgw_10.42.42.3: Starting osmo-mgw 12:20:44.299546 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21611): Launched 12:20:44.603942 run osmo-mgw_10.42.42.3(pid=21612): Launched 12:20:44.721684 run osmo-mgw_10.42.42.4: Starting osmo-mgw 12:20:45.232688 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21613): Launched 12:20:45.537798 run osmo-mgw_10.42.42.4(pid=21614): Launched 12:20:45.655535 run osmo-bsc_10.42.42.9: Starting osmo-bsc 12:20:46.285500 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21615): Launched 12:20:46.592565 run osmo-bsc_10.42.42.9(pid=21616): Launched 12:20:52.865053 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 12:20:53.396222 run rm-remote-dir(pid=21617): Launched 12:20:55.579766 run rm-remote-dir(pid=21617): Terminated: ok {rc=0} 12:20:55.914142 run mk-remote-dir(pid=21618): Launched 12:20:57.094605 run mk-remote-dir(pid=21618): Terminated: ok {rc=0} 12:20:57.380853 run scp-inst-to-remote(pid=21619): Launched 12:21:09.574321 run scp-inst-to-remote(pid=21619): Terminated: ok {rc=0} 12:21:09.925369 run mk-remote-dir(pid=21621): Launched 12:21:11.113545 run mk-remote-dir(pid=21621): Terminated: ok {rc=0} 12:21:11.416498 run scp-cfg-to-remote(pid=21622): Launched 12:21:12.602596 run scp-cfg-to-remote(pid=21622): Terminated: ok {rc=0} 12:21:12.950388 run reload-dsp-firmware(pid=21624): Launched 12:21:15.138015 run reload-dsp-firmware(pid=21624): Terminated: ok {rc=0} 12:21:15.479266 run osmo-bts-sysmo(pid=21625): Launched 12:21:19.715134 tst ping_idle_ping.py:36: Waiting for bts to be ready... 12:21:21.431717 run rm-remote-dir(pid=21626): Launched 12:21:23.615812 run rm-remote-dir(pid=21626): Terminated: ok {rc=0} 12:21:23.957073 run mk-remote-dir(pid=21627): Launched 12:21:26.135534 run mk-remote-dir(pid=21627): Terminated: ok {rc=0} 12:21:26.412522 run scp-inst-to-bts(pid=21628): Launched 12:21:39.597815 run scp-inst-to-bts(pid=21628): Terminated: ok {rc=0} 12:21:39.918514 run mk-remote-run-dir(pid=21631): Launched 12:21:42.092276 run mk-remote-run-dir(pid=21631): Terminated: ok {rc=0} 12:21:42.368179 run scp-cfg-to-sysmobts(pid=21632): Launched 12:21:44.539311 run scp-cfg-to-sysmobts(pid=21632): Terminated: ok {rc=0} 12:21:44.859440 run osmo-pcu(pid=21634): Launched 12:21:44.980166 tst ping_idle_ping.py: using MSISDN 3990 12:21:45.148432 bus /gobi_6: Setting Powered True 12:21:48.669780 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3990', subscriber_id=1} 12:21:48.942361 bus /gobi_6: Setting Online False 12:21:50.128245 bus /gobi_6: Setting Powered False 12:21:57.110469 bus /gobi_6: Setting Powered True 12:21:58.507009 bus /gobi_6: Setting Online True 12:22:00.925174 tst /gobi_6: Connect to ('901', '70') 12:22:01.197183 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 12:22:01.310065 tst ping_idle_ping.py:47: waiting for modems to attach... 12:22:07.951012 tst /gobi_6: Already registered with network ('901', '70') 12:22:19.752410 tst ping_idle_ping.py:51: waiting for modems to attach to data services... 12:22:43.825873 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 12:22:43.942033 tst ping_idle_ping.py:56: Setting up data plane for "'/gobi_6/context2'" 12:22:44.387206 run move_netns(pid=21637): Launched 12:22:45.582421 run move_netns(pid=21637): Terminated: ok {rc=0} 12:22:45.933190 run ifup(pid=21640): Launched 12:22:47.128234 run ifup(pid=21640): Terminated: ok {rc=0} 12:22:47.479606 run dhcp(pid=21643): Launched 12:22:48.667342 run dhcp(pid=21643): Terminated: ok {rc=0} 12:22:48.796885 tst ping_idle_ping.py:60: [1] Running 10 ping requests for "'/gobi_6/context2'" 12:22:49.203355 run ping1(pid=21659): Launched 12:23:09.422140 run ping1(pid=21659): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61↪ping_idle_ping.py↪/gobi_6↪ping1(pid=21659)] 12:23:09.558426 run ping1(pid=21659): stdout: | (launched: 2022-09-08_12:22:49.140188) | 21660: waiting for 21661 | PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. | | --- 10.42.42.6 ping statistics --- | 10 packets transmitted, 0 received, 100% packet loss, time 204ms | | child exited with 1 12:23:09.696573 run ping1(pid=21659): stderr: | (launched: 2022-09-08_12:22:49.187567) 12:23:09.857273 tst ping_idle_ping.py:61: ERR: Error: ping1(pid=21659): launch_sync(): local process exited with status 1 [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61↪ping_idle_ping.py↪/gobi_6↪ping1(pid=21659)] [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61] 12:23:09.864608 tst ping_idle_ping.py:61: Test FAILED (159.8 sec) 12:23:09.871267 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21576): Terminating (SIGTERM) 12:23:09.913750 run osmo-hlr_10.42.42.2(pid=21577): Terminating (SIGTERM) 12:23:09.956274 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21578): Terminating (SIGTERM) 12:23:10.000115 run osmo-stp_10.42.42.5(pid=21579): Terminating (SIGTERM) 12:23:10.045318 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21580): Terminating (SIGTERM) 12:23:10.110501 run osmo-ggsn_10.42.42.6(pid=21585): Terminating (SIGTERM) 12:23:10.154432 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21607): Terminating (SIGTERM) 12:23:10.253769 run osmo-sgsn_10.42.42.7(pid=21608): Terminating (SIGTERM) 12:23:10.299032 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21609): Terminating (SIGTERM) 12:23:10.342627 run osmo-msc_10.42.42.8(pid=21610): Terminating (SIGTERM) 12:23:10.386385 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21611): Terminating (SIGTERM) 12:23:10.428230 run osmo-mgw_10.42.42.3(pid=21612): Terminating (SIGTERM) 12:23:10.468910 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21613): Terminating (SIGTERM) 12:23:10.510022 run osmo-mgw_10.42.42.4(pid=21614): Terminating (SIGTERM) 12:23:10.550498 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21615): Terminating (SIGTERM) 12:23:10.591930 run osmo-bsc_10.42.42.9(pid=21616): Terminating (SIGTERM) 12:23:10.632538 run osmo-bts-sysmo(pid=21625): Terminating (SIGTERM) 12:23:10.673239 run osmo-pcu(pid=21634): Terminating (SIGTERM) 12:23:10.714777 --- ParallelTerminationStrategy: PID 21576 died... 12:23:10.734379 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21576): Terminated: ok {rc=0} 12:23:10.780653 --- ParallelTerminationStrategy: PID 21577 died... 12:23:10.800847 run osmo-hlr_10.42.42.2(pid=21577): Terminated: ok {rc=0} 12:23:10.848725 --- ParallelTerminationStrategy: PID 21578 died... 12:23:10.868390 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21578): Terminated: ok {rc=0} 12:23:10.953170 --- ParallelTerminationStrategy: PID 21579 died... 12:23:11.007010 run osmo-stp_10.42.42.5(pid=21579): Terminated: ok {rc=0} 12:23:11.056155 --- ParallelTerminationStrategy: PID 21580 died... 12:23:11.076202 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21580): Terminated: ok {rc=0} 12:23:11.124568 --- ParallelTerminationStrategy: PID 21585 died... 12:23:11.144531 run osmo-ggsn_10.42.42.6(pid=21585): Terminated: ok {rc=0} 12:23:11.192785 --- ParallelTerminationStrategy: PID 21607 died... 12:23:11.212812 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21607): Terminated: ok {rc=0} 12:23:11.261160 --- ParallelTerminationStrategy: PID 21609 died... 12:23:11.281434 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21609): Terminated: ok {rc=0} 12:23:11.330462 --- ParallelTerminationStrategy: PID 21610 died... 12:23:11.351111 run osmo-msc_10.42.42.8(pid=21610): Terminated: ok {rc=0} 12:23:11.400254 --- ParallelTerminationStrategy: PID 21611 died... 12:23:11.420982 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21611): Terminated: ok {rc=0} 12:23:11.470480 --- ParallelTerminationStrategy: PID 21612 died... 12:23:11.491797 run osmo-mgw_10.42.42.3(pid=21612): Terminated {rc=15} 12:23:11.541150 --- ParallelTerminationStrategy: PID 21613 died... 12:23:11.562143 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21613): Terminated: ok {rc=0} 12:23:11.611798 --- ParallelTerminationStrategy: PID 21614 died... 12:23:11.632611 run osmo-mgw_10.42.42.4(pid=21614): Terminated {rc=15} 12:23:11.681993 --- ParallelTerminationStrategy: PID 21615 died... 12:23:11.703206 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21615): Terminated: ok {rc=0} 12:23:11.752869 --- ParallelTerminationStrategy: PID 21616 died... 12:23:11.773691 run osmo-bsc_10.42.42.9(pid=21616): Terminated: ok {rc=0} 12:23:11.823365 --- ParallelTerminationStrategy: PID 21625 died... 12:23:11.844207 run osmo-bts-sysmo(pid=21625): Terminated: ok {rc=0} 12:23:11.894032 --- ParallelTerminationStrategy: PID 21634 died... 12:23:11.915048 run osmo-pcu(pid=21634): Terminated: ok {rc=0} 12:23:12.132197 --- ParallelTerminationStrategy: PID 21608 died... 12:23:12.154472 run osmo-sgsn_10.42.42.7(pid=21608): Terminated: ok {rc=0} 12:23:12.379481 bus /gobi_6: Setting Online False 12:23:16.305442 bus /gobi_6: Setting Powered False 12:23:33.098188 bus /gobi_6: Setting Powered False --------------------------------------------------------------------- trial-2067 gprs:sysmo FAIL --------------------------------------------------------------------- 12:23:34.439390 tst trial-2067: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2067/run.2022-09-08_12-00-20/trial-2067.xml -------------------------------------------------------------------------------------------- trial-2067 FAIL -------------------------------------------------------------------------------------------- 12:23:34.658849 tst trial-2067: trial-2067: FAIL, 2 suites failed, 1 suites passed FAIL: nitb_sms:sysmo (fail: 1) FAIL: mo_mt_sms.py (4.8 sec) Error: osmo-nitb_10.42.42.2(pid=20869): Process ended prematurely: osmo-nitb_10.42.42.2(pid=20869) [trial-2067↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=20869)] FAIL: gprs:sysmo (fail: 4, pass: 1) FAIL: cs_paging_gprs_active.py (224.6 sec) KeyError: 'sum_sent' FAIL: iperf3.py (201.9 sec) KeyError: 'sum_sent' FAIL: iperf3m4.py (296.9 sec) KeyError: 'sum_sent' pass: ping.py (146.8 sec) FAIL: ping_idle_ping.py (159.8 sec) Error: ping1(pid=21659): launch_sync(): local process exited with status 1 [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61↪ping_idle_ping.py↪/gobi_6↪ping1(pid=21659)] PASS: sms:sysmo (pass: 1) pass: mo_mt_sms.py (113.7 sec) + rm -rf trial-2067/inst + cd trial-2067 + command -v journalctl + [ ! -z /bin/journalctl ] + readlink last_run + journalctl -u ofono -o short-precise --since 2022-09-08 12:00:17 + readlink last_run + tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2067-run.tgz run.2022-09-08_12-00-20 + tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2067-bin.tgz checksums.md5 open5gs.build-8.tgz osmo-bsc.build-16895.tgz osmo-bts-oc2g.build-5192.tgz osmo-bts-sysmo.build-7696.tgz osmo-bts.build-7170.tgz osmo-ggsn.build-6368.tgz osmo-hlr.build-8266.tgz osmo-hnbgw.build-2029.tgz osmo-mgw.build-8474.tgz osmo-msc.build-28089.tgz osmo-nitb.build-12352.tgz osmo-pcu-oc2g.build-3692.tgz osmo-pcu-sysmo.build-4973.tgz osmo-pcu.build-4935.tgz osmo-sgsn.build-28698.tgz osmo-stp.build-8803.tgz osmo-trx.build-4363.tgz osmocom-bb.build-4444.tgz srslte.build-19.tgz + exit 1 Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results [Checks API] No suitable checks publisher found. ERROR Gerrit response: ConflictFinished: FAILURE