![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
Failed
gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa.iperf3m4.py (from gprs_trx-lms-limenet-micro+mod-bts0-dynts-ipa)
Stacktrace
osmo-stp_10.42.42.5(pid=955): Process ended prematurely: osmo-stp_10.42.42.5(pid=955) [trial-2398↪gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=955)]
Standard Output
----------------------------------------------
trial-2398 gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa iperf3m4.py
----------------------------------------------
03:06:00.695002 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:00.709754 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x bts (candidates: 1)
03:06:00.886694 tst iperf3m4.py: using LAC 38113
03:06:01.016320 tst iperf3m4.py: using RAC 118
03:06:01.142602 tst iperf3m4.py: using CellId 38113
03:06:01.271359 tst iperf3m4.py: using BVCI 38114
03:06:01.297128 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.309595 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.322016 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.334298 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.347693 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.360988 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.375162 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.388340 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x ip_address (candidates: 9)
03:06:01.401794 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x modem (candidates: 4)
03:06:01.526737 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x modem (candidates: 4)
03:06:01.672504 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x modem (candidates: 4)
03:06:01.793587 tst gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa: Using 1 x modem (candidates: 4)
03:06:02.102282 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
03:06:02.239332 run iperf3-srv_10.42.42.10: Starting iperf3-srv
03:06:02.570198 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=940): Launched
03:06:02.845179 run iperf3-srv_10.42.42.10(pid=941): Launched
03:06:03.040904 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
03:06:03.174683 run iperf3-srv_10.42.42.10: Starting iperf3-srv
03:06:03.505426 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=942): Launched
03:06:03.780838 run iperf3-srv_10.42.42.10(pid=943): Launched
03:06:03.976833 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
03:06:04.110498 run iperf3-srv_10.42.42.10: Starting iperf3-srv
03:06:04.441126 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=944): Launched
03:06:04.716492 run iperf3-srv_10.42.42.10(pid=945): Launched
03:06:04.912643 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
03:06:05.045901 run iperf3-srv_10.42.42.10: Starting iperf3-srv
03:06:05.377356 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=946): Launched
03:06:05.652357 run iperf3-srv_10.42.42.10(pid=947): Launched
03:06:05.848281 tst iperf3m4.py:8: start network...
03:06:05.982145 run osmo-hlr_10.42.42.2: Starting osmo-hlr
03:06:06.487471 run create_hlr_db(pid=948): Launched
03:06:06.673275 bus /gobi_6: Setting Powered False
03:06:07.684238 run create_hlr_db(pid=948): Terminated: ok {rc=0}
03:06:08.114637 run pcap-recorder_any(filters='host 10.42.42.2')(pid=950): Launched
03:06:08.435785 run osmo-hlr_10.42.42.2(pid=951): Launched
03:06:08.557200 run osmo-stp_10.42.42.5: Starting osmo-stp
03:06:09.114945 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=954): Launched
03:06:09.436457 run osmo-stp_10.42.42.5(pid=955): Launched
03:06:09.554069 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
03:06:10.111673 run pcap-recorder_any(filters='host 10.42.42.6')(pid=958): Launched
03:06:10.226423 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
03:06:10.505311 run patchelf(pid=959): Launched
03:06:10.692366 run osmo-stp_10.42.42.5(pid=955): ERR: Terminated: ERROR {rc=1} [trial-2398↪gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=955)]
03:06:10.828934 run osmo-stp_10.42.42.5(pid=955): stdout:
| (launched: 2022-12-14_03:06:09.278548)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
03:06:10.965712 run osmo-stp_10.42.42.5(pid=955): stderr:
| �[0;m�[38;5;43m20221214030609471 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221214030609471 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221214030609471 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221214030609472 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221214030609472 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221214030609472 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221214030609472 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
03:06:11.098654 run osmo-stp_10.42.42.5(pid=955): stdout:
| (launched: 2022-12-14_03:06:09.278548)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
03:06:11.230666 run osmo-stp_10.42.42.5(pid=955): stderr:
| �[0;m�[38;5;43m20221214030609471 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221214030609471 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221214030609471 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221214030609472 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221214030609472 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221214030609472 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221214030609472 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
03:06:11.384396 run patchelf(pid=959): Terminating (SIGINT)
03:06:11.565276 run patchelf(pid=959): Terminated: ok {rc=0}
03:06:11.598433 tst iperf3m4.py:8: ERR: Error: osmo-stp_10.42.42.5(pid=955): Process ended prematurely: osmo-stp_10.42.42.5(pid=955) [trial-2398↪gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=955)] [trial-2398↪gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa↪iperf3m4.py:8]
03:06:11.608291 tst iperf3m4.py:8: Test FAILED (10.9 sec)
03:06:11.671951 run osmo-hlr_10.42.42.2(pid=951): ERR: Terminated: ERROR {rc=237} [trial-2398↪gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa↪iperf3m4.py↪iperf3m4.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=951)]
03:06:11.723591 run osmo-hlr_10.42.42.2(pid=951): stdout:
| (launched: 2022-12-14_03:06:08.278098)
03:06:11.775667 run osmo-hlr_10.42.42.2(pid=951): stderr:
| �[0;m�[1;31m20221214030608475 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221214030608475 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221214030608475 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221214030608476 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221214030608476 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221214030608476 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221214030608488 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2398/run.2022-12-14_02-07-46/gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa/iperf3m4.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221214030608497 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221214030608497 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221214030608497 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
03:06:11.820258 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=940): Terminating (SIGTERM)
03:06:11.862843 run iperf3-srv_10.42.42.10(pid=941): Terminating (SIGTERM)
03:06:11.905016 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=942): Terminating (SIGTERM)
03:06:11.947393 run iperf3-srv_10.42.42.10(pid=943): Terminating (SIGTERM)
03:06:11.989562 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=944): Terminating (SIGTERM)
03:06:12.031569 run iperf3-srv_10.42.42.10(pid=945): Terminating (SIGTERM)
03:06:12.074230 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=946): Terminating (SIGTERM)
03:06:12.116115 run iperf3-srv_10.42.42.10(pid=947): Terminating (SIGTERM)
03:06:12.158275 run pcap-recorder_any(filters='host 10.42.42.2')(pid=950): Terminating (SIGTERM)
03:06:12.200633 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=954): Terminating (SIGTERM)
03:06:12.242945 run pcap-recorder_any(filters='host 10.42.42.6')(pid=958): Terminating (SIGTERM)
03:06:12.252532 --- ParallelTerminationStrategy: PID 940 died...
03:06:12.309089 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=940): Terminated: ok {rc=0}
03:06:12.319053 --- ParallelTerminationStrategy: PID 941 died...
03:06:12.374972 run iperf3-srv_10.42.42.10(pid=941): Terminated {rc=256}
03:06:12.384808 --- ParallelTerminationStrategy: PID 942 died...
03:06:12.440529 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=942): Terminated: ok {rc=0}
03:06:12.450505 --- ParallelTerminationStrategy: PID 943 died...
03:06:12.506382 run iperf3-srv_10.42.42.10(pid=943): Terminated {rc=256}
03:06:12.516251 --- ParallelTerminationStrategy: PID 944 died...
03:06:12.572205 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=944): Terminated: ok {rc=0}
03:06:12.582189 --- ParallelTerminationStrategy: PID 945 died...
03:06:12.638232 run iperf3-srv_10.42.42.10(pid=945): Terminated {rc=256}
03:06:12.648213 --- ParallelTerminationStrategy: PID 946 died...
03:06:12.704280 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=946): Terminated: ok {rc=0}
03:06:12.714239 --- ParallelTerminationStrategy: PID 947 died...
03:06:12.770325 run iperf3-srv_10.42.42.10(pid=947): Terminated {rc=256}
03:06:12.780240 --- ParallelTerminationStrategy: PID 950 died...
03:06:12.836417 run pcap-recorder_any(filters='host 10.42.42.2')(pid=950): Terminated: ok {rc=0}
03:06:12.846421 --- ParallelTerminationStrategy: PID 954 died...
03:06:12.902587 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=954): Terminated: ok {rc=0}
03:06:12.912643 --- ParallelTerminationStrategy: PID 958 died...
03:06:12.968725 run pcap-recorder_any(filters='host 10.42.42.6')(pid=958): Terminated: ok {rc=0}
03:06:13.055135 bus /gobi_4: Setting Powered False
03:06:14.161115 bus /gobi_4: Setting Powered False
03:06:15.271681 bus /gobi_1: Setting Powered False
03:06:16.378696 bus /gobi_1: Setting Powered False
03:06:17.490585 bus /gobi_0: Setting Powered False
03:06:18.593186 bus /gobi_0: Setting Powered False
03:06:19.703393 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/iperf3m4.py", line 8, in <module>
setup_run_iperf3_test_parallel(4)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=955): Process ended prematurely: osmo-stp_10.42.42.5(pid=955) [trial-2398↪gprs:trx-lms-limenet-micro+mod-bts0-dynts-ipa↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=955)]