74gprs:sysmo11.0osmo-stp_10.42.42.5(pid=26879): Process ended prematurely: osmo-stp_10.42.42.5(pid=26879) [trial-2387↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26879)]2314cs_paging_gprs_active.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/cs_paging_gprs_active.py", line 38, in <module>
setup_run_iperf3_test_parallel(2, ready_cb=ready_cb_place_voicecall)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26879): Process ended prematurely: osmo-stp_10.42.42.5(pid=26879) [trial-2387↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26879)]
----------------------------------------------
trial-2387 gprs:sysmo cs_paging_gprs_active.py
----------------------------------------------
08:38:44.321882 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.332901 tst gprs:sysmo: Using 1 x bts (candidates: 1)
08:38:44.465071 tst cs_paging_gprs_active.py: using LAC 32762
08:38:44.595550 tst cs_paging_gprs_active.py: using RAC 122
08:38:44.725791 tst cs_paging_gprs_active.py: using CellId 32762
08:38:44.859530 tst cs_paging_gprs_active.py: using BVCI 32763
08:38:44.885211 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.896219 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.907204 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.918138 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.929082 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.940142 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.951155 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.962327 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:44.973396 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:38:45.097166 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:38:45.385314 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003...
08:38:45.517689 run iperf3-srv_10.42.42.10: Starting iperf3-srv
08:38:45.843844 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26598): Launched
08:38:46.162149 run iperf3-srv_10.42.42.10(pid=26599): Launched
08:38:46.364361 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004...
08:38:46.500470 run iperf3-srv_10.42.42.10: Starting iperf3-srv
08:38:46.909081 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26609): Launched
08:38:47.264373 run iperf3-srv_10.42.42.10(pid=26633): Launched
08:38:47.476868 tst cs_paging_gprs_active.py:38: start network...
08:38:47.623595 run osmo-hlr_10.42.42.2: Starting osmo-hlr
08:38:48.260715 run create_hlr_db(pid=26658): Launched
08:38:48.495281 bus /sierra_2: Setting Powered False
08:38:49.472594 run create_hlr_db(pid=26658): Terminated: ok {rc=0}
08:38:49.963721 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26873): Launched
08:38:50.391695 run osmo-hlr_10.42.42.2(pid=26874): Launched
08:38:50.630146 run osmo-stp_10.42.42.5: Starting osmo-stp
08:38:51.286925 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26877): Launched
08:38:51.720503 run osmo-stp_10.42.42.5(pid=26879): Launched
08:38:51.971380 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
08:38:53.605119 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26881): Launched
08:38:53.729157 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
08:38:54.055525 run patchelf(pid=26882): Launched
08:38:54.255908 run osmo-stp_10.42.42.5(pid=26879): ERR: Terminated: ERROR {rc=1} [trial-2387↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26879)]
08:38:54.402472 run osmo-stp_10.42.42.5(pid=26879): stdout:
| (launched: 2022-12-13_08:38:51.461089)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:38:54.549853 run osmo-stp_10.42.42.5(pid=26879): stderr:
| �[0;m�[38;5;43m20221213083851785 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083851785 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083851786 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083851786 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083851786 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083851786 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083851787 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:38:54.694209 run osmo-stp_10.42.42.5(pid=26879): stdout:
| (launched: 2022-12-13_08:38:51.461089)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:38:54.836657 run osmo-stp_10.42.42.5(pid=26879): stderr:
| �[0;m�[38;5;43m20221213083851785 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083851785 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083851786 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083851786 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083851786 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083851786 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083851787 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:38:55.000555 run patchelf(pid=26882): Terminating (SIGINT)
08:38:55.195040 run patchelf(pid=26882): Terminated: ok {rc=0}
08:38:55.232143 tst cs_paging_gprs_active.py:38: ERR: Error: osmo-stp_10.42.42.5(pid=26879): Process ended prematurely: osmo-stp_10.42.42.5(pid=26879) [trial-2387↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26879)] [trial-2387↪gprs:sysmo↪cs_paging_gprs_active.py:38]
08:38:55.243206 tst cs_paging_gprs_active.py:38: Test FAILED (11.0 sec)
08:38:55.309601 run osmo-hlr_10.42.42.2(pid=26874): ERR: Terminated: ERROR {rc=237} [trial-2387↪gprs:sysmo↪cs_paging_gprs_active.py↪cs_paging_gprs_active.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26874)]
08:38:55.364490 run osmo-hlr_10.42.42.2(pid=26874): stdout:
| (launched: 2022-12-13_08:38:50.127373)
08:38:55.419457 run osmo-hlr_10.42.42.2(pid=26874): stderr:
| �[0;m�[1;31m20221213083850472 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221213083850472 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221213083850472 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221213083850472 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221213083850472 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221213083850472 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221213083850490 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2387/run.2022-12-13_08-32-52/gprs:sysmo/cs_paging_gprs_active.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221213083850498 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221213083850498 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221213083850498 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
08:38:55.467344 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26598): Terminating (SIGTERM)
08:38:55.518166 run iperf3-srv_10.42.42.10(pid=26599): Terminating (SIGTERM)
08:38:55.564195 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26609): Terminating (SIGTERM)
08:38:55.610506 run iperf3-srv_10.42.42.10(pid=26633): Terminating (SIGTERM)
08:38:55.656739 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26873): Terminating (SIGTERM)
08:38:55.704498 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26877): Terminating (SIGTERM)
08:38:55.749521 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26881): Terminating (SIGTERM)
08:38:55.761747 --- ParallelTerminationStrategy: PID 26598 died...
08:38:55.821589 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26598): Terminated: ok {rc=0}
08:38:55.833670 --- ParallelTerminationStrategy: PID 26599 died...
08:38:55.891166 run iperf3-srv_10.42.42.10(pid=26599): Terminated {rc=256}
08:38:55.902202 --- ParallelTerminationStrategy: PID 26609 died...
08:38:55.960398 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26609): Terminated: ok {rc=0}
08:38:55.971817 --- ParallelTerminationStrategy: PID 26633 died...
08:38:56.031165 run iperf3-srv_10.42.42.10(pid=26633): Terminated {rc=256}
08:38:56.042450 --- ParallelTerminationStrategy: PID 26873 died...
08:38:56.101870 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26873): Terminated: ok {rc=0}
08:38:56.112910 --- ParallelTerminationStrategy: PID 26877 died...
08:38:56.172596 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26877): Terminated: ok {rc=0}
08:38:56.184419 --- ParallelTerminationStrategy: PID 26881 died...
08:38:56.244236 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26881): Terminated: ok {rc=0}
08:38:56.337809 bus /gobi_0: Setting Powered False
08:38:57.450065 bus /gobi_0: Setting Powered False
08:38:58.563686 bus /gobi_6: Setting Powered False
74gprs:sysmo9.0osmo-stp_10.42.42.5(pid=26891): Process ended prematurely: osmo-stp_10.42.42.5(pid=26891) [trial-2387↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26891)]2314iperf3.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/iperf3.py", line 8, in <module>
setup_run_iperf3_test_parallel(1)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26891): Process ended prematurely: osmo-stp_10.42.42.5(pid=26891) [trial-2387↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26891)]
----------------------------------------------
trial-2387 gprs:sysmo iperf3.py
----------------------------------------------
08:38:59.686198 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:38:59.699038 tst gprs:sysmo: Using 1 x bts (candidates: 1)
08:38:59.833585 tst iperf3.py: using LAC 32763
08:38:59.965168 tst iperf3.py: using RAC 123
08:39:00.095316 tst iperf3.py: using CellId 32763
08:39:00.231030 tst iperf3.py: using BVCI 32764
08:39:00.256599 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.268336 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.280179 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.292058 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.316768 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.329980 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.343059 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.355068 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:00.368327 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:39:00.682203 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003...
08:39:00.827116 run iperf3-srv_10.42.42.10: Starting iperf3-srv
08:39:01.219971 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26883): Launched
08:39:01.612549 run iperf3-srv_10.42.42.10(pid=26884): Launched
08:39:01.861137 tst iperf3.py:8: start network...
08:39:02.026988 run osmo-hlr_10.42.42.2: Starting osmo-hlr
08:39:02.702627 run create_hlr_db(pid=26885): Launched
08:39:02.910746 bus /gobi_6: Setting Powered False
08:39:03.915404 run create_hlr_db(pid=26885): Terminated: ok {rc=0}
08:39:04.381821 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26887): Launched
08:39:04.803620 run osmo-hlr_10.42.42.2(pid=26888): Launched
08:39:05.065499 run osmo-stp_10.42.42.5: Starting osmo-stp
08:39:05.726455 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26890): Launched
08:39:06.157326 run osmo-stp_10.42.42.5(pid=26891): Launched
08:39:06.398660 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
08:39:07.046595 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26893): Launched
08:39:07.164415 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
08:39:07.464932 run patchelf(pid=26894): Launched
08:39:07.656894 run osmo-stp_10.42.42.5(pid=26891): ERR: Terminated: ERROR {rc=1} [trial-2387↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26891)]
08:39:07.798651 run osmo-stp_10.42.42.5(pid=26891): stdout:
| (launched: 2022-12-13_08:39:05.891762)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:07.939507 run osmo-stp_10.42.42.5(pid=26891): stderr:
| �[0;m�[38;5;43m20221213083906230 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083906230 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083906230 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083906231 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083906231 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083906231 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083906231 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:08.076697 run osmo-stp_10.42.42.5(pid=26891): stdout:
| (launched: 2022-12-13_08:39:05.891762)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:08.211901 run osmo-stp_10.42.42.5(pid=26891): stderr:
| �[0;m�[38;5;43m20221213083906230 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083906230 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083906230 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083906231 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083906231 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083906231 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083906231 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:08.369641 run patchelf(pid=26894): Terminating (SIGINT)
08:39:08.554775 run patchelf(pid=26894): Terminated: ok {rc=0}
08:39:08.589632 tst iperf3.py:8: ERR: Error: osmo-stp_10.42.42.5(pid=26891): Process ended prematurely: osmo-stp_10.42.42.5(pid=26891) [trial-2387↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26891)] [trial-2387↪gprs:sysmo↪iperf3.py:8]
08:39:08.600548 tst iperf3.py:8: Test FAILED (9.0 sec)
08:39:08.667265 run osmo-hlr_10.42.42.2(pid=26888): ERR: Terminated: ERROR {rc=237} [trial-2387↪gprs:sysmo↪iperf3.py↪iperf3.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26888)]
08:39:08.720206 run osmo-hlr_10.42.42.2(pid=26888): stdout:
| (launched: 2022-12-13_08:39:04.547147)
08:39:08.773859 run osmo-hlr_10.42.42.2(pid=26888): stderr:
| �[0;m�[1;31m20221213083904902 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221213083904902 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221213083904902 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221213083904902 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221213083904902 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221213083904902 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221213083904915 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2387/run.2022-12-13_08-32-52/gprs:sysmo/iperf3.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221213083904932 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221213083904932 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221213083904932 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
08:39:08.821357 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26883): Terminating (SIGTERM)
08:39:08.868419 run iperf3-srv_10.42.42.10(pid=26884): Terminating (SIGTERM)
08:39:08.914031 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26887): Terminating (SIGTERM)
08:39:08.959822 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26890): Terminating (SIGTERM)
08:39:09.006041 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26893): Terminating (SIGTERM)
08:39:09.016695 --- ParallelTerminationStrategy: PID 26883 died...
08:39:09.078512 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26883): Terminated: ok {rc=0}
08:39:09.090139 --- ParallelTerminationStrategy: PID 26884 died...
08:39:09.148890 run iperf3-srv_10.42.42.10(pid=26884): Terminated {rc=256}
08:39:09.159768 --- ParallelTerminationStrategy: PID 26887 died...
08:39:09.218256 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26887): Terminated: ok {rc=0}
08:39:09.229925 --- ParallelTerminationStrategy: PID 26890 died...
08:39:09.287927 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26890): Terminated: ok {rc=0}
08:39:09.298841 --- ParallelTerminationStrategy: PID 26893 died...
08:39:09.357311 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26893): Terminated: ok {rc=0}
08:39:09.447815 bus /gobi_6: Setting Powered False
75gprs:sysmo12.0osmo-stp_10.42.42.5(pid=26910): Process ended prematurely: osmo-stp_10.42.42.5(pid=26910) [trial-2387↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26910)]2313iperf3m4.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/iperf3m4.py", line 8, in <module>
setup_run_iperf3_test_parallel(4)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26910): Process ended prematurely: osmo-stp_10.42.42.5(pid=26910) [trial-2387↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26910)]
----------------------------------------------
trial-2387 gprs:sysmo iperf3m4.py
----------------------------------------------
08:39:10.561682 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:10.573053 tst gprs:sysmo: Using 1 x bts (candidates: 1)
08:39:10.708148 tst iperf3m4.py: using LAC 32764
08:39:10.831327 tst iperf3m4.py: using RAC 124
08:39:10.954917 tst iperf3m4.py: using CellId 32764
08:39:11.078296 tst iperf3m4.py: using BVCI 32765
08:39:11.102424 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.113352 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.124035 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.134579 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.146982 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.158406 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.170356 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.181852 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:11.193607 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:39:11.317711 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:39:11.439741 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:39:11.562876 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:39:11.866993 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
08:39:12.007830 run iperf3-srv_10.42.42.10: Starting iperf3-srv
08:39:12.356459 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26896): Launched
08:39:12.650307 run iperf3-srv_10.42.42.10(pid=26897): Launched
08:39:12.857408 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
08:39:12.998887 run iperf3-srv_10.42.42.10: Starting iperf3-srv
08:39:13.348458 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26898): Launched
08:39:13.639238 run iperf3-srv_10.42.42.10(pid=26899): Launched
08:39:13.845391 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
08:39:13.985807 run iperf3-srv_10.42.42.10: Starting iperf3-srv
08:39:14.333690 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26900): Launched
08:39:14.624282 run iperf3-srv_10.42.42.10(pid=26901): Launched
08:39:14.830538 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
08:39:14.970606 run iperf3-srv_10.42.42.10: Starting iperf3-srv
08:39:15.322024 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26902): Launched
08:39:15.623530 run iperf3-srv_10.42.42.10(pid=26903): Launched
08:39:15.815832 tst iperf3m4.py:8: start network...
08:39:15.958290 run osmo-hlr_10.42.42.2: Starting osmo-hlr
08:39:16.563480 run create_hlr_db(pid=26904): Launched
08:39:16.748628 bus /gobi_6: Setting Powered False
08:39:17.767482 run create_hlr_db(pid=26904): Terminated: ok {rc=0}
08:39:18.185586 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26906): Launched
08:39:18.503260 run osmo-hlr_10.42.42.2(pid=26907): Launched
08:39:18.623845 run osmo-stp_10.42.42.5: Starting osmo-stp
08:39:19.177311 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26909): Launched
08:39:19.494847 run osmo-stp_10.42.42.5(pid=26910): Launched
08:39:19.610868 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
08:39:20.157496 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26912): Launched
08:39:20.269945 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
08:39:20.545500 run patchelf(pid=26913): Launched
08:39:20.730029 run osmo-stp_10.42.42.5(pid=26910): ERR: Terminated: ERROR {rc=1} [trial-2387↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26910)]
08:39:20.864304 run osmo-stp_10.42.42.5(pid=26910): stdout:
| (launched: 2022-12-13_08:39:19.337282)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:20.999000 run osmo-stp_10.42.42.5(pid=26910): stderr:
| �[0;m�[38;5;43m20221213083919513 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083919513 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083919513 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083919514 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083919514 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083919514 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083919514 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:21.128656 run osmo-stp_10.42.42.5(pid=26910): stdout:
| (launched: 2022-12-13_08:39:19.337282)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:21.258185 run osmo-stp_10.42.42.5(pid=26910): stderr:
| �[0;m�[38;5;43m20221213083919513 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083919513 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083919513 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083919514 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083919514 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083919514 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083919514 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:21.407827 run patchelf(pid=26913): Terminating (SIGINT)
08:39:21.583960 run patchelf(pid=26913): Terminated: ok {rc=0}
08:39:21.616236 tst iperf3m4.py:8: ERR: Error: osmo-stp_10.42.42.5(pid=26910): Process ended prematurely: osmo-stp_10.42.42.5(pid=26910) [trial-2387↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26910)] [trial-2387↪gprs:sysmo↪iperf3m4.py:8]
08:39:21.625518 tst iperf3m4.py:8: Test FAILED (11.1 sec)
08:39:21.687395 run osmo-hlr_10.42.42.2(pid=26907): ERR: Terminated: ERROR {rc=237} [trial-2387↪gprs:sysmo↪iperf3m4.py↪iperf3m4.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26907)]
08:39:21.737832 run osmo-hlr_10.42.42.2(pid=26907): stdout:
| (launched: 2022-12-13_08:39:18.345101)
08:39:21.789008 run osmo-hlr_10.42.42.2(pid=26907): stderr:
| �[0;m�[1;31m20221213083918525 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221213083918525 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221213083918525 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221213083918525 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221213083918526 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221213083918526 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221213083918537 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2387/run.2022-12-13_08-32-52/gprs:sysmo/iperf3m4.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221213083918545 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221213083918545 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221213083918545 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
08:39:21.832580 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26896): Terminating (SIGTERM)
08:39:21.873590 run iperf3-srv_10.42.42.10(pid=26897): Terminating (SIGTERM)
08:39:21.916304 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26898): Terminating (SIGTERM)
08:39:21.957008 run iperf3-srv_10.42.42.10(pid=26899): Terminating (SIGTERM)
08:39:21.997897 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26900): Terminating (SIGTERM)
08:39:22.038604 run iperf3-srv_10.42.42.10(pid=26901): Terminating (SIGTERM)
08:39:22.079374 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26902): Terminating (SIGTERM)
08:39:22.120110 run iperf3-srv_10.42.42.10(pid=26903): Terminating (SIGTERM)
08:39:22.160943 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26906): Terminating (SIGTERM)
08:39:22.201829 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26909): Terminating (SIGTERM)
08:39:22.242594 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26912): Terminating (SIGTERM)
08:39:22.252056 --- ParallelTerminationStrategy: PID 26896 died...
08:39:22.306973 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26896): Terminated: ok {rc=0}
08:39:22.316761 --- ParallelTerminationStrategy: PID 26897 died...
08:39:22.371113 run iperf3-srv_10.42.42.10(pid=26897): Terminated {rc=256}
08:39:22.380651 --- ParallelTerminationStrategy: PID 26898 died...
08:39:22.435184 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26898): Terminated: ok {rc=0}
08:39:22.444774 --- ParallelTerminationStrategy: PID 26899 died...
08:39:22.499250 run iperf3-srv_10.42.42.10(pid=26899): Terminated {rc=256}
08:39:22.508853 --- ParallelTerminationStrategy: PID 26900 died...
08:39:22.563227 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26900): Terminated: ok {rc=0}
08:39:22.572895 --- ParallelTerminationStrategy: PID 26901 died...
08:39:22.627202 run iperf3-srv_10.42.42.10(pid=26901): Terminated {rc=256}
08:39:22.636844 --- ParallelTerminationStrategy: PID 26902 died...
08:39:22.691204 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26902): Terminated: ok {rc=0}
08:39:22.700781 --- ParallelTerminationStrategy: PID 26903 died...
08:39:22.755094 run iperf3-srv_10.42.42.10(pid=26903): Terminated {rc=256}
08:39:22.764787 --- ParallelTerminationStrategy: PID 26906 died...
08:39:22.819135 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26906): Terminated: ok {rc=0}
08:39:22.829423 --- ParallelTerminationStrategy: PID 26909 died...
08:39:22.883409 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26909): Terminated: ok {rc=0}
08:39:22.893114 --- ParallelTerminationStrategy: PID 26912 died...
08:39:22.946941 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26912): Terminated: ok {rc=0}
08:39:23.030707 bus /gobi_4: Setting Powered False
08:39:24.139192 bus /gobi_4: Setting Powered False
08:39:25.248189 bus /gobi_1: Setting Powered False
08:39:26.354547 bus /gobi_1: Setting Powered False
08:39:27.462858 bus /gobi_0: Setting Powered False
08:39:28.569550 bus /gobi_0: Setting Powered False
08:39:29.679210 bus /gobi_6: Setting Powered False
57gprs:sysmo8.0osmo-stp_10.42.42.5(pid=26920): Process ended prematurely: osmo-stp_10.42.42.5(pid=26920) [trial-2387↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26920)]2331ping.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/ping.py", line 22, in <module>
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26920): Process ended prematurely: osmo-stp_10.42.42.5(pid=26920) [trial-2387↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26920)]
----------------------------------------------
trial-2387 gprs:sysmo ping.py
----------------------------------------------
08:39:30.747373 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:30.759069 tst gprs:sysmo: Using 1 x bts (candidates: 1)
08:39:30.886328 tst ping.py: using LAC 32765
08:39:31.024759 tst ping.py: using RAC 125
08:39:31.177279 tst ping.py: using CellId 32765
08:39:31.326574 tst ping.py: using BVCI 32766
08:39:31.359015 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:31.372154 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:31.384001 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:31.398665 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:31.415761 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:31.432257 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:31.448065 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:31.464628 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:39:31.843558 tst ping.py:19: start network...
08:39:31.980505 run osmo-hlr_10.42.42.2: Starting osmo-hlr
08:39:32.557679 run create_hlr_db(pid=26914): Launched
08:39:32.749507 bus /gobi_6: Setting Powered False
08:39:33.769006 run create_hlr_db(pid=26914): Terminated: ok {rc=0}
08:39:34.208616 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26916): Launched
08:39:34.542434 run osmo-hlr_10.42.42.2(pid=26917): Launched
08:39:34.662403 run osmo-stp_10.42.42.5: Starting osmo-stp
08:39:35.236047 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26919): Launched
08:39:35.567963 run osmo-stp_10.42.42.5(pid=26920): Launched
08:39:35.687259 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
08:39:36.264619 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26922): Launched
08:39:36.382453 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
08:39:36.671013 run patchelf(pid=26923): Launched
08:39:36.863091 run osmo-stp_10.42.42.5(pid=26920): ERR: Terminated: ERROR {rc=1} [trial-2387↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26920)]
08:39:37.003626 run osmo-stp_10.42.42.5(pid=26920): stdout:
| (launched: 2022-12-13_08:39:35.404357)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:37.144334 run osmo-stp_10.42.42.5(pid=26920): stderr:
| �[0;m�[38;5;43m20221213083935601 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083935601 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083935602 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083935602 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083935603 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083935603 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083935603 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:37.280314 run osmo-stp_10.42.42.5(pid=26920): stdout:
| (launched: 2022-12-13_08:39:35.404357)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:37.415938 run osmo-stp_10.42.42.5(pid=26920): stderr:
| �[0;m�[38;5;43m20221213083935601 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083935601 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083935602 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083935602 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083935603 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083935603 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083935603 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:37.572489 run patchelf(pid=26923): Terminating (SIGINT)
08:39:37.756031 run patchelf(pid=26923): Terminated: ok {rc=0}
08:39:37.789079 tst ping.py:22: ERR: Error: osmo-stp_10.42.42.5(pid=26920): Process ended prematurely: osmo-stp_10.42.42.5(pid=26920) [trial-2387↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26920)] [trial-2387↪gprs:sysmo↪ping.py:22]
08:39:37.798309 tst ping.py:22: Test FAILED (7.0 sec)
08:39:37.861764 run osmo-hlr_10.42.42.2(pid=26917): ERR: Terminated: ERROR {rc=237} [trial-2387↪gprs:sysmo↪ping.py↪ping.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26917)]
08:39:37.913637 run osmo-hlr_10.42.42.2(pid=26917): stdout:
| (launched: 2022-12-13_08:39:34.378205)
08:39:37.966280 run osmo-hlr_10.42.42.2(pid=26917): stderr:
| �[0;m�[1;31m20221213083934579 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221213083934579 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221213083934579 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221213083934579 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221213083934579 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221213083934579 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221213083934591 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2387/run.2022-12-13_08-32-52/gprs:sysmo/ping.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221213083934600 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221213083934600 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221213083934600 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
08:39:38.011253 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26916): Terminating (SIGTERM)
08:39:38.054572 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26919): Terminating (SIGTERM)
08:39:38.097777 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26922): Terminating (SIGTERM)
08:39:38.107727 --- ParallelTerminationStrategy: PID 26916 died...
08:39:38.164423 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26916): Terminated: ok {rc=0}
08:39:38.174607 --- ParallelTerminationStrategy: PID 26919 died...
08:39:38.230965 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26919): Terminated: ok {rc=0}
08:39:38.241166 --- ParallelTerminationStrategy: PID 26922 died...
08:39:38.297173 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26922): Terminated: ok {rc=0}
08:39:38.384989 bus /gobi_6: Setting Powered False
57gprs:sysmo7.0osmo-stp_10.42.42.5(pid=26930): Process ended prematurely: osmo-stp_10.42.42.5(pid=26930) [trial-2387↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26930)]2331ping_idle_ping.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/ping_idle_ping.py", line 27, in <module>
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26930): Process ended prematurely: osmo-stp_10.42.42.5(pid=26930) [trial-2387↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26930)]
----------------------------------------------
trial-2387 gprs:sysmo ping_idle_ping.py
----------------------------------------------
08:39:39.437353 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:39.449143 tst gprs:sysmo: Using 1 x bts (candidates: 1)
08:39:39.583206 tst ping_idle_ping.py: using LAC 32766
08:39:39.712521 tst ping_idle_ping.py: using RAC 126
08:39:39.838654 tst ping_idle_ping.py: using CellId 32766
08:39:39.962347 tst ping_idle_ping.py: using BVCI 32767
08:39:39.986382 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:39.997384 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:40.007988 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:40.018510 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:40.029669 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:40.040610 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:40.051599 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
08:39:40.062694 tst gprs:sysmo: Using 1 x modem (candidates: 4)
08:39:40.354504 tst ping_idle_ping.py:24: start network...
08:39:40.484425 run osmo-hlr_10.42.42.2: Starting osmo-hlr
08:39:40.974463 run create_hlr_db(pid=26924): Launched
08:39:41.153347 bus /gobi_6: Setting Powered False
08:39:42.173462 run create_hlr_db(pid=26924): Terminated: ok {rc=0}
08:39:42.613746 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26926): Launched
08:39:42.947459 run osmo-hlr_10.42.42.2(pid=26927): Launched
08:39:43.068561 run osmo-stp_10.42.42.5: Starting osmo-stp
08:39:43.645505 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26929): Launched
08:39:43.979096 run osmo-stp_10.42.42.5(pid=26930): Launched
08:39:44.099058 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
08:39:44.677713 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26932): Launched
08:39:44.796146 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
08:39:45.087011 run patchelf(pid=26933): Launched
08:39:45.279438 run osmo-stp_10.42.42.5(pid=26930): ERR: Terminated: ERROR {rc=1} [trial-2387↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26930)]
08:39:45.420882 run osmo-stp_10.42.42.5(pid=26930): stdout:
| (launched: 2022-12-13_08:39:43.813720)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:45.562201 run osmo-stp_10.42.42.5(pid=26930): stderr:
| �[0;m�[38;5;43m20221213083944009 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083944009 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083944009 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083944010 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083944010 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083944010 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083944010 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:45.698743 run osmo-stp_10.42.42.5(pid=26930): stdout:
| (launched: 2022-12-13_08:39:43.813720)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
08:39:45.834001 run osmo-stp_10.42.42.5(pid=26930): stderr:
| �[0;m�[38;5;43m20221213083944009 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221213083944009 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221213083944009 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221213083944010 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221213083944010 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221213083944010 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221213083944010 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
08:39:45.998441 run patchelf(pid=26933): Terminating (SIGINT)
08:39:46.179445 run patchelf(pid=26933): Terminated: ok {rc=0}
08:39:46.222342 tst ping_idle_ping.py:27: ERR: Error: osmo-stp_10.42.42.5(pid=26930): Process ended prematurely: osmo-stp_10.42.42.5(pid=26930) [trial-2387↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26930)] [trial-2387↪gprs:sysmo↪ping_idle_ping.py:27]
08:39:46.236023 tst ping_idle_ping.py:27: Test FAILED (6.8 sec)
08:39:46.319696 run osmo-hlr_10.42.42.2(pid=26927): ERR: Terminated: ERROR {rc=237} [trial-2387↪gprs:sysmo↪ping_idle_ping.py↪ping_idle_ping.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26927)]
08:39:46.386650 run osmo-hlr_10.42.42.2(pid=26927): stdout:
| (launched: 2022-12-13_08:39:42.782088)
08:39:46.456170 run osmo-hlr_10.42.42.2(pid=26927): stderr:
| �[0;m�[1;31m20221213083942981 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221213083942981 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221213083942981 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221213083942981 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221213083942981 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221213083942981 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221213083942993 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2387/run.2022-12-13_08-32-52/gprs:sysmo/ping_idle_ping.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221213083943002 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221213083943002 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221213083943002 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
08:39:46.515168 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26926): Terminating (SIGTERM)
08:39:46.563491 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26929): Terminating (SIGTERM)
08:39:46.608158 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26932): Terminating (SIGTERM)
08:39:46.620052 --- ParallelTerminationStrategy: PID 26926 died...
08:39:46.681938 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26926): Terminated: ok {rc=0}
08:39:46.692794 --- ParallelTerminationStrategy: PID 26929 died...
08:39:46.752854 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26929): Terminated: ok {rc=0}
08:39:46.764487 --- ParallelTerminationStrategy: PID 26932 died...
08:39:46.822013 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26932): Terminated: ok {rc=0}
08:39:46.911803 bus /gobi_6: Setting Powered False
5gprs:sysmo00