jenkins LTS upgrade
Failed
handover_2G:sysmo+bts1-trx-b200.handover.py (from handover_2G_sysmo+bts1-trx-b200)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f6e84097268>, timeout=10, timestep=1} [trial-813↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45↪handover.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-813 handover_2G:sysmo+bts1-trx-b200 handover.py ---------------------------------------------- 20:08:17.622434 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 20:08:17.633257 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x bts (candidates: 2) 20:08:17.763140 tst handover.py: using LAC 8106 20:08:17.890648 tst handover.py: using RAC 201 20:08:18.017750 tst handover.py: using CellId 8106 20:08:18.145982 tst handover.py: using BVCI 8102 20:08:18.170315 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x bts (candidates: 2) 20:08:18.311054 tst handover.py: using LAC 8107 20:08:18.446075 tst handover.py: using RAC 202 20:08:18.570940 tst handover.py: using CellId 8107 20:08:18.697333 tst handover.py: using BVCI 8103 20:08:18.721987 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 20:08:18.732845 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 20:08:18.743593 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 20:08:18.754581 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 20:08:18.765578 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 20:08:18.776468 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x modem (candidates: 2) 20:08:18.878083 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x modem (candidates: 2) 20:08:19.071233 run osmo-hlr_10.42.42.2: Starting osmo-hlr 20:08:19.551298 run create_hlr_db(pid=18125): Launched 20:08:19.682134 bus /sierra_1: Setting Powered False 20:08:20.758424 run create_hlr_db(pid=18125): Terminated: ok {rc=0} 20:08:21.161581 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18127): Launched 20:08:21.486181 run osmo-hlr_10.42.42.2(pid=18128): Launched 20:08:21.601623 run osmo-stp_10.42.42.5: Starting osmo-stp 20:08:22.135104 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18129): Launched 20:08:22.460068 run osmo-stp_10.42.42.5(pid=18130): Launched 20:08:22.575475 run osmo-msc_10.42.42.6: Starting osmo-msc 20:08:23.137656 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=18131): Launched 20:08:23.464441 run osmo-msc_10.42.42.6(pid=18132): Launched 20:08:26.641328 run osmo-mgw_10.42.42.3: Starting osmo-mgw 20:08:27.177766 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18134): Launched 20:08:27.503331 run osmo-mgw_10.42.42.3(pid=18135): Launched 20:08:27.618593 run osmo-mgw_10.42.42.4: Starting osmo-mgw 20:08:28.143857 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18136): Launched 20:08:28.469598 run osmo-mgw_10.42.42.4(pid=18137): Launched 20:08:28.585454 run osmo-bsc_10.42.42.7: Starting osmo-bsc 20:08:29.290546 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=18138): Launched 20:08:29.620940 run osmo-bsc_10.42.42.7(pid=18139): Launched 20:08:36.335612 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 20:08:36.879769 run rm-remote-dir(pid=18140): Launched 20:08:39.077049 run rm-remote-dir(pid=18140): Terminated: ok {rc=0} 20:08:39.422087 run mk-remote-dir(pid=18141): Launched 20:08:41.619356 run mk-remote-dir(pid=18141): Terminated: ok {rc=0} 20:08:41.909793 run scp-inst-to-remote(pid=18142): Launched 20:08:55.134420 run scp-inst-to-remote(pid=18142): Terminated: ok {rc=0} 20:08:55.495032 run mk-remote-dir(pid=18144): Launched 20:08:56.701448 run mk-remote-dir(pid=18144): Terminated: ok {rc=0} 20:08:57.010310 run scp-cfg-to-remote(pid=18145): Launched 20:08:59.217092 run scp-cfg-to-remote(pid=18145): Terminated: ok {rc=0} 20:08:59.571000 run reload-dsp-firmware(pid=18147): Launched 20:09:02.775910 run reload-dsp-firmware(pid=18147): Terminated: ok {rc=0} 20:09:03.124598 run osmo-bts-sysmo(pid=18148): Launched 20:09:08.447795 tst handover.py:42: ----- Report fragment: handover_2G:sysmo+bts1-trx-b200/handover.py/01_bts0_started: pass (32.0s) 20:09:08.599503 tst handover.py: using MSISDN 5241 20:09:08.786492 bus /sierra_1: Setting Powered True 20:09:12.284008 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5241', subscriber_id=1} 20:09:12.430992 tst handover.py: using MSISDN 5242 20:09:12.617808 bus /sierra_2: Setting Powered True 20:09:24.989697 tst handover.py:45: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f6e84097268>, timeout=10, timestep=1} [trial-813↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45↪handover.py↪/sierra_2] [trial-813↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45] 20:09:24.996900 tst handover.py:45: Test FAILED (67.4 sec) 20:09:25.032946 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18127): Terminating (SIGTERM) 20:09:25.075773 run osmo-hlr_10.42.42.2(pid=18128): Terminating (SIGTERM) 20:09:25.119260 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18129): Terminating (SIGTERM) 20:09:25.164874 run osmo-stp_10.42.42.5(pid=18130): Terminating (SIGTERM) 20:09:25.243896 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=18131): Terminating (SIGTERM) 20:09:25.292156 run osmo-msc_10.42.42.6(pid=18132): Terminating (SIGTERM) 20:09:25.335796 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18134): Terminating (SIGTERM) 20:09:25.378435 run osmo-mgw_10.42.42.3(pid=18135): Terminating (SIGTERM) 20:09:25.422477 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18136): Terminating (SIGTERM) 20:09:25.464315 run osmo-mgw_10.42.42.4(pid=18137): Terminating (SIGTERM) 20:09:25.504934 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=18138): Terminating (SIGTERM) 20:09:25.546186 run osmo-bsc_10.42.42.7(pid=18139): Terminating (SIGTERM) 20:09:25.586472 run osmo-bts-sysmo(pid=18148): Terminating (SIGTERM) 20:09:25.596265 --- ParallelTerminationStrategy: PID 18127 died... 20:09:25.649943 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18127): Terminated: ok {rc=0} 20:09:25.660222 --- ParallelTerminationStrategy: PID 18128 died... 20:09:25.715344 run osmo-hlr_10.42.42.2(pid=18128): Terminated: ok {rc=0} 20:09:25.726039 --- ParallelTerminationStrategy: PID 18129 died... 20:09:25.782130 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18129): Terminated: ok {rc=0} 20:09:25.792751 --- ParallelTerminationStrategy: PID 18130 died... 20:09:25.849259 run osmo-stp_10.42.42.5(pid=18130): Terminated: ok {rc=0} 20:09:25.859198 --- ParallelTerminationStrategy: PID 18131 died... 20:09:25.913285 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=18131): Terminated: ok {rc=0} 20:09:25.923234 --- ParallelTerminationStrategy: PID 18132 died... 20:09:25.977073 run osmo-msc_10.42.42.6(pid=18132): Terminated: ok {rc=0} 20:09:25.987029 --- ParallelTerminationStrategy: PID 18134 died... 20:09:26.040431 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18134): Terminated: ok {rc=0} 20:09:26.050640 --- ParallelTerminationStrategy: PID 18135 died... 20:09:26.104301 run osmo-mgw_10.42.42.3(pid=18135): Terminated {rc=15} 20:09:26.114306 --- ParallelTerminationStrategy: PID 18136 died... 20:09:26.168347 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18136): Terminated: ok {rc=0} 20:09:26.178395 --- ParallelTerminationStrategy: PID 18137 died... 20:09:26.231944 run osmo-mgw_10.42.42.4(pid=18137): Terminated {rc=15} 20:09:26.242162 --- ParallelTerminationStrategy: PID 18138 died... 20:09:26.295982 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=18138): Terminated: ok {rc=0} 20:09:26.306063 --- ParallelTerminationStrategy: PID 18139 died... 20:09:26.359405 run osmo-bsc_10.42.42.7(pid=18139): Terminated: ok {rc=0} 20:09:26.369638 --- ParallelTerminationStrategy: PID 18148 died... 20:09:26.423041 run osmo-bts-sysmo(pid=18148): Terminated: ok {rc=0} 20:09:26.532487 bus /sierra_2: Setting Online False 20:09:27.616941 bus /sierra_2: Setting Powered False 20:09:32.963236 bus /sierra_2: Setting Powered False 20:09:34.068968 bus /sierra_2: Setting Powered False 20:09:35.197835 bus /sierra_1: Setting Online False 20:09:36.286125 bus /sierra_1: Setting Powered False 20:09:41.967971 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/handover_2G/handover.py", line 45, in <module> hlr.subscriber_add(ms1) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f6e84097268>, timeout=10, timestep=1} [trial-813↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45↪handover.py↪/sierra_2]