![[Jenkins]](/jenkins/static/342c126a/images/svgs/logo.svg){kind=logo}
Failed
handover_2G:oc2g+bts1-trx-sysmocell5000.handover.py (from handover_2G_oc2g+bts1-trx-sysmocell5000)
Stacktrace
handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-159↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84]Standard Output
----------------------------------------------
trial-159 handover_2G:oc2g+bts1-trx-sysmocell5000 handover.py
----------------------------------------------
00:22:16.739923 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
00:22:16.754524 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x bts (candidates: 2)
00:22:16.896940 tst handover.py: using LAC 16015
00:22:17.029525 tst handover.py: using RAC 205
00:22:17.164677 tst handover.py: using CellId 16014
00:22:17.300763 tst handover.py: using BVCI 2693
00:22:17.327247 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x bts (candidates: 2)
00:22:17.483594 tst handover.py: using LAC 16016
00:22:17.616150 tst handover.py: using RAC 206
00:22:17.749413 tst handover.py: using CellId 16015
00:22:17.881996 tst handover.py: using BVCI 2694
00:22:17.908058 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
00:22:17.921753 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
00:22:17.935347 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
00:22:17.949048 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
00:22:17.962520 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
00:22:17.975922 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x modem (candidates: 2)
00:22:18.109195 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x modem (candidates: 2)
00:22:18.339659 run osmo-hlr_10.42.42.2: Starting osmo-hlr
00:22:18.847884 run create_hlr_db(pid=14354): Launched
00:22:18.989598 bus /sierra_2: Setting Powered False
00:22:20.072431 run create_hlr_db(pid=14354): Terminated: ok {rc=0}
00:22:20.523890 run pcap-recorder_any(filters='host 10.42.42.2')(pid=14356): Launched
00:22:20.879083 run osmo-hlr_10.42.42.2(pid=14357): Launched
00:22:21.000890 run osmo-stp_10.42.42.5: Starting osmo-stp
00:22:21.566083 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=14358): Launched
00:22:21.906783 run osmo-stp_10.42.42.5(pid=14359): Launched
00:22:22.027029 run osmo-msc_10.42.42.6: Starting osmo-msc
00:22:22.624002 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=14360): Launched
00:22:22.964984 run osmo-msc_10.42.42.6(pid=14361): Launched
00:22:23.089080 run osmo-mgw_10.42.42.3: Starting osmo-mgw
00:22:23.663023 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=14362): Launched
00:22:24.021264 run osmo-mgw_10.42.42.3(pid=14363): Launched
00:22:24.149733 run osmo-mgw_10.42.42.4: Starting osmo-mgw
00:22:24.728610 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=14364): Launched
00:22:25.076660 run osmo-mgw_10.42.42.4(pid=14365): Launched
00:22:25.205124 run osmo-bsc_10.42.42.7: Starting osmo-bsc
00:22:25.995486 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=14366): Launched
00:22:26.345986 run osmo-bsc_10.42.42.7(pid=14367): Launched
00:22:30.046553 run osmo-bts-oc2g: Starting OsmoBtsOC2G to connect to osmo-bsc_10.42.42.7
00:22:30.636923 run rm-remote-dir(pid=14368): Launched
00:22:31.861281 run rm-remote-dir(pid=14368): Terminated: ok {rc=0}
00:22:32.252590 run mk-remote-dir(pid=14369): Launched
00:22:33.476929 run mk-remote-dir(pid=14369): Terminated: ok {rc=0}
00:22:33.798623 run scp-inst-to-remote(pid=14370): Launched
00:22:40.018167 run scp-inst-to-remote(pid=14370): Terminated: ok {rc=0}
00:22:40.388084 run mk-remote-dir(pid=14372): Launched
00:22:41.608576 run mk-remote-dir(pid=14372): Terminated: ok {rc=0}
00:22:41.934220 run scp-cfg-to-remote(pid=14373): Launched
00:22:43.154950 run scp-cfg-to-remote(pid=14373): Terminated: ok {rc=0}
00:22:43.517130 run osmo-bts-oc2g(pid=14375): Launched
00:22:49.976283 tst handover.py:43: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/01_bts0_started: pass (19.8s)
00:22:50.142511 tst handover.py: using MSISDN 12918
00:22:50.335336 bus /sierra_2: Setting Powered True
00:22:53.883536 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='12918', subscriber_id=1}
00:22:54.047752 tst handover.py: using MSISDN 12919
00:22:54.237517 bus /sierra_3: Setting Powered True
00:23:00.739941 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='12919', subscriber_id=2}
00:23:01.040755 bus /sierra_2: Setting Online False
00:23:02.257704 bus /sierra_2: Setting Powered False
00:23:07.987800 bus /sierra_2: Setting Powered True
00:23:09.362371 bus /sierra_2: Setting Online True
00:23:11.995818 tst /sierra_2: Connect to ('901', '70')
00:23:12.357920 bus /sierra_3: Setting Online False
00:23:13.576005 bus /sierra_3: Setting Powered False
00:23:19.842779 bus /sierra_3: Setting Powered True
00:23:24.225770 bus /sierra_3: Setting Online True
00:23:26.578958 tst /sierra_2: Already registered with network ('901', '70')
00:23:27.026560 tst /sierra_3: Connect to ('901', '70')
00:23:27.213381 tst /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053'
00:23:27.346778 tst /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028'
00:23:27.552651 tst handover.py:54: waiting for modems to attach...
00:23:30.866800 tst handover.py:57: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/02.1_ms0_attach: pass (3.1s)
00:23:41.237832 tst /sierra_3: Already registered with network ('901', '70')
00:23:46.295399 tst handover.py:60: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/02.2_ms1_attach: pass (15.2s)
00:23:47.827505 tst handover.py:63: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/02.3_subscribed_in_msc: pass (1.3s)
00:23:52.541817 tst handover.py:68: dial success
00:23:55.270983 tst handover.py:75: answer success, call established and ongoing
00:23:55.589622 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
00:23:55.821781 tst handover.py:79: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/03_call_established: pass (3.0s)
00:23:55.977759 run osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
00:23:58.352390 run ssh_sc5_ccli(pid=14383): Launched
00:24:17.610475 run ssh_sc5_ccli(pid=14383): ERR: Terminated: ERROR {rc=255} [trial-159↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:83↪handover.py↪osmo-bts-trx↪sc5-trx↪ssh_sc5_ccli(pid=14383)]
00:24:17.780747 run ssh_sc5_ccli(pid=14383): stdout:
| (launched: 2020-12-07_00:23:58.166953)
00:24:17.948788 run ssh_sc5_ccli(pid=14383): stderr:
| (launched: 2020-12-07_00:23:58.224019)
| ssh: connect to host 10.42.42.112 port 22: No route to host
00:24:18.091698 run sc5-trx: Configuring SC5 TRX failed, retrying 9 more times
00:24:23.466107 run ssh_sc5_ccli(pid=14384): Launched
00:24:26.678700 run ssh_sc5_ccli(pid=14384): Terminated: ok {rc=0}
00:24:26.805993 run osmo-bts-trx: Waiting for sc5-trx to start up...
00:24:28.128436 run osmo-bts-trx(pid=14385): Launched
00:29:28.870059 tst handover.py:84: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/04.1_bts1_started: FAIL (333.0s)
00:29:28.877022 tst handover.py:84: ERR: Error: handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-159↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84] [trial-159↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84]
00:29:28.883580 tst handover.py:84: Test FAILED (432.1 sec)
00:29:28.925790 run pcap-recorder_any(filters='host 10.42.42.2')(pid=14356): Terminating (SIGTERM)
00:29:28.974452 run osmo-hlr_10.42.42.2(pid=14357): Terminating (SIGTERM)
00:29:29.024012 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=14358): Terminating (SIGTERM)
00:29:29.076020 run osmo-stp_10.42.42.5(pid=14359): Terminating (SIGTERM)
00:29:29.126352 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=14360): Terminating (SIGTERM)
00:29:29.178004 run osmo-msc_10.42.42.6(pid=14361): Terminating (SIGTERM)
00:29:29.226788 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=14362): Terminating (SIGTERM)
00:29:29.276883 run osmo-mgw_10.42.42.3(pid=14363): Terminating (SIGTERM)
00:29:29.326489 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=14364): Terminating (SIGTERM)
00:29:29.378255 run osmo-mgw_10.42.42.4(pid=14365): Terminating (SIGTERM)
00:29:29.428907 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=14366): Terminating (SIGTERM)
00:29:29.478002 run osmo-bsc_10.42.42.7(pid=14367): Terminating (SIGTERM)
00:29:29.526539 run osmo-bts-oc2g(pid=14375): Terminating (SIGTERM)
00:29:29.574834 run osmo-bts-trx(pid=14385): Terminating (SIGTERM)
00:29:29.585797 --- ParallelTerminationStrategy: PID 14356 died...
00:29:29.650131 run pcap-recorder_any(filters='host 10.42.42.2')(pid=14356): Terminated: ok {rc=0}
00:29:29.664031 --- ParallelTerminationStrategy: PID 14357 died...
00:29:29.736880 run osmo-hlr_10.42.42.2(pid=14357): Terminated: ok {rc=0}
00:29:29.749527 --- ParallelTerminationStrategy: PID 14358 died...
00:29:29.819470 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=14358): Terminated: ok {rc=0}
00:29:29.831843 --- ParallelTerminationStrategy: PID 14359 died...
00:29:29.901045 run osmo-stp_10.42.42.5(pid=14359): Terminated: ok {rc=0}
00:29:29.913731 --- ParallelTerminationStrategy: PID 14360 died...
00:29:29.983023 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=14360): Terminated: ok {rc=0}
00:29:29.995342 --- ParallelTerminationStrategy: PID 14362 died...
00:29:30.064941 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=14362): Terminated: ok {rc=0}
00:29:30.077732 --- ParallelTerminationStrategy: PID 14363 died...
00:29:30.147271 run osmo-mgw_10.42.42.3(pid=14363): Terminated {rc=15}
00:29:30.160021 --- ParallelTerminationStrategy: PID 14364 died...
00:29:30.234022 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=14364): Terminated: ok {rc=0}
00:29:30.246584 --- ParallelTerminationStrategy: PID 14365 died...
00:29:30.316139 run osmo-mgw_10.42.42.4(pid=14365): Terminated {rc=15}
00:29:30.328746 --- ParallelTerminationStrategy: PID 14366 died...
00:29:30.398600 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=14366): Terminated: ok {rc=0}
00:29:30.410925 --- ParallelTerminationStrategy: PID 14375 died...
00:29:30.480993 run osmo-bts-oc2g(pid=14375): Terminated {rc=36608}
00:29:32.509545 --- ParallelTerminationStrategy: PID 14361 died...
00:29:32.583222 run osmo-msc_10.42.42.6(pid=14361): Terminated: ok {rc=0}
00:29:33.267215 --- ParallelTerminationStrategy: PID 14367 died...
00:29:33.338872 run osmo-bsc_10.42.42.7(pid=14367): Terminated: ok {rc=0}
00:29:33.351111 --- ParallelTerminationStrategy: PID 14385 died...
00:29:33.420258 run osmo-bts-trx(pid=14385): Terminated: ok {rc=0}
00:29:33.574016 bus /sierra_3: Setting Online False
00:29:34.711567 bus /sierra_3: Setting Powered False
00:29:50.662218 bus /sierra_3: Setting Powered False
00:29:51.782696 bus /sierra_3: Setting Powered False
00:29:52.960697 bus /sierra_2: Setting Online False
00:29:54.101638 bus /sierra_2: Setting Powered False
00:30:05.640273 bus /sierra_2: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 673, in exec_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/sysmocom/suites/handover_2G/handover.py", line 84, in <module>
wait(bsc.bts_is_connected, bts1)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait
raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep)
osmo_gsm_tester.core.log.Error: handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-159↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84]