![[Jenkins]](/jenkins/static/342c126a/images/svgs/logo.svg){kind=logo}
Failed
handover_2G:oc2g+bts1-trx-sysmocell5000.handover.py (from handover_2G_oc2g+bts1-trx-sysmocell5000)
Stacktrace
handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-160↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84]Standard Output
----------------------------------------------
trial-160 handover_2G:oc2g+bts1-trx-sysmocell5000 handover.py
----------------------------------------------
05:20:17.751777 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
05:20:17.765634 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x bts (candidates: 2)
05:20:17.903060 tst handover.py: using LAC 16111
05:20:18.042020 tst handover.py: using RAC 46
05:20:18.171858 tst handover.py: using CellId 16110
05:20:18.303194 tst handover.py: using BVCI 2789
05:20:18.329268 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x bts (candidates: 2)
05:20:18.483695 tst handover.py: using LAC 16112
05:20:18.614068 tst handover.py: using RAC 47
05:20:18.743814 tst handover.py: using CellId 16111
05:20:18.874798 tst handover.py: using BVCI 2790
05:20:18.900961 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
05:20:18.915187 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
05:20:18.928932 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
05:20:18.942657 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
05:20:18.956273 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
05:20:18.969998 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x modem (candidates: 2)
05:20:19.102869 tst handover_2G:oc2g+bts1-trx-sysmocell5000: Using 1 x modem (candidates: 2)
05:20:19.333490 run osmo-hlr_10.42.42.2: Starting osmo-hlr
05:20:19.840617 run create_hlr_db(pid=28445): Launched
05:20:19.981849 bus /sierra_2: Setting Powered False
05:20:21.066068 run create_hlr_db(pid=28445): Terminated: ok {rc=0}
05:20:21.509886 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28447): Launched
05:20:21.859667 run osmo-hlr_10.42.42.2(pid=28448): Launched
05:20:21.988085 run osmo-stp_10.42.42.5: Starting osmo-stp
05:20:22.577809 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28449): Launched
05:20:22.935286 run osmo-stp_10.42.42.5(pid=28450): Launched
05:20:23.062831 run osmo-msc_10.42.42.6: Starting osmo-msc
05:20:23.692689 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28451): Launched
05:20:24.034550 run osmo-msc_10.42.42.6(pid=28452): Launched
05:20:24.158477 run osmo-mgw_10.42.42.3: Starting osmo-mgw
05:20:24.731880 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28453): Launched
05:20:25.087804 run osmo-mgw_10.42.42.3(pid=28454): Launched
05:20:25.209463 run osmo-mgw_10.42.42.4: Starting osmo-mgw
05:20:25.761846 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28455): Launched
05:20:26.110986 run osmo-mgw_10.42.42.4(pid=28456): Launched
05:20:26.239503 run osmo-bsc_10.42.42.7: Starting osmo-bsc
05:20:27.026987 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28457): Launched
05:20:27.376737 run osmo-bsc_10.42.42.7(pid=28458): Launched
05:20:31.110988 run osmo-bts-oc2g: Starting OsmoBtsOC2G to connect to osmo-bsc_10.42.42.7
05:20:31.716212 run rm-remote-dir(pid=28459): Launched
05:20:32.930184 run rm-remote-dir(pid=28459): Terminated: ok {rc=0}
05:20:33.298145 run mk-remote-dir(pid=28460): Launched
05:20:34.510954 run mk-remote-dir(pid=28460): Terminated: ok {rc=0}
05:20:34.820706 run scp-inst-to-remote(pid=28461): Launched
05:20:42.052219 run scp-inst-to-remote(pid=28461): Terminated: ok {rc=0}
05:20:42.435134 run mk-remote-dir(pid=28463): Launched
05:20:43.656305 run mk-remote-dir(pid=28463): Terminated: ok {rc=0}
05:20:43.976189 run scp-cfg-to-remote(pid=28464): Launched
05:20:45.187509 run scp-cfg-to-remote(pid=28464): Terminated: ok {rc=0}
05:20:45.541491 run osmo-bts-oc2g(pid=28466): Launched
05:20:52.003252 tst handover.py:43: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/01_bts0_started: pass (20.8s)
05:20:52.159882 tst handover.py: using MSISDN 13020
05:20:52.361457 bus /sierra_2: Setting Powered True
05:20:55.895247 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='13020', subscriber_id=1}
05:20:56.054657 tst handover.py: using MSISDN 13021
05:20:56.252781 bus /sierra_3: Setting Powered True
05:21:02.769482 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='13021', subscriber_id=2}
05:21:03.080627 bus /sierra_2: Setting Online False
05:21:04.286862 bus /sierra_2: Setting Powered False
05:21:10.036210 bus /sierra_2: Setting Powered True
05:21:11.424864 bus /sierra_2: Setting Online True
05:21:14.044399 tst /sierra_2: Connect to ('901', '70')
05:21:14.459306 bus /sierra_3: Setting Online False
05:21:15.674570 bus /sierra_3: Setting Powered False
05:21:21.858056 bus /sierra_3: Setting Powered True
05:21:26.250311 bus /sierra_3: Setting Online True
05:21:28.607249 tst /sierra_2: Already registered with network ('901', '70')
05:21:28.981182 tst /sierra_3: Connect to ('901', '70')
05:21:29.166867 tst /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053'
05:21:29.301752 tst /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028'
05:21:29.515540 tst handover.py:54: waiting for modems to attach...
05:21:32.827043 tst handover.py:57: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/02.1_ms0_attach: pass (3.1s)
05:21:43.711266 tst /sierra_3: Already registered with network ('901', '70')
05:21:48.758704 tst handover.py:60: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/02.2_ms1_attach: pass (15.7s)
05:21:50.310179 tst handover.py:63: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/02.3_subscribed_in_msc: pass (1.3s)
05:21:55.892031 tst handover.py:68: dial success
05:21:58.616259 tst handover.py:75: answer success, call established and ongoing
05:21:58.943767 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
05:21:59.172394 tst handover.py:79: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/03_call_established: pass (3.0s)
05:21:59.324758 run osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
05:22:01.677426 run ssh_sc5_ccli(pid=28596): Launched
05:22:20.911218 run ssh_sc5_ccli(pid=28596): ERR: Terminated: ERROR {rc=255} [trial-160↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:83↪handover.py↪osmo-bts-trx↪sc5-trx↪ssh_sc5_ccli(pid=28596)]
05:22:21.061188 run ssh_sc5_ccli(pid=28596): stdout:
| (launched: 2020-12-07_05:22:01.493234)
05:22:21.209715 run ssh_sc5_ccli(pid=28596): stderr:
| (launched: 2020-12-07_05:22:01.545902)
| ssh: connect to host 10.42.42.112 port 22: No route to host
05:22:21.337199 run sc5-trx: Configuring SC5 TRX failed, retrying 9 more times
05:22:26.700821 run ssh_sc5_ccli(pid=28598): Launched
05:22:29.913550 run ssh_sc5_ccli(pid=28598): Terminated: ok {rc=0}
05:22:30.039540 run osmo-bts-trx: Waiting for sc5-trx to start up...
05:22:31.361300 run osmo-bts-trx(pid=28599): Launched
05:27:32.122492 tst handover.py:84: ----- Report fragment: handover_2G:oc2g+bts1-trx-sysmocell5000/handover.py/04.1_bts1_started: FAIL (332.9s)
05:27:32.130744 tst handover.py:84: ERR: Error: handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-160↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84] [trial-160↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84]
05:27:32.139302 tst handover.py:84: Test FAILED (434.4 sec)
05:27:32.187675 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28447): Terminating (SIGTERM)
05:27:32.240850 run osmo-hlr_10.42.42.2(pid=28448): Terminating (SIGTERM)
05:27:32.296596 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28449): Terminating (SIGTERM)
05:27:32.353095 run osmo-stp_10.42.42.5(pid=28450): Terminating (SIGTERM)
05:27:32.408888 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28451): Terminating (SIGTERM)
05:27:32.464757 run osmo-msc_10.42.42.6(pid=28452): Terminating (SIGTERM)
05:27:32.518454 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28453): Terminating (SIGTERM)
05:27:32.571959 run osmo-mgw_10.42.42.3(pid=28454): Terminating (SIGTERM)
05:27:32.626283 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28455): Terminating (SIGTERM)
05:27:32.683795 run osmo-mgw_10.42.42.4(pid=28456): Terminating (SIGTERM)
05:27:32.736158 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28457): Terminating (SIGTERM)
05:27:32.791652 run osmo-bsc_10.42.42.7(pid=28458): Terminating (SIGTERM)
05:27:32.844627 run osmo-bts-oc2g(pid=28466): Terminating (SIGTERM)
05:27:32.897477 run osmo-bts-trx(pid=28599): Terminating (SIGTERM)
05:27:32.910954 --- ParallelTerminationStrategy: PID 28447 died...
05:27:32.980483 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28447): Terminated: ok {rc=0}
05:27:32.992916 --- ParallelTerminationStrategy: PID 28448 died...
05:27:33.062374 run osmo-hlr_10.42.42.2(pid=28448): Terminated: ok {rc=0}
05:27:33.075256 --- ParallelTerminationStrategy: PID 28449 died...
05:27:33.144782 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28449): Terminated: ok {rc=0}
05:27:33.157236 --- ParallelTerminationStrategy: PID 28450 died...
05:27:33.225586 run osmo-stp_10.42.42.5(pid=28450): Terminated: ok {rc=0}
05:27:33.238013 --- ParallelTerminationStrategy: PID 28451 died...
05:27:33.306783 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28451): Terminated: ok {rc=0}
05:27:33.319208 --- ParallelTerminationStrategy: PID 28453 died...
05:27:33.388883 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28453): Terminated: ok {rc=0}
05:27:33.401931 --- ParallelTerminationStrategy: PID 28454 died...
05:27:33.472203 run osmo-mgw_10.42.42.3(pid=28454): Terminated {rc=15}
05:27:33.484965 --- ParallelTerminationStrategy: PID 28455 died...
05:27:33.555249 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28455): Terminated: ok {rc=0}
05:27:33.567779 --- ParallelTerminationStrategy: PID 28456 died...
05:27:33.637084 run osmo-mgw_10.42.42.4(pid=28456): Terminated {rc=15}
05:27:33.649608 --- ParallelTerminationStrategy: PID 28457 died...
05:27:33.718512 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28457): Terminated: ok {rc=0}
05:27:33.730992 --- ParallelTerminationStrategy: PID 28466 died...
05:27:33.800511 run osmo-bts-oc2g(pid=28466): Terminated {rc=36608}
05:27:35.829469 --- ParallelTerminationStrategy: PID 28452 died...
05:27:35.899277 run osmo-msc_10.42.42.6(pid=28452): Terminated: ok {rc=0}
05:27:36.578314 --- ParallelTerminationStrategy: PID 28458 died...
05:27:36.644645 run osmo-bsc_10.42.42.7(pid=28458): Terminated: ok {rc=0}
05:27:36.656473 --- ParallelTerminationStrategy: PID 28599 died...
05:27:36.720820 run osmo-bts-trx(pid=28599): Terminated: ok {rc=0}
05:27:36.875805 bus /sierra_3: Setting Online False
05:27:38.031585 bus /sierra_3: Setting Powered False
05:27:53.640939 bus /sierra_3: Setting Powered False
05:27:54.761345 bus /sierra_3: Setting Powered False
05:27:55.940721 bus /sierra_2: Setting Online False
05:27:57.084597 bus /sierra_2: Setting Powered False
05:28:08.641143 bus /sierra_2: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 673, in exec_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/sysmocom/suites/handover_2G/handover.py", line 84, in <module>
wait(bsc.bts_is_connected, bts1)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait
raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep)
osmo_gsm_tester.core.log.Error: handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-160↪handover_2G:oc2g+bts1-trx-sysmocell5000↪handover.py:84]