Commit
af4e2317dc904a31f4ec756c63d2a16830a72d3d
by Neels Hofmeyr
add bsc.vty: always keep a BSC VTY connection ready
The BSC's VTY port will be used to trigger manual handover, and to retrieve a list of active lchans from the BSC, in the upcoming handover_2G test suite, I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Commit
ca155b6c37e16cdcdfd8fa95c4f2ada30b0b3270
by Neels Hofmeyr
bsc: add active lchan matching via vty
Will be used in upcoming handover_2G test suite in I0b2671304165a1aaae2b386af46fbd8b098e3bd8, which needs to verify that a handover actually ended up on the expected lchan.
Commit
7d79ea4d2a4f755850054149af6feb21303fca6c
by Neels Hofmeyr
log passed tests
In the end of a test suite, do not omit the passed tests. For example, running handover against N BTS combinations, it was hard to summarize which BTS models actually succeeded, with only the failures listed.
Besides the "FAIL" listings, now print something like this in the end:
Commit
ecce402f43b708bf178d603e2bd8587e72fdb3b1
by Neels Hofmeyr
configure fixed ARFCN per bts
Remove ARFCNs as a concept from resource pool, assign a fixed ARFCN to each BTS and TRX in the resource pools.
Using ARFCNs on specific bands as resources was an idea that is hard to implement, because specific BTS dictate selection of bands which influences which ARFCNs can be picked. That means reserving ARFCN resources is only possible after reserving specific BTS resources, but the tester is currently not capable of such two-stage resolution.
Writing handover tests, I got the problem that both BTS in a scenario attempt to use the same ARFCN.
The by far easiest solution is to assign one fixed ARFCN to each BTS and TRX. If ever needed, a scenario modifier can still configure different ARFCNs.
(Due to uncertainty about OC2G operation stability, I prefer to leave OC2G on ARFCN 50, as it happened to end up being configured before this patch.)
Commit
fc38393f8278d0ca6db904be142efaa91af7cca3
by Neels Hofmeyr
process: add get_output_mark() and grep_output()
Allow showing log lines matching specific regexes, from a specific start point of a log.
My use case is to echo the handover related logging after an expected handover failed, so that the reason is visible already in the console output of a jenkins run. So far I would need to open the endless bsc log and look up the matching place in it to get a conclusion about why a handover failed.
Commit
081e89f477d3a0209d1dddb652040c43b2b0b6ca
by Neels Hofmeyr
fix: line nr in test name in wrong places
test.Test() overrides name() in order to provide source line number information. However, overriding name() is the wrong place for that, as name() is also often used for identifying an object - when listing the tests of a suite, the line number should not appear in the test name. For example, the line number sometimes ends up in the test results in jenkins, making 'foo.py' and 'foo.py:23' two distinct report items.
Instead, add a separate function Origin.src() that defaults to name(), but specific classes can override src() if they wish to provide more detailed information with the object name.
Override src() in Test, not name().
Use src() in backtraces.
The suite_test.ok shows that the backtracing in the log remains unchanged, but the place where the test name is printed is corrected: I am 'test_suite' / 'hello_world.py:23' becomes I am 'test_suite' / 'hello_world.py' (Notice that "[LINENR]" in suite_test.ok is a masking of an actual number, done within the selftest suite)
Commit
3b493f35678a574a06dfbece61a2deda1bcc73ad
by Neels Hofmeyr
do not redirect_stdout in test scripts
In a test, I called print() on a multi-line string and saw the log showing each line 0.2 seconds apart. redirect.stdout seems to be pretty inefficient.
Instead, put a print() function into the testenv, to directly call log() on the strings passed to print().
The initial idea for redirect_stdout was that we could print() in any deeper functions called from a test script. But we have no such nested print() anywhere, only in test scripts themselves.
As a result of this, a multi-line print() in test scripts now no longer puts the log prefix (timestamp, test name...) and suffix (backtrace / source position) to each single line, but prints the multiline block between a single log prefix and suffix -- exactly like the log() function does everywhere else.
I actually briefly implemented adding the log prefix to each separate line everywhere, but decided that it is not a good idea: in some places we log config file snippets and other lists, and prepending the log prefix to each line makes pasting such a snippet from (say) a jenkins log super cumbersome. And the log prefix (backtrace) attached on each separate line makes multiline blocks very noisy, unreadable.
Commit
12ed99628cf1c5b3bb91e948cb290bc5f27bad98
by Neels Hofmeyr
implement test.get_log_output()
Retrieve a test's own logging. The aim is to provide logging belonging to a given report fragment in the junit XML output, will be used by upcoming test.report_fragment() feature.
In this case it is immediately obvious from looking at the jenkins results analyzer that bts1 is the cause of the test failure, and it is visible which parts of the test are flaky, over time.
First user Will be the upcoming handover_2G suite, in I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Commit
16c8be41c97e5fa7395ca9d83a7562c162e456d1
by Neels Hofmeyr
report fragment: strip ansi colors from junit XML
Jenkins does support showing ANSI colors on the web, but apparently not in the junit results output. Strip ansi colors from report fragment <system-out> text, to make it less annoying to read those on jenkins.
Commit
05439d70296533e79cfe23e7136e4e7a636bc233
by Neels Hofmeyr
ctrl: set socket timeout
The blocking CTRL socket must have a timeout.
So far we have always been reading from the CTRL socket in a blocking way. So far we are always lucky that we are indeed getting a response to receive. Should the CTRL interface ever fail to answer, the tester would be stuck infinitely.
Commit
f79a86fc2583be35d35bd7e7cee3899dd8d55b1f
by Neels Hofmeyr
ctrl: increment message IDs for CTRL commands
Implicitly use distinct IDs for CTRL commands. This is a prerequisite for matching GET_REPLY IDs to GET IDs, to correctly handle CTRL responses without counting on being lucky.
Rather use 'use_id' as var name instead of overloading the 'id' keyword.
CTRL interface interaction was mostly inherited from the first legacy implementation of osmo-gsm-tester, and it was a pain to look at from the start. Now, while I'm close to the topic, I want this to improve:
Properly match a GET_REPLY/SET_REPLY to a sent GET/SET by the message ID.
Completely drop the do_get() and do_set(), which were not useful for correct handling of the CTRL request and response messaging. The API to use by callers is set_var(), get_var()/get_int_var() and get_trap(). These call the internal _sendrecv() (or for TRAP only _recv()) functions. Make it so that tese work both on an already connected OsmoCtrl, as well as one that needs to establish a (short) connection, so that both are trivially possible:
# one CTRL connection stays open with OsmoCtrl(...) as ctrl: ctrl.get_var('var1') ctrl.get_var('var2') ctrl.get_var('var3')
and
# get_var() opens a connection, does the GET and closes again OsmoCtrl(...).get_var('var1')
Do away with doubling the instances OsmoCtrl and e.g. OsmoBscCtrl. Rather make OsmoBscCtrl a child class of OsmoCtrl, which means that we no longer have bsc.ctrl().ctrl(), just bsc.ctrl().
Have VERB_* constants instead of dup'd strings.
Apply to / simplify all callers of OsmoCtrl.
Some of these changes are similar to recently added OsmoVty.
Commit
012a17da6a0c0628f890786d1b6c4a5ecb18cccd
by Neels Hofmeyr
keep Ctrl connections open for bsc, msc and nitb objects
The pattern to use 'with' to keep a CTRL connection open adds indents to every test script that wants to avoid multiple reconnections to the CTRL. Instead, keeping a single open connection that is cleaned up on {bsc,msc,nitb} object cleanup ensures that a) the program started up successfully and opened a CTRL port, b) always has a CTRL open without having to worry about it and c) keeps test scripts less complex/indented/crufted.
(These are all current users of the OsmoCtrl API.)
Commit
fb8c02fbd5419f37b5261f3166d8b3b2970e599f
by Neels Hofmeyr
process output: do not leak ANSI colors past logged stdout/stderr
In jenkins, I still saw incidents of the entire log becoming colored after a colored stderr snippet was printed to the log. Make absolutely sure that no unterminated ANSI coloring is leaked.
Commit
f80f7cc5c46bddf63c0f41042f28e7f62aec45af
by Neels Hofmeyr
OsmoCtrl: use one global common counter for CTRL IDs
It is easier to traverse debugging logs if the CTRL request and response IDs are globally unique across all programs and tests. Before this, we were using 0 almost everywhere.
(This is not strictly needed for correctness, since each CTRL client has its own request ID scope; just we open fairly many separate CTRL clients all the time in our tests.)
Commit
4cd6f14be2908230ad52d138d8a9697103c47eca
by andre
jenkins-build-srslte.sh: allow custom binaries
this allows to specify the environmental var wanted_binaries_bin that will be included in the trial package. if the variable isn't set, all three main srsLTE binaries are included
Commit
2df63d0f2fb2065633c4d7e1038a61803989d6ed
by andre
enb_srs: adopt throughput thresholds for QAM256
it turned out that we have to reduce the MCS when using QAM256, especially for 6 PRB as subframe 0 and 5 contains PBCH and PSS signals, so the available REs are reduced.
The eNB scheduler now has this limitation in mind and lowers the MCS.
Commit
9c7fd618d226dc532b9408c329cfc2380de4850a
by andre
srsepc.conf.tmpl: increase default value of T3413 (paging timer)
we've seen that the default value of 2s might be a bit tight. the value is network dependent so no real default value exists. however, 6s seems to be a good compromise.
Commit
a8263f40e9c3baf4a6ba34b213ff2fe87c0993fd
by nils.fuerste
Introduce Android UEs as new modems
To expand the test capacities we would like to introduce Android UEs as new modems. Currently the following tests are supported: - Ping - iPerf3 DL/UL - RRC Mobile MT Ping
Infrastructure explaination: The Android UEs are connected to the OGT Units via USB. We activate tethering and set up a SSH server (with Dropbear). We chose tethering over WiFi to have a more stable route for the ssh connection. We forward incoming connections to the OGT unit hosting the Android UE(s) on specific ports to the UEs via iptables. This enables OGT to issue commands directly to the UEs. In case of local execution we use ADB to issue commands to the AndroidUE. The set up was tested with 5 Android UEs connected in parallel but it should be scalable to the number of available IPs in the respective subnet. Furthermore, we need to cross compile Dropbear and iPerf3 to use them on the UEs. These tools have to be added to the $PATH variable of the UEs.
Examplary set up: In this example we have two separate OGT units (master and slave) and two Android UEs that are connected to the slave unit. An illustration may be found here: https://ibb.co/6BXSP2C
On UE 1: ip address add 192.168.42.130/24 dev rndis0 ip route add 192.168.42.0/24 dev rndis0 table local_network dropbearmulti dropbear -F -E -p 130 -R -T /data/local/tmp/authorized_keys -U 0 -G 0 -N root -A
On UE 2: ip address add 192.168.42.131/24 dev rndis0 ip route add 192.168.42.0/24 dev rndis0 table local_network dropbearmulti dropbear -F -E -p 131 -R -T /data/local/tmp/authorized_keys -U 0 -G 0 -N root -A
On OGT slave unit: sudo ip link add name ogt type bridge sudo ip l set eth0 master ogt sudo ip l set enp0s20f0u1 master ogt sudo ip l set enp0s20f0u2 master ogt sudo ip a a 192.168.42.1/24 dev ogt sudo ip link set ogt up
Now we have to manually connect to every UE from OGT Master to set up SSH keys and verify that the setup works. Therefore, use: ssh -p [UE-PORT] root@[OGT SLAVE UNIT's IP]
Finally, to finish the setup procedure create the remote_run_dir for Android UEs on the slave unit like following: mkdir /osmo-gsm-tester-androidue chown jenkins /osmo-gsm-tester-androidue
Commit
f249a026a06812c667e037eb1b609d48922e13d7
by andre
enb,ue,process: move stdout counter helper to process objects
this patch adds the stdout counter to count events happening on the stdout (known from the UE already) to the common process class so they can also be used from the eNB (and other objects)
In addition, we add a PRACH counter to be used for tests.
Commit
29263b7389520ec2269d6852a09f96ef8da79573
by andre
4g: add checks for received PRACHs for ping tests
the tests already check for sent PRACHs, this will also check for received PRACHs on the eNB to make sure we don't detect any fake PRACHs during the tests.
Commit
a81801558e7bb6ef0f0a32ab078d302c91e940a8
by andre
enb: Set cipher and integrity via scenario file
This patch enables setting cipher and integrity algorithms in Amarisoft eNB and srsENB via scenario files. If no settings are defined following defaults are applied: - Cipher algorithm: EEA0, EEA2, EEA1 - Integrity algorithm: EIA2, EIA1, EIA0
Example of setting cipher algorithms: - 4g:srsue-rftype@uhd+srsenb-rftype@uhd+mod-enb-cipher@eea1+mod-enb-cipher@eea0+mod-enb-nprb@6
Commit
b79a0c5e75dda5c08f8fa5dd9bcd6a8bcea106c7
by andre
scenarios: remove ncells@ scenario and define number of cells explicitly
the specific cell scenarios files already contain cell-related information so having a extra ncells scenario seems too much overhead. furthermore, in more complex tests its helps to reduce the filename/path lenght to avoid hitting the 255 character limit
Commit
7aa14e45f455f87c1dfe9966d759093af9f6d6fc
by andre
srsue.conf.tmpl: reduce nof PHY threads to 2 for ZMQ
we've detected a possible race condition during the Msg3 transmission that caused the thread that sets the Msg3 grant to be delayed. The PHY worker that executed TTI+2 finished by that time already and didn't even see the pending UL grant.
This is issue is more likely to happen on loaded system, for example when running parallel ZMQ jobs. We therefore decided to reduce the number of parallel PHY workers to 2 so the issue is circumvented.
![[Jenkins]](/jenkins/static/342c126a/images/svgs/logo.svg){kind=logo}
