Commit
f6f38151ba83c736210d793dbbb4679802e5cf74
by Neels Hofmeyr
add bsc.vty: always keep a BSC VTY connection ready
The BSC's VTY port will be used to trigger manual handover, and to retrieve a list of active lchans from the BSC, in the upcoming handover_2G test suite, I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Commit
983bf9b289a60b553ac0ccb4e45c281762b5100b
by Neels Hofmeyr
bsc: add active lchan matching via vty
Will be used in upcoming handover_2G test suite in I0b2671304165a1aaae2b386af46fbd8b098e3bd8, which needs to verify that a handover actually ended up on the expected lchan.
Commit
49ef9dc1762d54312b5d439b490079a3f0d0f240
by Neels Hofmeyr
log passed tests
In the end of a test suite, do not omit the passed tests. For example, running handover against N BTS combinations, it was hard to summarize which BTS models actually succeeded, with only the failures listed.
Besides the "FAIL" listings, now print something like this in the end:
Commit
a41b1616c238e55f7377e50a8b11628c636a6a82
by Neels Hofmeyr
configure fixed ARFCN per bts
Remove ARFCNs as a concept from resource pool, assign a fixed ARFCN to each BTS and TRX in the resource pools.
Using ARFCNs on specific bands as resources was an idea that is hard to implement, because specific BTS dictate selection of bands which influences which ARFCNs can be picked. That means reserving ARFCN resources is only possible after reserving specific BTS resources, but the tester is currently not capable of such two-stage resolution.
Writing handover tests, I got the problem that both BTS in a scenario attempt to use the same ARFCN.
The by far easiest solution is to assign one fixed ARFCN to each BTS and TRX. If ever needed, a scenario modifier can still configure different ARFCNs.
(Due to uncertainty about OC2G operation stability, I prefer to leave OC2G on ARFCN 50, as it happened to end up being configured before this patch.)
Commit
bc5a11532f1e7d60260f80e02c0208cff83de95f
by Neels Hofmeyr
process: add get_output_mark() and grep_output()
Allow showing log lines matching specific regexes, from a specific start point of a log.
My use case is to echo the handover related logging after an expected handover failed, so that the reason is visible already in the console output of a jenkins run. So far I would need to open the endless bsc log and look up the matching place in it to get a conclusion about why a handover failed.
Commit
e09d0bf72d5724e92094440d3edffef7e75f0e0d
by Neels Hofmeyr
fix: line nr in test name in wrong places
test.Test() overrides name() in order to provide source line number information. However, overriding name() is the wrong place for that, as name() is also often used for identifying an object - when listing the tests of a suite, the line number should not appear in the test name. For example, the line number sometimes ends up in the test results in jenkins, making 'foo.py' and 'foo.py:23' two distinct report items.
Instead, add a separate function Origin.src() that defaults to name(), but specific classes can override src() if they wish to provide more detailed information with the object name.
Override src() in Test, not name().
Use src() in backtraces.
The suite_test.ok shows that the backtracing in the log remains unchanged, but the place where the test name is printed is corrected: I am 'test_suite' / 'hello_world.py:23' becomes I am 'test_suite' / 'hello_world.py' (Notice that "[LINENR]" in suite_test.ok is a masking of an actual number, done within the selftest suite)
Commit
4ec6203a2fc922c8548c0908f29d79dfeb117bdc
by Neels Hofmeyr
do not redirect_stdout in test scripts
In a test, I called print() on a multi-line string and saw the log showing each line 0.2 seconds apart. redirect.stdout seems to be pretty inefficient.
Instead, put a print() function into the testenv, to directly call log() on the strings passed to print().
The initial idea for redirect_stdout was that we could print() in any deeper functions called from a test script. But we have no such nested print() anywhere, only in test scripts themselves.
As a result of this, a multi-line print() in test scripts now no longer puts the log prefix (timestamp, test name...) and suffix (backtrace / source position) to each single line, but prints the multiline block between a single log prefix and suffix -- exactly like the log() function does everywhere else.
I actually briefly implemented adding the log prefix to each separate line everywhere, but decided that it is not a good idea: in some places we log config file snippets and other lists, and prepending the log prefix to each line makes pasting such a snippet from (say) a jenkins log super cumbersome. And the log prefix (backtrace) attached on each separate line makes multiline blocks very noisy, unreadable.
Commit
eb01700682672b24f7edd7079bda73979d90d1ff
by Neels Hofmeyr
implement test.get_log_output()
Retrieve a test's own logging. The aim is to provide logging belonging to a given report fragment in the junit XML output, will be used by upcoming test.report_fragment() feature.
In this case it is immediately obvious from looking at the jenkins results analyzer that bts1 is the cause of the test failure, and it is visible which parts of the test are flaky, over time.
First user Will be the upcoming handover_2G suite, in I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Commit
7f9c124d83bf15b5563fb888df7fcbdc8d18df35
by Neels Hofmeyr
report fragment: strip ansi colors from junit XML
Jenkins does support showing ANSI colors on the web, but apparently not in the junit results output. Strip ansi colors from report fragment <system-out> text, to make it less annoying to read those on jenkins.
Commit
5390c29aecfe65e18970efed4252484676b1163f
by Neels Hofmeyr
ctrl: set socket timeout
The blocking CTRL socket must have a timeout.
So far we have always been reading from the CTRL socket in a blocking way. So far we are always lucky that we are indeed getting a response to receive. Should the CTRL interface ever fail to answer, the tester would be stuck infinitely.
Commit
69b1f8a64d2e85f62f33c4582ce23e93708c85df
by Neels Hofmeyr
ctrl: increment message IDs for CTRL commands
Implicitly use distinct IDs for CTRL commands. This is a prerequisite for matching GET_REPLY IDs to GET IDs, to correctly handle CTRL responses without counting on being lucky.
Rather use 'use_id' as var name instead of overloading the 'id' keyword.
CTRL interface interaction was mostly inherited from the first legacy implementation of osmo-gsm-tester, and it was a pain to look at from the start. Now, while I'm close to the topic, I want this to improve:
Properly match a GET_REPLY/SET_REPLY to a sent GET/SET by the message ID.
Completely drop the do_get() and do_set(), which were not useful for correct handling of the CTRL request and response messaging. The API to use by callers is set_var(), get_var()/get_int_var() and get_trap(). These call the internal _sendrecv() (or for TRAP only _recv()) functions. Make it so that tese work both on an already connected OsmoCtrl, as well as one that needs to establish a (short) connection, so that both are trivially possible:
# one CTRL connection stays open with OsmoCtrl(...) as ctrl: ctrl.get_var('var1') ctrl.get_var('var2') ctrl.get_var('var3')
and
# get_var() opens a connection, does the GET and closes again OsmoCtrl(...).get_var('var1')
Do away with doubling the instances OsmoCtrl and e.g. OsmoBscCtrl. Rather make OsmoBscCtrl a child class of OsmoCtrl, which means that we no longer have bsc.ctrl().ctrl(), just bsc.ctrl().
Have VERB_* constants instead of dup'd strings.
Apply to / simplify all callers of OsmoCtrl.
Some of these changes are similar to recently added OsmoVty.
Commit
f2c26510c95baa7294ea3b2e9d453a8eea76416f
by Neels Hofmeyr
keep Ctrl connections open for bsc, msc and nitb objects
The pattern to use 'with' to keep a CTRL connection open adds indents to every test script that wants to avoid multiple reconnections to the CTRL. Instead, keeping a single open connection that is cleaned up on {bsc,msc,nitb} object cleanup ensures that a) the program started up successfully and opened a CTRL port, b) always has a CTRL open without having to worry about it and c) keeps test scripts less complex/indented/crufted.
(These are all current users of the OsmoCtrl API.)
Commit
6d564748944379c28b9326a9c2058b10f43b618e
by Neels Hofmeyr
process output: do not leak ANSI colors past logged stdout/stderr
In jenkins, I still saw incidents of the entire log becoming colored after a colored stderr snippet was printed to the log. Make absolutely sure that no unterminated ANSI coloring is leaked.
Commit
a143686448794ddc58d3932334f04ef723697f42
by Neels Hofmeyr
OsmoCtrl: use one global common counter for CTRL IDs
It is easier to traverse debugging logs if the CTRL request and response IDs are globally unique across all programs and tests. Before this, we were using 0 almost everywhere.
(This is not strictly needed for correctness, since each CTRL client has its own request ID scope; just we open fairly many separate CTRL clients all the time in our tests.)
![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
