Commit
765b87f0d075e5a515d10874025c2aba6f40b2cd
by Neels Hofmeyr
add bsc.vty()
The BSC's VTY port will be used to trigger manual handover, and to retrieve a list of active lchans from the BSC, in the upcoming handover_2G test suite, I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Commit
c68ad760ee104b79843abaa9f0a555a55ab2d047
by Neels Hofmeyr
bsc: add active lchan matching via vty
Will be used in upcoming handover_2G test suite in I0b2671304165a1aaae2b386af46fbd8b098e3bd8, which needs to verify that a handover actually ended up on the expected lchan.
Commit
cf1c5ee1c9a7af2cec2714ee3895587af6e9132b
by Neels Hofmeyr
log passed tests
In the end of a test suite, do not omit the passed tests. For example, running handover against N BTS combinations, it was hard to summarize which BTS models actually succeeded, with only the failures listed.
Besides the "FAIL" listings, now print something like this in the end:
Commit
5d69f056b4530d325bdd8a25640ffd5bacee82d0
by Neels Hofmeyr
configure fixed ARFCN per bts
Remove ARFCNs as a concept from resource pool, assign a fixed ARFCN to each BTS and TRX in the resource pools.
Using ARFCNs on specific bands as resources was an idea that is hard to implement, because specific BTS dictate selection of bands which influences which ARFCNs can be picked. That means reserving ARFCN resources is only possible after reserving specific BTS resources, but the tester is currently not capable of such two-stage resolution.
Writing handover tests, I got the problem that both BTS in a scenario attempt to use the same ARFCN.
The by far easiest solution is to assign one fixed ARFCN to each BTS and TRX. If ever needed, a scenario modifier can still configure different ARFCNs.
(Due to uncertainty about OC2G operation stability, I prefer to leave OC2G on ARFCN 50, as it happened to end up being configured before this patch.)
Commit
2e4091136a5a6999572b284a295b853f0b017935
by Neels Hofmeyr
process: add get_output_mark() and grep_output()
Allow showing log lines matching specific regexes, from a specific start point of a log.
My use case is to echo the handover related logging after an expected handover failed, so that the reason is visible already in the console output of a jenkins run. So far I would need to open the endless bsc log and look up the matching place in it to get a conclusion about why a handover failed.
Commit
46e3a5f35460e68d729f6732d52252caa59ba993
by Neels Hofmeyr
fix: line nr in test name in wrong places
test.Test() overrides name() in order to provide source line number information. However, overriding name() is the wrong place for that, as name() is also often used for identifying an object - when listing the tests of a suite, the line number should not appear in the test name. For example, the line number sometimes ends up in the test results in jenkins, making 'foo.py' and 'foo.py:23' two distinct report items.
Instead, add a separate function Origin.src() that defaults to name(), but specific classes can override src() if they wish to provide more detailed information with the object name.
Override src() in Test, not name().
Use src() in backtraces.
The suite_test.ok shows that the backtracing in the log remains unchanged, but the place where the test name is printed is corrected: I am 'test_suite' / 'hello_world.py:23' becomes I am 'test_suite' / 'hello_world.py' (Notice that "[LINENR]" in suite_test.ok is a masking of an actual number, done within the selftest suite)
Commit
8f0e8e5c14e232faf9f6574a245cf5cbb38e0c34
by Neels Hofmeyr
do not redirect_stdout in test scripts
In a test, I called print() on a multi-line string and saw the log showing each line 0.2 seconds apart. redirect.stdout seems to be pretty inefficient.
Instead, put a print() function into the testenv, to directly call log() on the strings passed to print().
The initial idea for redirect_stdout was that we could print() in any deeper functions called from a test script. But we have no such nested print() anywhere, only in test scripts themselves.
As a result of this, a multi-line print() in test scripts now no longer puts the log prefix (timestamp, test name...) and suffix (backtrace / source position) to each single line, but prints the multiline block between a single log prefix and suffix -- exactly like the log() function does everywhere else.
I actually briefly implemented adding the log prefix to each separate line everywhere, but decided that it is not a good idea: in some places we log config file snippets and other lists, and prepending the log prefix to each line makes pasting such a snippet from (say) a jenkins log super cumbersome. And the log prefix (backtrace) attached on each separate line makes multiline blocks very noisy, unreadable.
Commit
db444e4ceed32c4a56865bb56595c7cf101fa48c
by Neels Hofmeyr
implement test.get_log_output()
Retrieve a test's own logging. The aim is to provide logging belonging to a given report fragment in the junit XML output, will be used by upcoming test.report_fragment() feature.
In this case it is immediately obvious from looking at the jenkins results analyzer that bts1 is the cause of the test failure, and it is visible which parts of the test are flaky, over time.
First user Will be the upcoming handover_2G suite, in I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Commit
466859d1f03677659d3748c600c1e71dfeb485ad
by Neels Hofmeyr
report fragment: strip ansi colors from junit XML
Jenkins does support showing ANSI colors on the web, but apparently not in the junit results output. Strip ansi colors from report fragment <system-out> text, to make it less annoying to read those on jenkins.
Commit
3394a7fd5f4332c91b4dcc50d817b931dc3256ce
by Neels Hofmeyr
ctrl: set socket timeout
The blocking CTRL socket must have a timeout.
So far we have always been reading from the CTRL socket in a blocking way. So far we are always lucky that we are indeed getting a response to receive. Should the CTRL interface ever fail to answer, the tester would be stuck infinitely.
Commit
bd88c8d9faffc0f3d3cf2a5f1c0ee8bf7e9069ed
by Neels Hofmeyr
ctrl: increment message IDs for CTRL commands
Implicitly use distinct IDs for CTRL commands. This is a prerequisite for matching GET_REPLY IDs to GET IDs, to correctly handle CTRL responses without counting on being lucky.
Rather use 'use_id' as var name instead of overloading the 'id' keyword.
CTRL interface interaction was mostly inherited from the first legacy implementation of osmo-gsm-tester, and it was a pain to look at from the start. Now, while I'm close to the topic, I want this to improve:
Properly match a GET_REPLY/SET_REPLY to a sent GET/SET by the message ID.
Completely drop the do_get() and do_set(), which were not useful for correct handling of the CTRL request and response messaging. The API to use by callers is set_var(), get_var()/get_int_var() and get_trap(). These call the internal _sendrecv() (or for TRAP only _recv()) functions. Make it so that tese work both on an already connected OsmoCtrl, as well as one that needs to establish a (short) connection, so that both are trivially possible:
# one CTRL connection stays open with OsmoCtrl(...) as ctrl: ctrl.get_var('var1') ctrl.get_var('var2') ctrl.get_var('var3')
and
# get_var() opens a connection, does the GET and closes again OsmoCtrl(...).get_var('var1')
Do away with doubling the instances OsmoCtrl and e.g. OsmoBscCtrl. Rather make OsmoBscCtrl a child class of OsmoCtrl, which means that we no longer have bsc.ctrl().ctrl(), just bsc.ctrl().
Have VERB_* constants instead of dup'd strings.
Apply to / simplify all callers of OsmoCtrl.
Some of these changes are similar to recently added OsmoVty.
Commit
8f3d4ad85c81eb1d898fd15aaf568a32eff7552f
by Neels Hofmeyr
process output: do not leak ANSI colors past logged stdout/stderr
In jenkins, I still saw incidents of the entire log becoming colored after a colored stderr snippet was printed to the log. Make absolutely sure that no unterminated ANSI coloring is leaked.
Commit
2d8d51cd3e5db65679cba2129e514f4b2f049b7f
by Neels Hofmeyr
OsmoCtrl: use one global common counter for CTRL IDs
It is easier to traverse debugging logs if the CTRL request and response IDs are globally unique across all programs and tests. Before this, we were using 0 almost everywhere.
(This is not strictly needed for correctness, since each CTRL client has its own request ID scope; just we open fairly many separate CTRL clients all the time in our tests.)
![[Jenkins]](/jenkins/static/342c126a/images/svgs/logo.svg){kind=logo}
