Changes

osmo-msc: Use osmo-mgw instead of osmo-bsc_mgcp
osmo-msc now requires osmo-mgw and it's not expected to work properly
anymore with old mgcp code.
Change-Id: I93033f7b6133a4914f12f37511ad870b3dea3201

osmo-trx: Add cfg template
Since commit 3da1f8352e337fb032bf7a58c2909d3ba918e237, osmo-trx requires
a cfg file to start successfully.
Change-Id: I8f0e0d0a9fa849f41c3cccc4059431f62f956eac

jenkins: use env POKY_VERSION POKY_PATH to find poky
Instead of hardcoded values, use defaults and allow jenkins to override
the defaults.
Change-Id: I1403b32cdf30420f3b02c41c2627b71fcbeae9ab

jenkins: update poky version 2.3.2
The sysmobts 2017 releases are using poky 2.3.2
Change-Id: If6e8a5702e4c9b24e758e3aa5f390e786c51c384

contrib: build-osmo-bts: Remove dropped dependency openbsc
Since osmo-bts ec33b0397f5d71248c5834513d4be7b9b0e46366, it doesn't
require openbsc anymore to build.
Change-Id: I8a1918971388afad41308629c1851614d1381f25

contrib: build-osmo-bsc: Add aibsip-find and ipaccess-config binaries to
archive
They will be required by Nanobts class.
Change-Id: Ib0e003f74603c3146aa76d581ab493f960f73ab5

modem: get IMSI from ofono
There's no need to specify the IMSI manually in resource config and it's
also prone to errors. Let's take it from ofono. Add a 'sim' feature to
allow modem to auto-discover it, otherwise if not supported leave that
feature out of the config for that modem and an imsi can still be
manually providen.
Change-Id: I20f9e8d97775293925205e4ea576d814214bf1a8

example: Maintain different resources.conf for RnD and prod
We still want to maintain this file in the same osmo-gsm-tester repo
because we frequently neef to update the config when adding new
features.
Until now only 1 file was maintained (which was used for RnD setup), and
then when runnin in prod the jenkins script used sed to change the file
to accomodate slightly changes. This way is too hacky, so let's just
maintain too separate files, keeping the original resources.conf key
name used by osmo-gsm-tester free, so that jenkins job can symlink one
of the 2 files to it.
Take the chance to remove OctoBTS and Sysmocell5k from the RnD resources
file, as we don't have those them.
Change-Id: Ifec851c7ac6fca6b294e57dfe86b92f214ae8f42

modem: workaround ofono crash
Since commit bfd0b2310cf09e32cb7d5dbe74ec57606a7d2aab, the IMSI is
retreived from ofono. To get the IMSI, the modme must be powered ON
previously. The imsi() method is called before calling connect() in
order to set up the IMSI in the HLR. As a result, the following
behaviour occurs:
- Powered=true (in imsi())
- SimManager.GetProperties() (several of them)
- Powered=false (power_cycle() in connect()
It seems powering off the modem immediately after using the SIM service
makes ofono crash. Proof of it is that crashes were seen mainly on tests
using only 1 MS, in which we don't spend that much time between
SimManager.GetProperties() and Powered=false for the same modem.
Let's workaround the crash for now increasing the time between using the
SIM services and powering the modem off.
Related: OS#3064
Change-Id: Ief052cac5a862d6ef9391d40c294ba017387506c

Create Pcu abstract class and make OsmoPcu inherit from it
This base class will be used to describe the required accessors for all
PCU objects.
It is introduced in this commit and will be further used in the future
when adding a Dummy PCU object which will be used by NanoBts object.
Change-Id: Ia3fd4551d1f2932362f99f7d44d65f8ae4fd1979

bts_osmo.py: ready_for_pcu is of public access from tests, move it
accordingly
Change-Id: Ie8978854404897ae5de7e3100d55d86c7a5c1df1

Create Bts abstract class and make OsmoBts inherit from it
This base class will be used to describe the required accessors for all
BTS objects, be it an osmocom BTS or not.
It is introduced in this commit and will be further used in the future
when adding a NanoBts object.
Change-Id: Ic13133e61abda73a8b507c1a1bd7b98c677460f9

pcu.py: Introduce PcuDummy class
It will be used in later commits by BTS without proper PCU control such
as NanoBts.
Change-Id: I37ba38f7d81134e5f0ca28fa684fdb09c753bb04

Introduce PowerSupply interface and PowerSupplySispm
File powersupply.py defines the interface to be used by child classes
implementing it. It also provides helpers to allocate a child class
based on configuration provided ('type' field).
File powersupply_sispm.py is an implementation using pysispm [1], as
it's the one used to control the programmable power socket we have right
now.
This kind of class will be used in later commits by Nanobts class, as we
want to poweroff the Nanobts completelly when not in use.
Using it requires the following extra dependencies:
$ apt-get install python3-usb
$ pip3 install pysispm
Related: OS#3040
[1] https://github.com/xypron/pysispm
Change-Id: I981c260eca1a61657147e6d83b4226618088223c

Introduce ip.access nanobts support
Change-Id: Ibaea025b3a503dfe897d36701234445de6d49f82

default-suites.conf: Add nanobts related tests
Change-Id: I054db78e6a71514133e4e3c3158342b74af44201

nanobts: Bind to specific addr instead of specific iface
This way root access is not required.
Change-Id: I02f6fc9e1cb1ea1e1d950eafe22c231a18100f98

schema: Accept band GSM-900
Change-Id: I274c3010f33776b2967ee01de027c44b794cc5d4

example: Add scenarios to select BTS for a specific band
Change-Id: Id830536c9337151818901cb25c3a568a77cdbf5e

nanobts: Add support to run nanobts on the 900 band
Change-Id: I13b494630ea835f4bed38ccc77f6a4d5e18326af

example: Run tests on both nanobts with different bands
Change-Id: Iee2220e1770caa891713dbd18e8f4a4741c1fbe4

templates: osmo-bsc: Remove no longer available cfgs
osmo-bsc c74a5616bf1c77ac40ddd92f4927dca7a1b45bc8 removed these cfg
options as they are dead code.
Change-Id: I2fb63d31d5d7424b48ed2cfc694d09db072db485

event_loop: Use glib as mainloop impl and move modem to use event_loop
Several benefits:
- We can add APIs to poll on fds in the future (for smpp socket for
instance) instead of using busy polling.
- During wait(), we now block in the glib mainloop instead of sleeping
0.1 secs and not handling events during that time.
- We remove glib mainloop specific bits from modem.py
Change-Id: I8c3bc44bbe443703077110cdc67207e9cbb43767

Drop event_loop global functions and import MainLoop
Take the chance to identify and drop modules importing event_loop but
not using it.
Change-Id: Ifa1940cd00138ebc72fbcfd45a0e28246f7a5ed9

nanobts: Attempt at fixing race condition using ipaccess-config after
power up
It seems there's a period of time in between the nanoBTS starts
announcing itself and it can actually be managed by the BSC
(ipaccess-config in this case). If ipaccess-config is called quickly
after nanoBTS it first announced, its TCP conn will be rejected with a
TCO RST, and ipaccess-config will fail. Let's see if waiting a few
seconds is enough to have this working more reliably.
Change-Id: Iec8adb2cd13d808a6ca1843dc95b81bd1a97d1c0

ms: Create a cumulative distribution function class
We are using the CDF to decide which percentage of the jobs should be
running at a given point. The x-axis is time and the y-axis the
percentage of how many jobs should be running.
There are three functions to do this. The first one is a constant which
would result in everything being started right now, one to start them
linearly and the last (formula from Qt/3rdparty) to first accelerate and
decelerate slowly.
Change-Id: I9e3064f4c3c4c7af5d3491f850090516e541f4d3

ms: Create a simple epoll (or kqueue) based event loop
Create a C-like single process event loop. It could be powered by
select/epoll or kqueue. It should scale to many open fds but we will not
have that many.
Change-Id: Iea06f33870cab9f21e9a1a1feb9758467343dd29

ms: Create an event server to handle Unix datagram messages
Create an EventServer that will create a unix domain socket and dispatch
incoming datagram messages. The lua remotes are not passing credentials
so this is a one way communication channel for now. Tests can register
to handle the message.
Change-Id: Ida97c570e8e741410f2dba4a231a8058ca96da25

ms: Add lua script support utilities
Add a JSON encoder and a small module to sent registration and other
events per unix datagram socket.
json.lua fetched using:
$ wget -O src/osmo_ms_driver/lua/json.lua \
https://raw.githubusercontent.com/rxi/json.lua/master/json.lua
Change-Id: I43ae84a944c7f33e41d5de0880d4aaab3378809b

ms: Lua part of location update testing
The lua part to start the MS and then signal the first successful
Location Update.
Change-Id: Ica5aa0c2f86d0e5d8a2bc4dc0652de18762dd156

ms: Create template for the osmocom-bb mobile application
Change-Id: I9296f42edfab57762f8dd317d63231298cda5430

ms: Create a starter for virtphy and mobile application
In the long run we might not want to start the virtphy but for now
virtphy+mobile belong together. Start virtphy first as mobile will not
handle a missing socket gracefully.
Change-Id: I5c6d742842d7f3e0a1858436ef3f8634d8c0582d

ms: Add a first test to use all parts of the system
This is an interim solution but is bringing all parts together. We will
need to:
* Abstract this into a base class
* Be able to mix different tests without interfering with each
  other (e.g. 10k LU tests, 2k SMS sending)
* The event loop will need to handle multiple timers/timeouts
* Stats printing should print more information and test pass/fail
* The test should quit early if everything has already passed
Change-Id: Id3277ed0f0f9ee734569bedd4752564eb68c9cfd

ms: Add a main function to start all of it
Add a main file to start everything. All parameters need to be made
configurable. Composition of testcase and IMSI ranges need to be
configurable as well. This is left for future commits. Right now it can
execute a single UL test.
Start with: export PATH=../osmocom-bb/src/host/layer23/src/mobile:$PATH
export PATH=../osmocom-bb/src/host/virt_phy/src/:$PATH export
PYTHONPATH=$PWD/src
python3 -mosmo_ms_driver
Change-Id: I58c938500a067eebb213750e56d8bf4d8af43df2

event_loop: Avoid leaking active timeouts on poll exception
It was spotted that when an error ocurred during poll generating an
exception to finish the test (osmo-msc ended prematurely), then the
active wait() condition was kept being checked for later tests, making
all followup tests fail.
That's because in case of exception we were not making sure the timeout
is removed and it kept firing forever.
Change-Id: I5c93cd67b2b83c0ecee96fbc6c4200b419bdf73f

contrib: Enable address-sanitize in builds
Don't enable it for binaries built with sysmocom's cross-toolchain since
it doesn't contain required libasan.
Also, don't enable it on openbsc since we don't have an enable-sanitize
flag there.
Change-Id: Ibef6c2fc497bce0d799116a854a48872f9ca16d7

contrib: build-osm-bts-sysmo: openbsc dep no longer needed
Change-Id: Iafd5ee1df001719ff6165ada996359b92116671e

contrib: build-osmo-trx: Enable addr sanitize for osmo-trx
Change-Id: Iad90c7c8c2098680276de000c36bec0224f9746d

build-osmo-trx: Disable asan in osmo-trx builds
Change-Id: Ic4ebbc3f66fd9de2343d486ed20a0f08fc11e326

modem: Catch exception: call removed while waiting to become active
This can happen while in a test we use: wait(ms_mo.call_is_active,
mo_cid)
And then answer fails for whatever reason, after a timeout ofono will
remove the call object:
/sierra_2: DBG: 'org.ofono.VoiceCallManager'.CallRemoved() ->
/sierra_2/voicecall01
As a result, during next call o call_is_active() will try to get the
call object, but it doesn't exist anymore and an exception will be
created in method call_state during call to systembus_get().
Change-Id: I02b7e76425754372756493761819f18f1e3106c1

osmo-trx binary is now called osmo-trx-uhd
Change-Id: Ibf111ba89b24964127b9c1db7102953e1dba3181

bts_osmotrx: Don't use binary name as inst name
Change-Id: Iee20738f2aa750aa9035a9e54ad667d8753ea12e

Check physical channel configs
Change-Id: I5894678e0d6f306173cc550dd6806a7d6f8a4c11

bts: Refactor conf_for_bsc_osmo
Move duplicated code into a method in the superclass.
Change-Id: Ie27932f94142f667c3fb8c054b77e04afa0d5cbb

bts: Pass defaults config name at construct time
It will be needed later at construction time when resolving the num of
trx.
Change-Id: I2c6a46ead14010f714897ea178917327215823e7

bts: Move conf_for_bsc_prepare from bts_osmo to bts
nanobts inherits directly from bts.Bts, so we need to move
conf_for_bsc_prepare there to make nanobts work again.
Fixes: 530681fd86c4a4dab18e4575c9cdf339623ffa49 Change-Id:
I4305eefb4056ed7614118532e271efe780f4a87f

bts: Allow setting amount of TRX and timeslot in cfg and from test at
runtime
The num_trx attribute for a given BTS states the number of TRX to be
used by that BTS. If more than num_trx are configured in trx_list in the
cfg file, then only up to num_trx are taken into account. If a num_trx
value higher than max_trx is specified throuygh config file or at
runtime by the test, an exception is raised explaining the issue.
The num/max_trx attributes are overlayed along the config levels
(generic -> bsc_bts -> specific bts-type -> specific resource object).
This way we can specify a long list of trx+timeslot config in the
generic config (bsc_bts), and tune for each model and specific BTS which
is the desired default number of TRX, as well as the maximum supported
per type.
Change-Id: I7f46eaf7a16f03268653299c93600c0443f691ac

suite: Set suite subdir lib part of syspath so tests can import their
own modules
Change-Id: I3307c3ab314bb7a88d3c22c7c270c9f05fe6e03e

tests: voice: Move logic to lib/testlib.py
This way we can have several tests sharing similar logic.
Change-Id: I5cfcc56970380a7c5400186fac5f504263d63c5f

tests: voice: test voice calls on TCH/H timeslots
Change-Id: I897757a21fee516ce96808a180121b09f06a0fea

tests: voice: Move mo_mt_call.py to mo_mt_call_tchf.py
Change-Id: I7306e62ad4c4294370690cf9006871f798a7b9ec

tests: dyn_ts_*: Add tests to verify dynamic timeslots during phone
calls
Two different test suites are created since we want to run them with
different HW, because some HW support one type of PDCH channels, and
other HW supports the other one.
Change-Id: Id5e61eaff39ac7a6585dc7de2aeb2469dd2ce726

default-suites.conf: Add dyn ts suites
Change-Id: Iec7c8d8c3617f11a5ee12c02bd75ed6822d76aa9

default-suites.conf: Fix typo in previous commit
Change-Id: Ib1903324df8eb9f506b62dc0b9d2aa4a4a5179d7

suites: Remove rebase artifacts from latest commits
Change-Id: I8afc446d4291810c63bfaeabb7167113667dd8d1

Unload suite local modules after suite exit to avoid collisions
Since sys.path is modified idynamically to load modules from "lib"
subdir of each suite, from python env point of view all those modules
share a namespace. As a result, there can be name collisions.
If a name collision appears (eg test1 loads "testlib.py" and test2
afterwards also loads its own "testlib.py"), then python interpreter
thinks the testlib.py module is already loaded, so test2 ends up using
"testlib.py" from test1.
The way to solve this is to make suite local modules to live only
through the scope of the suite, and unload the modules once the suite is
finished.
Change-Id: I4efe815f85bc4ec2ca91aa9c2d3a369048f21571

util: Fix unloading of suite lib import path
Change-Id: Iac81151f5baec092a3d15bbe6541b5bc007e6a03

tests: dyn_ts_*: Add tests to verify dyn ts with gprs disabled
Change-Id: If943ae0fb4e56787df02b09dea7387cb13eb9db3

contrib: common: Record repo curr rev in have_repo instead of build_repo
This way scripts making use only of have_repo (to have build specific
insturctions) can still benefit from that code.
Change-Id: I03149942ac1d279a15c71df3eb880c331c8fc155

contrib: common: create_bin_tgz supports sbin dir now
This is a preparation for inclusion of osmocom-bb build job, which
installs binaries such as osmocon in sbin.
Change-Id: I562721af82e481e926ee65dbcd9dcc5b55057ae8

contrib: Add jenkins-build-osmocom-bb.sh to build osmocon
Change-Id: Ic5b29951d3fd143dce5826138c84301a9797b1ef

testenv: Provide tests with access to process module
Some tests may want to start/stop/manages processes themselves. By using
the process module from osmo-gsm-tester, we make that possible easily,
providing with useful features such as requesting a suite to stop it on
cleanup.
Change-Id: I8e018107cbec81299ec2228bfff933fb3b0bb6cd

contrib: build-osmocom-bb: Download and install osmocom-bb fw from OBS
As it's quite complex to build osmocom firmware nowadays (a specific old
release of gcc and binutils are required to build it), instead download
the already built binaries from OBS.
Change-Id: I2571268d91e2430a270e51438608e7dbd121e76d

templates: osmo-bts: Use BTS assigned IP addr to bind CTRL and VTY iface
Change-Id: Ib1c3022a6c531ef8c6a5ba31927762806abc37ac

bsc: Add API to mock BSC RSL IP addr
Change-Id: I9ca67964e656943b50967a27a88054fd3e484672

process: Move cleanup log inside cleanup method
Change-Id: I72ad453e9dc8340946d7e7d041643fe8ef97439f

Add option to expect bts/pcu failures and respawn its processes
Some tests may want to reproduce some scenarios in which it is expected
that a BTS process is stopped, for instance if the BSC link is dropped.
Provide a keepalive parameter to start() for bts and pcu objects to
inform suite that failures are expected and that it should keep them
alive in case that ocurrs by respawning the BTS process.
Change-Id: Ia2a7539f9fad457125ac9b60a52a52999e885ba8

osmo-bsc.cfg.tmpl: Remove unused nd recently removed option dest
This VTY cmd has been removed in osmo-bsc.git
68e4be9c845e25f53aea511b1c57340a9da5825c.
Change-Id: I48c0b8a7b2a0088887f1309efc40bcf1da0f1626

powersupply_sispm: Add port number to log name
This way it can easily be spotted which port is being triggered in case
someone wants to manually test it.
Change-Id: Ibd2c70170d97a786e37b6838b1ed6d4eb637ad8b

event_loop: Log parameters during wait timeout exception
Change-Id: I1cf104392bfd11ad263264738eb4d8fa60550a5d

osmo_ms_driver: Use the two new config options for the mobile
Avoid being stuck in c7 state and use the c7-any-timeout with a low
value. Do not attempt to store SMS in the home directory (if they are
received).
Change-Id: I4bb59130a04852bc43af9ef3602b129f40e9e170

ggsn: Bind VTY and CTRL ifaces to assigned ip addr
Change-Id: I26735a21cd53183dfe0a75e5bd24954d583c354f

ms: Use the new API to pass credentials to the event server
We are using the "autobind" feature of Linux to get a special socket
address. This allows us to have roughly 2^20 clients on the system.
Change-Id: Ie9b9ac6267f40345baf7dbb3becaecf264a5df5f

resources.conf: Move nanobts address to differentiate them from local
IPs
nanobts IP addresses are assigned through DHCP, and are not local to the
main unit. Let's use another subset for this DHCP pool as we usually use
.50ish for static local IP addresses.
Change-Id: Ibdb0dd97a490aaa555a7bf53cf43cc5a5533a012