Changes

suites/gprs/: beautify output result of ping tests

Print ping output at the end and replace the whole test stdout report
with ping execution.

Change-Id: I90001ddc6bc1145ac9d115aee762f4f3500e8dc8

epc: Replace rlc_drb_mode with qci attribute

Change-Id: I21b7d7009f9b0df8ec4f59189a65cfc26f164f04

amarisoft_enb: Support zmq rf_driver

Change-Id: I958834d99584e33f632b839a0bd0acccc2379049

amarisoft_rf_driver_temp: fix tx time offset for ZMQ driver

Change-Id: Icee282837e62e02a17b93efba4c22756efa2a5d3

amarisoft_enb: add handling of num_cells modifier

the num_cells modifier allows to create multiple cells in a single enb
both cells are by default configured for CA

Change-Id: Ie9f16ce06ee56002c01097f2d338278f09ea4337

srsue.conf.tmpl: fix the configuration of UE params for carrier aggregation

the num_carriers is parsed as a string in the conf dict and therefore
needs to converted to int before matching

also changed the num_carriers to be of type UINT

Change-Id: I1386812d32e1181ba666720bbb875bf9bbce0f51

contrib: build amarisoft: Set trx_zmq.so RPATH to point to copied dependency libsrslte_rf.so

In the ENB case it's not really required since osmo-gsm-tester makes use
of LD_LIBRARY_PATH. However, since the UE is potentially run with
capabilitites to create tunnels and alike, LD_LIBRARY_PATH cannot be
used and lteue will fail to load trx_zmq.so (because it cannot find its
dependency libsrslte_rf.so).

Change-Id: Ifcbc6ed1c4c465b4cfab4c5689cde44624095f29

pcu: Improve logging and gsmtap

Change-Id: I506a39b114eed55a185d329cea5e9d6ba379a2ae

templates: Improve cs7 config in osmo-{bsc,msc}

Change-Id: I0aa0057c85e606d45e67d01b818db7549033bdbd

amarisoftUE: adding basic support for Amarisoft UE

Change-Id: Idda0d3a040663969dd71781814198b47fff7daf3

amarisoftue.cfg.tmpl: fix UE bandwidth/rate config and power on automatically

Change-Id: I2d63055d0648a1758905fd277c20edaeec8f9b85

enb: add measurement modifier to scenario files

this commit adds basic support for configuring measurements in the
eNB config. It currently support A1, A2, and A3 events.
By default UE measurements are turned off and need to be enabled by
configuring an event with:

+mod-enb-meas-event@{name},{report_type},{value},{hysterisis},{time to trigger}

For example one can update the a2 and a3 event with:

+mod-enb-meas-event@a2,rsrq,-40,2,480+mod-enb-meas-event@a3,rsrp,6,1,512

Change-Id: Ia7657be2396886840570bc41645450a268b4cfff

pcu: Enable 'egprs only' vty option if egprs requested

Change-Id: Idd0b5bf8769d693480268c0a0b89dbfd63779e48

Move suites/ dir inside example/

example/paths.conf and documentation are updated accordingly.

Test suites should have been moved a long time ago, since the they are
user or setup-specific based on what needs to be tested.

Change-Id: I154b19979b545deba8b232b60172903f63fd9e28

amarisoft_ue: fix UHD support

the Amarisoft UE seems to be only working with UHD when the bandwidth
param is provided. It seems the set_bandwidth() API is used.

Change-Id: I3f20a5a8ca80c63374f0ad500fd2794e2dc45479

srs/amarisoft{ue,enb}: Fix b200 specific rf_dev_args

Commit 12ae6d79a5b53a213f619999ef0230c1dae88bb7 switched the rf_dev_type
to be in lowercase but forgot to change these conditions.

Change-Id: I91d3877381b4e6343dc4aa0465e699341ed6648d

utils/bin: Support passing several bin files to setcap helper scripts

Change-Id: Ifa43f895ae0796d086f35778263aaba07f2e1b7a

amarisoft_ue: Implement data plane

Use the ifup script to set up the netns + configure the tun device
created by lteue. Use it also as a hook to know when the UE is attached.
Since tun setup is done by arch-optimized lte-avx(2) binaries, we also
need to give capabilitites to them (instead of allowing inheritance of
caps in general in the setcap script).

Change-Id: I1e228b1296eac8e4cb183c2faa735b0468abe124

pcu_{sysmo,oc2g}: Inherit from pcu abstract class

For some reason these classes were still not inheriting from the base
class (probably because it was added after they existed) and they
started failing after recent changes which presumed they where already
inheriting.

Change-Id: I88217baf490ec45373d0743585377ed2639a8cfd

bts_oc2g: Fix missing import remote

Change-Id: I318683b8dc6215f5bf863cfb1581e2c3bfa05621

bts_oc2g: Fix ready_for_pcu() waiting forever

Change-Id: Id33e527591164b87d1e14b7922247b726267ad6e

Get rid of install top directory

Some content there is out of date, or should be mvoed to utils/ with the
rest of installable stuff.

Change-Id: I17ee94b175deb88faff71e7788c482093693f8a8

Move example/ and ttcn3/ to sysmocom/ and sysmocom/ttcn3

This content is mostly specific to sysmocom's setup (thought it can
still be re-used by other people if willing to).

symlinks in ttcn3/ pointing to example/ are redone to point to
sysmocom/.

Change-Id: I364e0859c0851b9167907a4d948b659a0f6a19fb

Introduce doc/examples/ directory

It contains simple usual setups to get started with osmo-gsm-tester,
such as creating a 4G network using srsLTE or a 2G network using the
osmocom stack (+ ofono modems).

Change-Id: I55d5b1152da201799af21a77ef6c562f97bd94fc

Rename object classes to be {obj_class}_{implementation}

Change-Id: Ieafd96d7ee5b4baef08ca767629b93834f6a8c7c

Create core directory to contain most of code not in object classes used by tests

Change-Id: I9aec6c55ccd71894182057d36e0025b69925d314

Move object classes used by tests into obj/ subdir

Change-Id: I0b2963cf00acd1f9036dca2e87aa3b87a0f8066d

Rename modem.py to ms_ofono.py

Change-Id: I050970986b0b3fbc23b756c5b7c524cb8b7c1327

iperf3: Avoid using --logfile arg

Some older iperf3 versions don't support the --logfile arg. Let's
instead parse the json from stdout. Old --logfile behavior is left in
code but disabled since it's a nicer (less hacky) way to get results,
because parsing from stdout means we need to avoid reading some content.

Change-Id: Ia1b7fa4dae074089878963aeddf0006aea68e31f

enb_amarisoft: fix enb conf template name

issue has been introduced during refactor of enb class names

Change-Id: Ib3ad6451c3979440be28d8811f51ce5d1c103518

enb_srs.py: Pass filenames to main cfg file template instead of cmdline args

Change-Id: Ifec6a8a9c29b94634f6ac7ea50aae9b8e18ae3cb

epc_srs.py: Pass filenames to main cfg file template instead of cmdline args

Change-Id: I1ac3a3cace7ec2b0f85c5d9a5f97d5007d7c65ac

ms_srs.py: Pass filenames to main cfg file template instead of cmdline args

Change-Id: Id2a6ba17d087d2ff67fe10b1dae6434259e74366

enb: add optional gtp_bind_addr paramter to enb class

this allows to optionally specify a the GTP bind_address for a enb object
in the resources.conf

this address is then used for binding the listening socket for gtp connections
which, in turn, allows to run the enb and epc on the same host.

when gtp_bind_addr is not specified, addr is used by default

this is especially useful if eNB and EPC run on the same host
and ZMQ is used for RF between eNB and UE. Two eNB addresses
are needed in this case, one bind address and one "remote-accessable"
address for the UEs ZMQ radio to connect to

Change-Id: I4fe38f31401428ed3e74e1d0b64960f0af122d37

powersupply: Import sublcass module only if used

This way we avoid unconditionally importing all subclass dependencies,
and make them optional based on whether the setup has devices of that
type or not.

Change-Id: I0d2f8b26364b45f0d837cc51078784d1d0fa7ea1

check_dependencies: Import modules dynamically and find related debian packages

This way we don't need to manually add new imports here or drop unusued
ones. It also makes sure local imports in all our py files is correct.
For instance, running the script already caught an issue which is added
to this patch (osmo_ms_driver/__main__.py).

This new version of the script also allows specifying subsets of
features to skip when checking for dependencies. This way, for instance
somebody not willing to use a sispm powersupply can stil check all the
needed dependencies are fine.

This new tool will make it easier to slowly make some dependencies only
used by some object test classes optional (for instance, python-smpplib
if user doesn't want to run an ESME node).

It also allows to retrieve the required debian/manually installed
packages when run with "-p" option:
"""
Debian packages:
        libpython3.5-minimal:amd64
        python3-gi
        python3-six
        libpython3.5-stdlib:amd64
        python3-pygments
        python3-yaml
        python3-mako
        python3-numpy
        python3-markupsafe

Modules without debian package (pip or setuptools?):
        usb                  [dpkg-query: no path found matching pattern /usr/local/lib/python3.5/dist-packages/usb/_interop.py]
        pydbus               [dpkg-query: no path found matching pattern /usr/local/lib/python3.5/dist-packages/pydbus/proxy.py]
        smpplib              [dpkg-query: no path found matching pattern /usr/local/lib/python3.5/dist-packages/smpplib/command_codes.py]
        sispm                [dpkg-query: no path found matching pattern /usr/local/lib/python3.5/dist-packages/sispm/__init__.py]
"""

Change-Id: I29ddf8971837754abd930d847bd1036e8e510de6

enb: Add cell_list attribute

It allows configuring each cell_list up to a maximum of 'num_cells'.

Change-Id: I4fd30a3e37789b76bfaddc2beba1815154daab7f

powersupply.py: Fix import module line typo

Recent commit changed the import line so it runs only when required, but
added a typo making tests uing it fail.

Change-Id: Id3e5c16adc763a9b79cc464937c541abd15f98de
Fixes: 2b959580b9affe729e966c506656b7f47926c4bf

srsenb_rr.conf.tmpl: Fix whitespace

Change-Id: Ic2b4d8e2468984b096a870c878fa5d032ea4130f

enb: Implement initial RF emulation API

Two implementations are provided:
* Amarisoft Ctrl interface (websocket)
* Mini-Circuits Programmable Attenuator (HW, HTTP API) [1]

in Amarisoft ENBs, if no rfemu is configured explicitly, the Ctrl
interface one is used by default, while still being possible to use the
HW one.

[1] https://www.minicircuits.com/pdfs/RC4DAT-6G-60.pdf

Change-Id: Ie98a3fb9bcd2b87b96ecbb5b79e0f53981892a32

enb: Add attribute to set 'id'

Change-Id: I8ce296c7875d557062dc13e7093ce0400050f2ce

srsue, amarisoft/srsENB: Support setting rx/tx_gain attributes

Change-Id: I4228ba8746a54db4c17443718c01069d431361b9

AmarisoftUE: Fix setting tx/rx gain attr

Fixes: a6d6304afccbd37462acf9ad8207d6ac29f25da9
Change-Id: I344313863390a00604045dcc18e552aa789b282f

AmarisoftENB: Support setting PCI value

Change-Id: Ibffb95f9b55e145d76ba64119a73ef9f1ed7cdef

enb: Introduce support for neighbour cell list

Change-Id: If604f1f2643a73f8c4da6ae3e9d24cd6c7e52d06

srsenb: Support neighbour and secondary cell lists in config file

Change-Id: I8fe0703b7d9db69658c112066404d2da1c3d1e09

amarisoftENB: Fix running with num_cells=2

The B200 device being used in the physical setup doesn't support several
RF ports, so let's hardcode it to 0 for now.

Change-Id: Id4c31be2bd0c75f469e2293ade011c7b7ceedb1c

jenkins-build-amarisoft: turn off other RF libs when building for zmq driver

Change-Id: Ie4819d3f454d74e14829d02fbdc62a12f2d54cd5

jenkins-build-common.sh: also pass additional build flags to CMake

Change-Id: I1c5998d31201505d0a2532f22f905ec155d6c094

amarisoft_enb: use two antenna ports for TM > 1

Change-Id: Ia241f96341780760d482d389097659b0f7b897e8

enb: refactored rf_args handling for ZMQ-based radio

this patch moves the rf_dev_args creation for both eNB types
into the eNB base class, since they are identical.

the patch also fixes the arguments for all CA and MIMO configurations

Change-Id: I8ca3ed83e65dc07927385267e5970bc4f5b120d5

ms_srs: fix ZMQ radio arguments for CA and MIMO configs

similar to the eNB patches this fixes the CA and MIMO radio
parameters. In pricinple this could also be reused for the
AmarisoftUE but since we currently don't have a means to test
it I left it for a future commit.

Change-Id: I16b00009f97d61a9ab4715e5d727024661d3d0be

srsenb_rr.conf.tmpl: extend CQI and SR subframe scheduling

this fixes an issues when allocating CQI resources for multiple carriers
or multiple users

Change-Id: I8b9345502003da2f5b6841a98b77375656e84d8c

Generate schemas dynamically from pieces provided by each object class

This way we benefit from:
* knowing which attributes are used/required by each object class and
  subclass
* Having validation function definitions near the class going to use them

Change-Id: I8fd6773c51d19405a585977af4ed72cad2b21db1

Move object subclass instance allocation logic out of suite.py/resource.py

This way we get rid of object related content in resource.py and we can
finally move it to core/ in next commit.

Change-Id: Ia5b861115ae3ae1b90149863c4265dcc730b8cd4

Move resource.py to core/

Change-Id: I0f36a5e61ff0ef14d08165c9c184acdf2d6e6901

Move test.py and report.py to core/

Change-Id: Ibb7fa5ab40bcf1e59705bdd2c2c5a76025b2b544

Move all obj/ references in suite.py to testenv.py

Change-Id: If4ab39be7a97d33e82c5a34e2a10dfec38613a4e

Move trial.py and suite.py to core/

Change-Id: Ic9a4c90c190e9d8e1829fbef83a9243f3c6f655b

Update obj classes in obj/ to use testenv instead of suite_run

Change-Id: Idf41243a497de7bc507b9f1cd1b38640f456f815

tests: Replace 'suite' with 'tenv'

Change-Id: I777ec59ffd83b861ddc09e85e52d7a8896264b8c

testenv: deferr importing obj module class until use is required

Change-Id: Ie073256a03bb2d038ef7f715d534afb1c53bb241

check_dependencies: allow passing any module name to skip

Change-Id: I53a384d54f32d766f70bd7e0bfbd9702939c8aa8

ms_srs: import numpy module on first use

Change-Id: I2ce974954028bdbc7bc1561c4538b909edee2cb1

esme: import smpplib module on first use

Change-Id: Icc92937fdada9210f788805dbc42b80014e345e8

ms_ofono: import pydbus and GLib modules on first use

Change-Id: I754e1290eb5593c0ef168f74747f42db1b50f16c

powersupply_sispm: import sispm and usb modules on first use

Change-Id: I6045bb739e5c69972014a9d27be3a24b7ad61c9a

rfemu_amarisoftctrl: import websocket module on first use

Change-Id: I568af52ffb7ac57222697f5d861a532c0e2a70de

doc: Drop redundant app-name in install section

Change-Id: I6ec3c42efd3eb958f832ff9a797ddaa3dae4a99a

doc: install.adoc: Correctly split optional dependencies

Change-Id: Id8b5dcd15fd86c670915b928112699d831e6e7b0

Rework README file

Drop old one in doc/ which contains lots of outadated stuff already
placed in the User Manual.
Write a new markdown README in the root directory with pointers to
places in the repo and the issue tracket, ansible setup, etc.

Change-Id: I9febe56223eaeadc9d31291dd0110a11470e3ab1

osmo-gsm-tester.py: Fix import error when using default-suites.conf

Recent commit refactoring stuff introduced an import error only
triggered when default-suites.conf is used (no -s param).
This commit fixes the issue.

Fixes: ea8c3d4b1235ef703e54cd963bc96f6716168387
Change-Id: I1534cfba55c69ed5c1354175f8b5df9b656b7d82

junit: Replace invalid xml characters
Otherwise junit parser in jenkins fails: org.dom4j.DocumentException:
Error on line 20231 of document  : An invalid XML character (Unicode:
0x1b) was found in the element content of the document.
Fixes: 5bbdab8d95dc739112a6d6999d438b5e142392b8 Change-Id:
Ia629e43bba01e50fd718c16404a7796d4f4e3713

srsue: Remove old comment not longer applying
Change-Id: I1718dbb701c5dcd5a4081d3d91ae9f192418e7df

srsue: Enable metrics csv file generation
Change-Id: I393afc49f553d9ebda29717c27d5eb907b3c5b60

srsue: Enable pcap through config file
Change-Id: I15219d707aea778fefa8911362278442fa09550c

srsue,srsenb: Fix zmq base_rate calculation and share value
Change-Id: I3e7817e822fb4d1f8bc9ef798162edf95a74d442

iperf3: Support configuring data direction (dl vs ul)
Change-Id: I6b5b740beac22015c952ce826c034031ad4c4465

tests: Improve logging of iperf3 results
Change-Id: I6563c11f4ad6e9519400268e63944cdf8973dad0

iperf3: Support setting run time
Change-Id: I93e857bd682c733c3ba36234b9bf1e1cc85eddef

utils: Add limits.d dir with sample config files
Same as the ones described in osmo-gsm-tester User Manual and same as
ones deployed by ansible scripts. Let's rather keep them here.
Change-Id: I8b124a8a1637666536772439b920e7b91e61533c

scenarios: Use parametrized scenario for rftype
Change-Id: Ib3d7db13d7de492eee5ad25cf5c88239be9e9fbf

srsue: Introduce metrics verification procedures
Change-Id: Ib1da58615cdc4f53ac1a27080e94e5b47760c508

suites/4g: Introduce iperf4 downlink test
Change-Id: Ifc4bb7394120d57ca6d26459be1cf8365a16249c

Allow overwriting test's report stdout text from inside test code
Change-Id: Ie1c0ecf7464209a4241665025fda33ae624fc049

remote: Set debug logfile of ssh_sigkiller.sh to /dev/null
Otherwise processes run eventually as root (due to being run on another
netns) may change its permissions and other processes later will fail to
write to it (Permission Denied).
Change-Id: Ide092db082937be7c2fb16d60e099c85dc1941b7

Improve junit xml generated fields
* trial ('testsuites' node): Add fields 'tests', 'errors', 'failures',
'time'.
* testsuite: Add fields 'errors', 'failures', 'skipped',
'disabled'
* test: Add field 'classname' (empty).
Fix and improve suite.py and test.py to count errors, skipped, failures
properly.
Change-Id: Ie2d10cee88a9c0d829e4620553164cf3150e8e5c

suites/4g/ping.py: Overwrite test stdout with ping output
Change-Id: I83403a21c46758132b4fbb930000df67d8650016

srsenb: Update config files
Latest ENB config file grammar changed and needs to be updated to work
with next versions of srsENB.
Change-Id: I84db30dc57ea9e0b05208fa86d88d42d492b237f

srsepc: Update log file output matching
New version of srsEPC improved the information logged during S1 setup
request, which we use to wait for the ENB to be connected.
Change-Id: Ia2338e3161c66ef7c1ab50b024a70de661edc780

srsue,srsenb: Configure num_ports for tmode > 1
Change-Id: I35d8ee907738845e729f58c23f377a42bb665672

srsenb_sib.conf: Fix warning about wrong prach_freq_offset value
WARNING: Invalid PRACH configuration - prach=(2, 8) does not fit into
the eNB PRBs=(0, 6).
      Consider changing the "prach_freq_offset" value to 0 in the
sib.conf file when using 6 PRBs.
Change-Id: Ie4374faca62a4d0289f78674eadec7eb5d045dfd

srsue: Use logfile instead of stdout
newer srsLTE has fixes to exit properly after 3 seconds and then flush
the log file.
Change-Id: I9801ce932ad08b7fbb6d14a66e1dba64d4af38f7

config.py: use var to hold defaults.conf string
This way we have all config file names together in same place.
Change-Id: If35739f21f1414efdaba71acbb60661213ac58df

Move RESOURCES_CONF from resource.py to config.py
This way we have all known config file names in one place, in the module
containing the implementation to retrieve them
(config.read_config_file()).
Change-Id: I8304dda70268b16dbc316af882fecdbf20ac25ec

Drop wrong vty cfg line ms-power-control
Those two BTS backends never had their own MS Power Control Loop, so
they should be using the "osmo" common one. This wrong value "dsp" used
to be accepted for no good reason, but it's no longer the case. Let's
remove the line to let the backend take the default one (osmo).
Change-Id: I842fce2b8185c53d1684747afcfc3ae8fba1afee

osmo_ms_driver: Store cfg files and logs in correct test rundir
This way all the content is available as an artifact when we run it
inside osmo-gsm-tester.
Change-Id: I3ab493e6aafef90bb0c5d98e552840fd11c43079

doc/manual: Refactor, rewrite, improve and update most of the User
Manual
* Some TODOs are added as comments which actually require code changes.
These are details which showed up as incongruences or missing bits
while writing the documentation for them.
* Some sections are introduced but still waiting to be writen soon:
** Debugging section
** Docker Setup section
** Ansible Setup section
** Troubleshooting (add jenkins red cross button sending kill -9)
** resources.conf attribute list needs to be converted to a table
* Device related setup needs to be updated and extended
* Parametrized scenarios need to be documented
* 4G resources documentation needs to be added.
Change-Id: Ifc2a3c74d45336cc988b76c0ff68a85311e4dd40

doc/manuals: Write proper content for Docker Setup section
Change-Id: Ie883adbf03a3b5c9a852182c01d15a212749ce04

doc/manuals: Write proper content for Ansible Setup section
Change-Id: Iffd3beefa50dd5f18908b4c3f7e4deb169feca62

doc/manuals: Write section explaining parametrized scenarios
Change-Id: I04b7fa46c2a282095bbb8f6d69ade87ff86f910c

doc/manuals: Fix title out of sequence
asciidoc: WARNING: install.adoc: line 450: section title out of
sequence: expected level 3, got level 4
Change-Id: I0c881405e66cc6815533bb884016db20cfdff3dc

srs{ue,enb,epc}: Disable pcap by default, allow enabling by param
Take the change to fix several small things and support recording pcap
in srsENB.
pcap generation can be enabled with scenario cfg-srs-enable-pcap.
Change-Id: Ia096a9be7efb2123f95115c751e2402fb4fec935

doc/manuals: Introduce Troubleshooting section about SIGKILL fix
Add a section describing how to clean up and recover osmo-gsm-tester
state after a sigkill is used.
Change-Id: I4841ab6d44a122140e6352df1fb6543418adc033

doc/manuals: Add proper content for Debugging section
Change-Id: Iaed2544d59f7c5dd69eef3ddfade544b35c837cd

doc/manuals: Several format and styles fixes for config.adoc
Change-Id: I6371e09d6d88f137e7611bd60ac97b86414d8c47

scenarios: Drop unnecessary srsue-rftype-soapy.conf
There's already a parametrized scenario handling it. We don't use any
soapy backend for srsue anyway yet.
Change-Id: Ie680178d2b16139473b5899803fb69a715a79216

doc/manuals: Introduce chapter explaining config data mangling
Explains how configurations bits are arranged and mangled to provide a
last config set.
Change-Id: I5c8f136e743b6beb02956b3436aa3d1010205141

srsenb.conf.tmpl: update to new version
Change-Id: I204d03307aaaddcb15fed6f7310b64d09c43f64c

fix lookup table for maximum DL/UL rate for srsENB
Change-Id: Ieb027eaf5d8f4d774d0e29207f7a3b75dec6ee5b

relax max_rate for iperf UL/DL tests
Change-Id: Ia243156931918d5ab013c64c37823bc2e400798e

process: add stdin_write method to communicate with child on stdin
Change-Id: I3e5d843911998bb50726d93e3dfd3de584dc81a5

srs_ue: send t+Enter to stdin after launching app
this enables the console trace which is printed to stdout and provides
human readable radio statistics
Change-Id: If1c998aff1c5c11d3fa636fb60cb4991e0def768

srs_ue: run srsUE as normal RemoteProcess
running with RemoteProcessFixIgnoreSIGHUP was needed bc srsUE used to
ignore SIGHUP that is delivered to the process when the SSH session is
terminated causing the UE to not quit properly. This isn't the case
anymore so we can use RemoteProcess for spawning the UE.
Furthermore, the wrapper script didn't pass along stdin to it's child
process, i.e., srsUE. Therefore the console trace can't be seen if the
run through the wrapper.
Change-Id: Iab4dafc58167620362a36ec9e904b8d49a7d5fc1

srs{enb,ue}.conf.tmpl: make sure to only use a single PHY thread when
running with ZMQ
Change-Id: Icc1e67705c0354ece77c037f224a9c756a16daaa

srs_ue: add scenario for airplane mode simulation in srsUE
Change-Id: I5db2ff325ce4b4236c1ceec0831dd7420d52dedb

ms: add optional param to specify fixed APN IP address assigned to a MS
this allows to specify a fixed APN address for each subscriber in the
HLR. If the param is not provided in the resources config, the default
allocation type is used, which is to dynamically allocate an IP in the
supported range.
Change-Id: Ie6fcd7987dc7bc62a686bee3a10c67e3dd813419

srs_enb: add num_cells param to srsENB
this is needed to support the upcoming multi-cell feature in srsENB.
It's required for carrier aggregation (CA) but can also be used for
(currently) two independent 4G cells.
This commit adds baseline support for the parameter. Upcoming commits
will add hooks to configure the cell's EARFCN, etc.
Change-Id: I6d1658bf820163aee03b64b0f4ac68046c243f37

srs_ue: adding num_carriers paramter and corresponding modifier
this allows to configure a UE to support multiple carriers
Change-Id: I4fa0a0c6e52efb17aa0b07e55abb5470d8028e8b

srs_{enb,ue}: add secondary cell and set dl/ul freq in eNB/UE config
accordingly
Change-Id: I3730856003fffda4f3c3f9e835d62ddc312c4c31

srs_enb: enable console trace, use RemoteProcess class
In order for the console trace to work over SSH this commit switches to
use the default RemoteProcess class. Note that srsENB now correctly
handles the SIGHUP signal.
Change-Id: I26391eb8409e14b34987e0b338e10613fa157e06

srs_enb: remove unneeded enb parameters
Change-Id: I1f2e18c6552fc4a6e3a68d75a85744fadf26aacb

srs_enb: remove num_cells getter
Change-Id: Ic5c45bb17440d2ee241840591baf02930a0b3470

srs_ue: remove hard-coded num phy thread param
Change-Id: Ia4fac188ecbe6aba393a68ec0291b51cd1441029

defaults.conf: set num_carriers default value
Change-Id: Ibe94fada0a40c499dd516408756c4ef7988ccec5

Introduce initial Amarisoft ENB support
* object class expects the amarisoft software to be placed in directory
provided by env var AMARISOFT_PATH_ENB. On local runs, it will run the
software from there. On remote runs it will copy over that directory to
the slave.
* Configuration provided works with 6 PRBs, but probably won't work for
other values (yet).
* ZMQ support not yet available, only UHD.
One can select the Amarisoft ENB by declaring it in resources.conf:
"""
- label: AmarisoftENB-B200
type: amarisoftenb
rf_dev_type: uhd
rf_dev_args:
"type=b200,serial=317B9FE,recv_frame_size=9232,send_frame_size=9232"
remote_user: jenkins
addr: 10.12.1.206
"""
And running:
"""
$ export AMARISOFT_PATH_ENB=/path/to/binaries
$ osmo-gsm-tester.py ... -s
4g:amarisoftenb-rftype@uhd+srsue-rftype@UHD+mod-enb-nprb@6 -t ping
"""
Change-Id: I1ddf4962ea6438ad977382ab201b724028ca46b3

srsenb: start_locally(): Fix missing class name in attribute access
Change-Id: I5e11dc9c3c0dcf4808a3ad67fe96580480fe47d1

srsenb: No need to generates values dict for every template file
Let's only do all the dictionary processing once and not for each
template file.
Change-Id: I4684d26375fce1e2ae28d3eccdc4f607fa5411a9

srsue: start_locally(): Fix missing class name in attribute access
Change-Id: Iab4196b12ee12a06ba2129053f2291c1e4c284ff

{srs,amarisoft}enb:Move shared code to parent generic class eNodeB
Change-Id: Ica0f1d4179a905d6dfc77d3e8eb08ac6cd5c3d2e

amarisoft_enb: Drop unused lib variable
Change-Id: Iea3e77a4b9f25f2781f617cd4bfd0ba4b34c709e

Introduce support for AmarisoftEPC
* A new abstract generic base class EPC is created
* srsEPC and AmarisoftEPC inherit from that class
* options are loaded from defaults.conf in cascade. First generic "epc",
afterwards the specific enb type.
* A new scenario is added to select the EPC type to use. srsEPC is the
default unless stated by an scenario.
* AmarisoftEPC delegates setup of the tun IP address to an "ifup"
script. As a result, since we run without root permissions (ony with
CAP_NET_ADMIN), the ifup script itself is unablet o set the IP
address. To solve this, we introduce a new osmo-gsm-tester helper
script which must be installed in the slave node which can be called
through sudo to increase privileges to do so.
With this commit, I can already get srsUE<->amarisoftENB<->amarisoftEPC
to pass ping and iperf3 4g tests.
Change-Id: Ia50ea6a74b63b2d688c8d683aea11416ad40a6d3

srs_{ue,enb}: change UHD stream args and buffer size as function of cell
width for B2XX
Change-Id: I2de0526ee9bd2f7bb75c7d529b3f52d75117c5cc

Make amarisoft license server optional
* Since license_server_addr logic is generic among all amarisoft
software, let's move it to its own config category "amarisoft", which
all amarisoft classes (amarisoft_{enb,epc}.py as of now) will import
during configure().
* The license server is disabled by default (value 0.0.0.0) since the
usual case is to have a license in the local system.
* the license server can be enabled using scenario
cfg-amarisoft-license@A.B.C.D where A.B.C.D is the address to use.
Change-Id: Id5456ad9497e91370857b80ff9a389111d3611ff

iperf3: add config to adjust the duration of the iperf run
the time is passed as a string param and is then converted into seconds
when literals "h" or "m" are found. So it would accept 2m and would
convert it to 120s, for example.
Example:
+cfg-iperf3-time@15+
Change-Id: Iff28816f83670751e9e91de31ec59b1b0ad8fc0d

srsenb_rr.conf.temp: patch scheduler variables for 6 PRB cells
Change-Id: Ifdab42da8184cdf8367c5be1639a0faea3e68db1

srs{enb,ue}: Use lowcase UHD as rf_type string
Older versions of srsLTe.git required passing UHD in caps to the conf
file, but that's not the case anymore. Let's use lowcase then.
Change-Id: If8a289ba110229d31b62b0d3adc1ae97324e6d27

config.py: Allow escaping commas in parametrized scenario names
comma character ',' is used in the command line to split between
parameters, which means a parameter value itself couldn't contain it.
This commit allows passing a comma inside a character value by escaping
it with '\,'.
Change-Id: Ic0bd9a029137a59e8c4a32b807eba7a64fcfa51f

srs_{ue,enb}: Add modifier to pass arbitrary cmd line arguments
Change-Id: Ie438c8aa9992dbc61b2133473a83d565cf5c7056

Introduce contrib/jenkins-build-amarisoft.sh
Prepares inst tgz archives to be consumed by osmo-gsm-tester from
Amarisoft prebuilt binaries.
Change-Id: I372606fc5ff9f6bb3ef848f2fe229b83382ddc09

amarisoft_enb: Add missing default values in constructor
Change-Id: I61c04df14468b38cfe4afaaeb5404367838ff0a9

contrib/jenkins-build-amarisoft.sh: Make build of amarisoft trx_zmq.so
optional
That module is only available so far in a private repository, so allow
still other amarisof tusers to build the bundles with UHD support.
Change-Id: I361f179c8e7f379173fa9109028e139fc4c8c251

contrib/jenkins-build-amarisoft.sh: Fix typo copying trx_uhd.so to UE
Change-Id: I0c9b83b9adf0548e78363462744644bc528c2d29

Add modem resource features attributes: 2g, 3g, 4g
This allows selecting modem by supported network type. It will allow
using 4G modems with type different than srsue in 4g suite in the
future.
Change-Id: I38bcf6abf789f52c3ed0bee7911567fa872e2491

contrib/*sysmo.sh: Fix default poky SDK version to 2.3.4
Current latest poky SDK is 2.3.4, and that's the one we have installed
in all buildhosts.
Change-Id: I8c855456c7f94a3d6ee58ca550a056a0fea61a71

contrib/*sysmo.sh: Fix default poky SDK path
ansible scripts are installing the SDK in "poky-sdk" dir, not "poky"
one.
Change-Id: I002fde53684cd4a43c9f22e09264056f2a950c24

config: Fix reading empty yaml default-suites.conf
during config.read(), on empty file yaml.safe_load() returns None, which
was then later converted to string "None" by _standardize(), and
osmo-gsm-tester.py was not catching "not combination_strs" condition.
Change-Id: I07d7daab8f8f4238db140f0a0311f3d1d41e6cb0

contrib: Fix default variable value syntax
Change-Id: I23abf33881f64dd89c0bf5ef970972908d337366

contrib: srslte: Allow changing repo name
Change-Id: I06e6566ac3df9369b391bc88f288f72588088ef1

suites/debug: Allow jumping into python debugger
Change-Id: Ia6262299981f1d7ac2d81c22b7b9b007e096259a

iperf3: Fix netns param not used
Change-Id: I2768ec0c6d2cbfc8411382a983e1ba17cb5f097e

srsLTE: Support configuring different RF backends
Before this patch, only virtual RF through ZeroMQ was supported. This
patch allows configuring srsUE and srsENB to use a real SDR with
UHD/SoapySDR backend connected through a physical RF network, while
still keeping compatibility to run on virtual RF ZeroMQ network, based
on the resources used (controlled by scenarios). For instance, one can
first run a suite through the phyisical RF (using 2 UHD-controlled SDRs)
and afterwards with ZeroMQ using the following default-suites.conf:
- 4g:srsenb-rftype-uhd+srsue-rftype-uhd
- 4g:srsenb-rftype-zmq+srsue-rftype-zmq
Change-Id: I7dbbe328f4c0225fe74e878bb2da13fe39ccf049

remote: Fix comment typo
Change-Id: Ie466a95d354dc8c9d6960f713948426dadc58241

srsue: Create netns on demand
Already existing script osmo-gsm-tester_netns_setup.sh is modified to
support only creating the entns without moving an iface to it.
Change-Id: I1b7e186b0146f932fe37fbea68e4dfa3120b7a74

Add test log to junit output
Change-Id: Ieb5566a41ecf3a9512db579eb37e8d8ed5325057

srs_enb: Make num_prb a config parameter
Change-Id: I336e71f53556596050cf3eb55a7810a009dbf7a8

enb: Add modifiers for num_prb
Change-Id: I0b8c834f1c67cc4bf4d07d03358ba2012331cbde

Introduce parametrized scenario files support
The idea is to have something similar to systemd template unit files:
https://fedoramagazine.org/systemd-template-unit-files/
Specially for modifiers, one finds the situation where same scenario
structure has to be created with lots of different values. For instance,
let's say we want to test with different eNodeB num_prb values:
[6, 15, 25, 50, 75,100] Right now we'd need to create one scenario file
for each of them, for instance: mod-enb-nprb6.conf mod-enb-nprb15.conf
mod-enb-nprb25.conf mod-enb-nprb50.conf mod-enb-nprb75.conf
mod-enb-nprb100.conf
And each of them containing something like (changing the num_prb value):
""" modifiers:
enb:
- num_prb: 75
"""
Instead, we can now have one unique file mod-enb-nprb@.conf:
""" modifiers:
enb:
- num_prb: ${param1}
""" The general syntax is: "scenario-name@param1,param2,param3". So "@"
splits between scenario name and parameter list, and "," splits between
parameters.
For instance, one can now run following suite with scenario:
"4g:srsenb-rftype-uhd+srsue-rftype-uhd+mod-enb-nprb@75"
Related: OS#4424 Change-Id: Icfcba15b937225aa4b1f322a8005fcd57db1d1ca

Use parametrized scenario to set ENB num_prb
Change-Id: Iec7d7e6312aee6f4e4ce6710bc78a144beff2f39

Introduce LTE Transmission Mode and RLC DRB mode params
Change-Id: I38f247f266ba736cdbb6f36ba5673b8fd46a8d67

ttcn3: Refactor ttcn3 launching bits into a testlib
This way new tests can be more easily created which run some specific
TTCN3 test.
Change-Id: Ic61c7b7db9cf3050dc4b101ef0fb181421577424

ttcn3: Allow passing ttcn3 extra module params from osmo-gsm-tester
Change-Id: Iaa2a5647801b4f152f72bf4b40ac79f18608df87

Allow passing remote env to process running in remote host
Change-Id: Icc0ae8d79ca30ed0a289b67546735251fc46cb10

Add handlers to run process under a specific netns on a remote host
It will be used later to run iiperf3-cli connected to srsue on another
host.
Change-Id: I8d223fc302df42446f5876ba020cfbea94be09b9

remote: Improve wrapper script handling proper exit of ssh-remote
processes
Improvements include:
* Avoid race condition between receiving signal and process not yet
started
* Make sure process is killed with -9 if process is still alive a while
after
we killed it (SIGINT, SIGTERM). This is useful for processes which
sometimes hang during shutdown like srsue in some conditions.
Change-Id: I3c656b008a3c2b2bb453a59e51d338cb272fa50b

jenkins-build-srslte: Temporarily Switch git repo to own-controlled one
During integration of the different srsLTE processes, some bugs or
misbehaviors appear which require fixes in srsLTE. Let's use our own
temporary fork until integration is complete and all patches are merged
in upstream.
Change-Id: I6414d9855cf9c004385cc280e5a60e82f8e4fbad

remote: Add more helpers to operate on remote hosts
Change-Id: I034314839675038762e00750b069eee8fcb70a5c

utils: Move shell script helpers to subdir
Change-Id: I86ed3b0bb779c50e57912cf6c2f1ee9fa6e5f557

util: Add sudoers.d/ example directory
Change-Id: Ib9ab4d88bf2fb94d04240169e0203b72616f58b0

Inroduce RunNode class
This class will be used to hold information for a run node, that is, a
target system or environment were a process or task is run. It
superseeds in functionality the old ip_address resource, which will
eventually be droped in favor of RunNode.
Change-Id: I647bedf116aa9a570d925a5281c9491c9032e343

util: Add helper setcap_netsys_admin()
This API was already added for RemoteHost but forgot to add it too for
local systems. It will be used by srsue once support for it is added.
Change-Id: I734e910af90fd25bed27c24b60346064b5fde67e

iperf3: Support running iperf3 remotely
The new RunNode class is used and ip_address support will be dropped
eventually, replaced by the former.
Change-Id: Ib803d7774cb502c7d07443d7720a7b013684faa8

Fix remote and process RemoteNetNSProcess
Change-Id: Icdc917968a1310e287dd98994420519a605be9c0

powersupply_intellinet: Log target ip+dev_port
Change-Id: Iafd6e3aaf1d3b4d91ac6cf06f2a7fe94fd1203c3

resources.conf.prod: sysmocell500 new firmware now supports TRXDv1
Change-Id: Iec50662580479bf7b6dc08d08a1de8075b2da078

Initial srsLTE support
2 tests (iperf3, ping) working against a full srs{UE,ENB,EPC} network
using ZeroMQ backend for RF (so no real RF support yet, that will come
next).
Related: OS##4295, OS#4296
Change-Id: I290c0d79258a9f94f00c7ff2e1c6c5579c0e32f4

default-suites.conf: Run 4g test suite (srsLTE ZeroMQ)
Change-Id: I3f8c3588d7b139f044462d02069919256126bf57

contrib: Add scripts to build srsLTE software
Change-Id: Id9d63920a44a80af187e649c9be5fd7498fa5f44

Re-add accidentally removed update_version.sh
Last commit removed this file accidentally.
Change-Id: I030f927af26d4750e88662e591cde3eea6b810c7

Introduce RemoteHost and refactor code to use it
Let's move code related to coping stuff to remote hosts and managing
remote processes under a class where relevant information is stored.
This simplifies parameters being passed all over and allows to reuse
more code.
Change-Id: Ifff5ded8fdb28e8ef267cebe6c5f30a910cae11a

ttcn3: Allow changing TTCN3 test from within osmo-gsm-tester
Change-Id: I02bf406e310f5bbab7b2e0bca47fb120418e241c

WIP: Add initial srsLTE support
Change-Id: I396d7fca64ad6f5afa8556058d4749219136fa4d

Introduce RemoteHost and refactor code to use it
Let's move code related to coping stuff to remote hosts and managing
remote processes under a class where relevant information is stored.
This simplifies parameters being passed all over and allows to reuse
more code.
Change-Id: Ifff5ded8fdb28e8ef267cebe6c5f30a910cae11a

ttcn3: Allow changing TTCN3 test from within osmo-gsm-tester
Change-Id: I02bf406e310f5bbab7b2e0bca47fb120418e241c

contrib: Add scripts to build srsLTE software
Change-Id: Id9d63920a44a80af187e649c9be5fd7498fa5f44

Re-add accidentally removed update_version.sh
Last commit removed this file accidentally.
Change-Id: I030f927af26d4750e88662e591cde3eea6b810c7