Changes

junit: Replace invalid xml characters
Otherwise junit parser in jenkins fails: org.dom4j.DocumentException:
Error on line 20231 of document  : An invalid XML character (Unicode:
0x1b) was found in the element content of the document.
Fixes: 5bbdab8d95dc739112a6d6999d438b5e142392b8 Change-Id:
Ia629e43bba01e50fd718c16404a7796d4f4e3713

srsue: Remove old comment not longer applying
Change-Id: I1718dbb701c5dcd5a4081d3d91ae9f192418e7df

srsue: Enable metrics csv file generation
Change-Id: I393afc49f553d9ebda29717c27d5eb907b3c5b60

srsue: Enable pcap through config file
Change-Id: I15219d707aea778fefa8911362278442fa09550c

srsue,srsenb: Fix zmq base_rate calculation and share value
Change-Id: I3e7817e822fb4d1f8bc9ef798162edf95a74d442

iperf3: Support configuring data direction (dl vs ul)
Change-Id: I6b5b740beac22015c952ce826c034031ad4c4465

tests: Improve logging of iperf3 results
Change-Id: I6563c11f4ad6e9519400268e63944cdf8973dad0

iperf3: Support setting run time
Change-Id: I93e857bd682c733c3ba36234b9bf1e1cc85eddef

utils: Add limits.d dir with sample config files
Same as the ones described in osmo-gsm-tester User Manual and same as 
ones deployed by ansible scripts. Let's rather keep them here.
Change-Id: I8b124a8a1637666536772439b920e7b91e61533c

scenarios: Use parametrized scenario for rftype
Change-Id: Ib3d7db13d7de492eee5ad25cf5c88239be9e9fbf

srsue: Introduce metrics verification procedures
Change-Id: Ib1da58615cdc4f53ac1a27080e94e5b47760c508

suites/4g: Introduce iperf4 downlink test
Change-Id: Ifc4bb7394120d57ca6d26459be1cf8365a16249c

Allow overwriting test's report stdout text from inside test code
Change-Id: Ie1c0ecf7464209a4241665025fda33ae624fc049

remote: Set debug logfile of ssh_sigkiller.sh to /dev/null
Otherwise processes run eventually as root (due to being run on another 
netns) may change its permissions and other processes later will fail to 
write to it (Permission Denied).
Change-Id: Ide092db082937be7c2fb16d60e099c85dc1941b7

Improve junit xml generated fields
* trial ('testsuites' node): Add fields 'tests', 'errors', 'failures',
 'time'.
* testsuite: Add fields 'errors', 'failures', 'skipped',
 'disabled'
* test: Add field 'classname' (empty).
Fix and improve suite.py and test.py to count errors, skipped, failures 
properly.
Change-Id: Ie2d10cee88a9c0d829e4620553164cf3150e8e5c

suites/4g/ping.py: Overwrite test stdout with ping output
Change-Id: I83403a21c46758132b4fbb930000df67d8650016

srsenb: Update config files
Latest ENB config file grammar changed and needs to be updated to work 
with next versions of srsENB.
Change-Id: I84db30dc57ea9e0b05208fa86d88d42d492b237f

srsepc: Update log file output matching
New version of srsEPC improved the information logged during S1 setup 
request, which we use to wait for the ENB to be connected.
Change-Id: Ia2338e3161c66ef7c1ab50b024a70de661edc780

srsue,srsenb: Configure num_ports for tmode > 1
Change-Id: I35d8ee907738845e729f58c23f377a42bb665672

srsenb_sib.conf: Fix warning about wrong prach_freq_offset value
WARNING: Invalid PRACH configuration - prach=(2, 8) does not fit into
the eNB PRBs=(0, 6).
      Consider changing the "prach_freq_offset" value to 0 in the
sib.conf file when using 6 PRBs.
Change-Id: Ie4374faca62a4d0289f78674eadec7eb5d045dfd

srsue: Use logfile instead of stdout
newer srsLTE has fixes to exit properly after 3 seconds and then flush
the log file.
Change-Id: I9801ce932ad08b7fbb6d14a66e1dba64d4af38f7

config.py: use var to hold defaults.conf string
This way we have all config file names together in same place.
Change-Id: If35739f21f1414efdaba71acbb60661213ac58df

Move RESOURCES_CONF from resource.py to config.py
This way we have all known config file names in one place, in the module 
containing the implementation to retrieve them
(config.read_config_file()).
Change-Id: I8304dda70268b16dbc316af882fecdbf20ac25ec

Drop wrong vty cfg line ms-power-control
Those two BTS backends never had their own MS Power Control Loop, so 
they should be using the "osmo" common one. This wrong value "dsp" used 
to be accepted for no good reason, but it's no longer the case. Let's 
remove the line to let the backend take the default one (osmo).
Change-Id: I842fce2b8185c53d1684747afcfc3ae8fba1afee

osmo_ms_driver: Store cfg files and logs in correct test rundir
This way all the content is available as an artifact when we run it 
inside osmo-gsm-tester.
Change-Id: I3ab493e6aafef90bb0c5d98e552840fd11c43079

doc/manual: Refactor, rewrite, improve and update most of the User
Manual
* Some TODOs are added as comments which actually require code changes.
 These are details which showed up as incongruences or missing bits
 while writing the documentation for them.
* Some sections are introduced but still waiting to be writen soon:
** Debugging section
** Docker Setup section
** Ansible Setup section
** Troubleshooting (add jenkins red cross button sending kill -9)
** resources.conf attribute list needs to be converted to a table
* Device related setup needs to be updated and extended
* Parametrized scenarios need to be documented
* 4G resources documentation needs to be added.
Change-Id: Ifc2a3c74d45336cc988b76c0ff68a85311e4dd40

doc/manuals: Write proper content for Docker Setup section
Change-Id: Ie883adbf03a3b5c9a852182c01d15a212749ce04

doc/manuals: Write proper content for Ansible Setup section
Change-Id: Iffd3beefa50dd5f18908b4c3f7e4deb169feca62

doc/manuals: Write section explaining parametrized scenarios
Change-Id: I04b7fa46c2a282095bbb8f6d69ade87ff86f910c

doc/manuals: Fix title out of sequence
asciidoc: WARNING: install.adoc: line 450: section title out of
sequence: expected level 3, got level 4
Change-Id: I0c881405e66cc6815533bb884016db20cfdff3dc

srs{ue,enb,epc}: Disable pcap by default, allow enabling by param
Take the change to fix several small things and support recording pcap 
in srsENB.
pcap generation can be enabled with scenario cfg-srs-enable-pcap.
Change-Id: Ia096a9be7efb2123f95115c751e2402fb4fec935

doc/manuals: Introduce Troubleshooting section about SIGKILL fix
Add a section describing how to clean up and recover osmo-gsm-tester 
state after a sigkill is used.
Change-Id: I4841ab6d44a122140e6352df1fb6543418adc033

doc/manuals: Add proper content for Debugging section
Change-Id: Iaed2544d59f7c5dd69eef3ddfade544b35c837cd

doc/manuals: Several format and styles fixes for config.adoc
Change-Id: I6371e09d6d88f137e7611bd60ac97b86414d8c47

scenarios: Drop unnecessary srsue-rftype-soapy.conf
There's already a parametrized scenario handling it. We don't use any 
soapy backend for srsue anyway yet.
Change-Id: Ie680178d2b16139473b5899803fb69a715a79216

doc/manuals: Introduce chapter explaining config data mangling
Explains how configurations bits are arranged and mangled to provide a 
last config set.
Change-Id: I5c8f136e743b6beb02956b3436aa3d1010205141

srsenb.conf.tmpl: update to new version
Change-Id: I204d03307aaaddcb15fed6f7310b64d09c43f64c

fix lookup table for maximum DL/UL rate for srsENB
Change-Id: Ieb027eaf5d8f4d774d0e29207f7a3b75dec6ee5b

relax max_rate for iperf UL/DL tests
Change-Id: Ia243156931918d5ab013c64c37823bc2e400798e

process: add stdin_write method to communicate with child on stdin
Change-Id: I3e5d843911998bb50726d93e3dfd3de584dc81a5

srs_ue: send t+Enter to stdin after launching app
this enables the console trace which is printed to stdout and provides
human readable radio statistics
Change-Id: If1c998aff1c5c11d3fa636fb60cb4991e0def768

srs_ue: run srsUE as normal RemoteProcess
running with RemoteProcessFixIgnoreSIGHUP was needed bc srsUE used to
ignore SIGHUP that is delivered to the process when the SSH session is
terminated causing the UE to not quit properly. This isn't the case
anymore so we can use RemoteProcess for spawning the UE.
Furthermore, the wrapper script didn't pass along stdin to it's child
process, i.e., srsUE. Therefore the console trace can't be seen if the
run through the wrapper.
Change-Id: Iab4dafc58167620362a36ec9e904b8d49a7d5fc1

srs{enb,ue}.conf.tmpl: make sure to only use a single PHY thread when
running with ZMQ
Change-Id: Icc1e67705c0354ece77c037f224a9c756a16daaa

srs_ue: add scenario for airplane mode simulation in srsUE
Change-Id: I5db2ff325ce4b4236c1ceec0831dd7420d52dedb

ms: add optional param to specify fixed APN IP address assigned to a MS
this allows to specify a fixed APN address for each subscriber in the 
HLR. If the param is not provided in the resources config, the default 
allocation type is used, which is to dynamically allocate an IP in the 
supported range.
Change-Id: Ie6fcd7987dc7bc62a686bee3a10c67e3dd813419

srs_enb: add num_cells param to srsENB
this is needed to support the upcoming multi-cell feature in srsENB. 
It's required for carrier aggregation (CA) but can also be used for
(currently) two independent 4G cells.
This commit adds baseline support for the parameter. Upcoming commits
will add hooks to configure the cell's EARFCN, etc.
Change-Id: I6d1658bf820163aee03b64b0f4ac68046c243f37

srs_ue: adding num_carriers paramter and corresponding modifier
this allows to configure a UE to support multiple carriers
Change-Id: I4fa0a0c6e52efb17aa0b07e55abb5470d8028e8b

srs_{enb,ue}: add secondary cell and set dl/ul freq in eNB/UE config
accordingly
Change-Id: I3730856003fffda4f3c3f9e835d62ddc312c4c31

srs_enb: enable console trace, use RemoteProcess class
In order for the console trace to work over SSH this commit switches to
use the default RemoteProcess class. Note that srsENB now correctly
handles the SIGHUP signal.
Change-Id: I26391eb8409e14b34987e0b338e10613fa157e06

srs_enb: remove unneeded enb parameters
Change-Id: I1f2e18c6552fc4a6e3a68d75a85744fadf26aacb

srs_enb: remove num_cells getter
Change-Id: Ic5c45bb17440d2ee241840591baf02930a0b3470

srs_ue: remove hard-coded num phy thread param
Change-Id: Ia4fac188ecbe6aba393a68ec0291b51cd1441029

defaults.conf: set num_carriers default value
Change-Id: Ibe94fada0a40c499dd516408756c4ef7988ccec5

Introduce initial Amarisoft ENB support
* object class expects the amarisoft software to be placed in directory
 provided by env var AMARISOFT_PATH_ENB. On local runs, it will run the
 software from there. On remote runs it will copy over that directory to
 the slave.
* Configuration provided works with 6 PRBs, but probably won't work for
 other values (yet).
* ZMQ support not yet available, only UHD.
One can select the Amarisoft ENB by declaring it in resources.conf:
"""
- label: AmarisoftENB-B200
 type: amarisoftenb
 rf_dev_type: uhd
 rf_dev_args:
"type=b200,serial=317B9FE,recv_frame_size=9232,send_frame_size=9232"
 remote_user: jenkins
 addr: 10.12.1.206
"""
And running:
"""
$ export AMARISOFT_PATH_ENB=/path/to/binaries
$ osmo-gsm-tester.py ... -s
4g:amarisoftenb-rftype@uhd+srsue-rftype@UHD+mod-enb-nprb@6 -t ping
"""
Change-Id: I1ddf4962ea6438ad977382ab201b724028ca46b3

srsenb: start_locally(): Fix missing class name in attribute access
Change-Id: I5e11dc9c3c0dcf4808a3ad67fe96580480fe47d1

srsenb: No need to generates values dict for every template file
Let's only do all the dictionary processing once and not for each 
template file.
Change-Id: I4684d26375fce1e2ae28d3eccdc4f607fa5411a9

srsue: start_locally(): Fix missing class name in attribute access
Change-Id: Iab4196b12ee12a06ba2129053f2291c1e4c284ff

{srs,amarisoft}enb:Move shared code to parent generic class eNodeB
Change-Id: Ica0f1d4179a905d6dfc77d3e8eb08ac6cd5c3d2e

amarisoft_enb: Drop unused lib variable
Change-Id: Iea3e77a4b9f25f2781f617cd4bfd0ba4b34c709e

Introduce support for AmarisoftEPC
* A new abstract generic base class EPC is created
* srsEPC and AmarisoftEPC inherit from that class
* options are loaded from defaults.conf in cascade. First generic "epc",
 afterwards the specific enb type.
* A new scenario is added to select the EPC type to use. srsEPC is the
 default unless stated by an scenario.
* AmarisoftEPC delegates setup of the tun IP address to an "ifup"
 script. As a result, since we run without root permissions (ony with
 CAP_NET_ADMIN), the ifup script itself is unablet o set the IP
 address. To solve this, we introduce a new osmo-gsm-tester helper
 script which must be installed in the slave node which can be called
 through sudo to increase privileges to do so.
With this commit, I can already get srsUE<->amarisoftENB<->amarisoftEPC 
to pass ping and iperf3 4g tests.
Change-Id: Ia50ea6a74b63b2d688c8d683aea11416ad40a6d3

srs_{ue,enb}: change UHD stream args and buffer size as function of cell
width for B2XX
Change-Id: I2de0526ee9bd2f7bb75c7d529b3f52d75117c5cc

Make amarisoft license server optional
* Since license_server_addr logic is generic among all amarisoft
 software, let's move it to its own config category "amarisoft", which
 all amarisoft classes (amarisoft_{enb,epc}.py as of now) will import
 during configure().
* The license server is disabled by default (value 0.0.0.0) since the
 usual case is to have a license in the local system.
* the license server can be enabled using scenario
 cfg-amarisoft-license@A.B.C.D where A.B.C.D is the address to use.
Change-Id: Id5456ad9497e91370857b80ff9a389111d3611ff

iperf3: add config to adjust the duration of the iperf run
the time is passed as a string param and is then converted into seconds
when literals "h" or "m" are found. So it would accept 2m and would
convert it to 120s, for example.
Example:
+cfg-iperf3-time@15+
Change-Id: Iff28816f83670751e9e91de31ec59b1b0ad8fc0d

srsenb_rr.conf.temp: patch scheduler variables for 6 PRB cells
Change-Id: Ifdab42da8184cdf8367c5be1639a0faea3e68db1

srs{enb,ue}: Use lowcase UHD as rf_type string
Older versions of srsLTe.git required passing UHD in caps to the conf 
file, but that's not the case anymore. Let's use lowcase then.
Change-Id: If8a289ba110229d31b62b0d3adc1ae97324e6d27

config.py: Allow escaping commas in parametrized scenario names
comma character ',' is used in the command line to split between 
parameters, which means a parameter value itself couldn't contain it. 
This commit allows passing a comma inside a character value by escaping 
it with '\,'.
Change-Id: Ic0bd9a029137a59e8c4a32b807eba7a64fcfa51f

srs_{ue,enb}: Add modifier to pass arbitrary cmd line arguments
Change-Id: Ie438c8aa9992dbc61b2133473a83d565cf5c7056

Introduce contrib/jenkins-build-amarisoft.sh
Prepares inst tgz archives to be consumed by osmo-gsm-tester from 
Amarisoft prebuilt binaries.
Change-Id: I372606fc5ff9f6bb3ef848f2fe229b83382ddc09

amarisoft_enb: Add missing default values in constructor
Change-Id: I61c04df14468b38cfe4afaaeb5404367838ff0a9

contrib/jenkins-build-amarisoft.sh: Make build of amarisoft trx_zmq.so
optional
That module is only available so far in a private repository, so allow 
still other amarisof tusers to build the bundles with UHD support.
Change-Id: I361f179c8e7f379173fa9109028e139fc4c8c251

contrib/jenkins-build-amarisoft.sh: Fix typo copying trx_uhd.so to UE
Change-Id: I0c9b83b9adf0548e78363462744644bc528c2d29

Add modem resource features attributes: 2g, 3g, 4g
This allows selecting modem by supported network type. It will allow 
using 4G modems with type different than srsue in 4g suite in the
future.
Change-Id: I38bcf6abf789f52c3ed0bee7911567fa872e2491