Skip to content

GitLab

openairinterface5G
openairinterface5G

OAITestSetup

Last edited by Cedric Roux
Page history

Note: the documentation below does not reflect anymore the exact setup we use at EURECOM. See this file for a short description of the current configuration.

OAI_Test_Setup_only_figure.png

OAI test setup is intended to be used to make reproducible tests (compilation + simulation + Real HW) for each version committed to develop/master branch of openairinterface5g/openair-cn Gitlab repository. Here are the important points to be noted:

  • Please do not install OAI EPC on ALU EPC or vice versa. The two EPC installations are incompatible and mess up with each other.

  • The figure describing OAI Test setup is located here: OAITestSetup/OAI_Test_Setup_only_figure.docx

  • Calisson to be always used as eNB. Stevens to be always used as UE. Nano to be always used as core network (OAI EPC). Amerique to be always used as core network (ALU EPC).

  • Band 7 is used because of duplexers

  • eNB USRP is on band 7, No of antennas=2TX/2RX .

  • eNB EXMIMO is on band 7, No of antennas=2TX/2RX .

  • UE USRP is on band 7, No of antennas=1TX/2RX .

  • UE EXMIMO is on band 7, No of antennas=1TX/2RX .

  • Mozart is server grade PC with 10 CPUs, so it is best to use it to run autotests (./build_oai -s) for performance reasons. It also has USRP X310 connected to it.

  • Real-time tests with hardware must be run for following scenarios:

    1. Test A: OAI eNB (calisson)(noS1) <-> OAI UE (stevens)(noS1) for both USRP B210 (TDD/FDD) + EXMIMO (TDD + FDD) + 5/10/20 MHz + TM 1,2
    2. Test B: OAI EPC(calisson) + OAI eNB (calisson) <-> COTS UE (stevens) for both USRP B210 (TDD/FDD) /EXMIMO (TDD + FDD) + 5/10/20 MHz + TM 1,2
    3. Test C: OAI EPC(nano) + OAI eNB (calisson) <-> COTS UE (stevens) for both USRP B210 (TDD/FDD)/EXMIMO (TDD + FDD) + 5/10/20 MHz + TM 1,2
    4. Test D: ALU EPC(Amerique) + OAI eNB (calisson) <-> COTS UE (stevens) for both USRP B210/EXMIMO (TDD + FDD) + 5/10/20 MHz + TM 1,2
    5. Test with two COTS UEs (low priority for current EURECOM projects)
    6. TM 5,6 do not work, so excluded currently from testing
    7. TDD tests are not possible with USRP
    8. TDD stick is needed for TDD tests. For the moment we do not do TDD tests as we do not have EXMIMO TDD and USRP does not have TDD switch
    9. The tests are mostly throughput tests by running iperf for 5 minutes

  • All the test results need to be stored in following directory: /mnt/sradio/TEST_RESULTS (check if it is mounted)

  • Assuming your version for develop branch is 1234. Then do the following steps:

  • Merge from develop to YOUR_BRANCH before running tests. Please commit any modified files into your branch (except some config files for UE/eNB)

  • For autotests (./build_oai -s)

    • The test results for autotests (./build_oai -s) are located in autotests/log
    • /mnt/sradio/TEST_RESULTS/BRANCH.NAME/BRANCH.COMMIT_ID/log
    • cp -rf autotests/log /mnt/sradio/TEST_RESULTS/BRANCH.NAME/BRANCH.VERSION/log

Autotests (compilation, execution test case class)

The autotests are currently present in branch feature-34-test_framework. The test cases are defined in config file cmake_targets/autotests/test_case_list.xml

The brief description of tests cases is in cmake_targets/autotests/README.txt

./build_oai -s (to run all the autotests)

./build_oai -s --run-group "01010* 015000" #To run specific test cases in group. This will only run (compilation, oaisim tests)

Autotests (lte-softmodem test case class)

See .gitlab-ci.yml for info on how python script is run for testing UE, eNB and EPC

Test results are stored in (EURECOM private repository)

  • /mnt/sradio/TEST_RESULTS/BRANCH.NAME/BRANCH.COMMIT_ID/log
  • TODO: The results will be mirrored on dropbox/google drive automatically

UE Autotests (lte-softmodem-noS1 test case class)

OAI UE test setup

Test Cases that must pass to merge any branch to develop.

  • 0101+, 0103+, 0150+, 0151+, 0155+, 0157+, 0158+, 0161+, 0170+, 0175+, 0255+
  • The README.txt file for the explanation of above numbers, $OPENAIR_DIR/cmake_targets/autotests/README.txt
  • BladeRF tests (0165+) removed from testbench as it sends strong DC signal

Test cases that must pass to merge from develop to master

  • Please list exact Gitlab CI command with all the environment variable settings

Non-working Autotests

The following autotests are known to be broken:

  • RRH Compilation (010140, 010141 010142)
  • oaisim_noS1 (TM 5,6)
  • oaisim_noS1: 1 UE - 1 eNB (without PHY_abstraction)
  • 015100.dlsim.test5
  • MBSFN Tests Group: 0104*
  • PHY Unitary tests Group: 0150*
  • 20MHz lte-softmodem on USRP B210/EXMIMO
  • TDD tests have issues (oaisim + lte-softmodem)
  • Tests involving 2 antennas

Example Commands

    # Example command to trigger all the tests of your branch. The token is present here, https://gitlab.eurecom.fr/oai/openairinterface5g/triggers
    curl -X POST -F token=___TOKEN___ https://gitlab.eurecom.fr/ci/api/v1/projects/1/refs/__YOUR_BRANCH__/trigger
    # Examples of commands to trigger gitlab CI for you branch but only the tests with prefix 0101+ and 0102+. You can use the command below to also set other Gitlab CI environment variables. The token is present here, https://gitlab.eurecom.fr/oai/openairinterface5g/triggers
    curl -X POST -F token=___TOKEN___ -F "variables[OAI_TEST_CASE_GROUP]=\"0101+ 0102+ \" " https://gitlab.eurecom.fr/ci/api/v1/projects/1/refs/__YOUR_BRANCH__/trigger

Shared drive for storing test results (internal to EURECOM servers)

  • Add the line below to your /etc/fstab to mount the TEST_RESULTS
tyne:/global/umts2/share/packages/sradio  /mnt/sradio  nfs  auto 0 0

Instructions for replicating the test setup remotely

  • You need to install all OAI dependencies manually. The Gitlab CI scripts do not install the OAI dependencies automatically
  • You need to setup the network as shown in figure above. After that, you can setup aliases in DNS for machines (nano, calisson, mozart, amerique and stevens) based on your configuration.
  • Add a gitlab user and password that has sudo access without password on all the machines. You need to add the following lines in /etc/sudoers of ALL the machines
  • USER_NAME ALL=(ALL:ALL) ALL
  • USER_NAME ALL=(ALL:ALL) NOPASSWD: ALL
  • Install Gitlab runner, https://gitlab.com/gitlab-org/gitlab-ci-multi-runner
  • Configure Gitlab CI Server. You can create your own private project on Gitlab server by cloning existing openairinterface5g/openairCN repository and try out your settings
  • You need to register the Gitlab runner with Gitlab CI server using token and gitlab URL which are preset on the Gitlab CI server settings
  • You need to configure the Gitlab CI variables within your private project. You can look at existing Gitlab variables and their settings here, https://gitlab.eurecom.fr/oai/openairinterface5g/variables
  • You can then run the Gitlab CI based on presentation, OAI_testing_v1.pptx

Latest stable run on develop branch

Branch (Tag) Location
develop (tag v0.5 + additional commits) /mnt/sradio/TEST_RESULTS/develop/5a3b3ed0/log/

TODO

  • UE Dongle for TDD/FDD that supports AT commands
  • LTE Phone for TDD/FDD that supports AT commands
  • EXMIMO TDD card (x2)
  • Extra Antennas for TDD
  • Setup for CA testbench using EXMIMO
  • Setup for DRB, handover
  • Automated test setup integration with python(make pre-ci) and JuJu (Charm)
  • ALL: To review test cases (lte-softmodem) above and also cmake_targets/autotests/README.txt located in branch feature-34-test_framework (openairinterface5g repository)
  • Test cases for RRH/ARM/BladeRF (once the software is stable and released)
  • Feedback: Speedup oaisim_nos1 by using pre-compile cache generated from build_oai script
  • Wireshark pcap support for DCI, PDCCH
  • PHY layer logging (Todo: Florian)
  • Automated installation of Core NW/eNB pre-requisite SW on every run (currently all the dependencies are pre-installed)
  • Integrate with OAI EPC, currently only integrated with ALU EPC
  • Tests for OAI UE (w/live network and w/ OAI eNB)
  • Better Integration with GitLab CI
  • Mirror the results on Dropbox/Google Drive as well
  • Tests for BladeRF, Odroid ARM
  • Basic documentation/tutorial on running autotests/ setting up machines, Gitlab, etc.
  • List test cases which need to be passed as a minimum (for lte-softmodem) before merge to develop branch -> need discussion with team
  • Any ideas on how to reduce the test time -> need discussion (current time of 1 run of lte-softmodem test case is about 10 mins)
  • Test Setup Duplexers (Email from Dominique)
  • Setup Testbench for SODERA/BladeRF
  • Band 38 filter for TDD -> request from Dominique
  • Add rules for DRB establishment -> AI on Navid
  • Add wireshark trace to test DRB , PHY/wireshark logs for error
  • Have a web front-end where everything is accessible by just clicking everything (Comment from Cedric)
  • Wireshark traces are disabled due to error. Needs to be fixed asap
  • Fix LimeSDR tests, Install 16.04, fix 20 MHz tests, multi-UE tests, fix OAI EPC testbench, integgrate speedtest script in docker
  • Add oaisim tests with s1

Attachments