... | ... | @@ -185,7 +185,267 @@ These modes do not create log files nor generate an HTML report. |
|
|
|
|
|
The TesteNB mode corresponds to the Build and Test stage. This is the main mode.
|
|
|
|
|
|
In this mode, the python script parses the XML file (provided as an argument to option `--XMLTestFile`).
|
|
|
In this mode, the python script parses the XML file (provided as an argument to option `--XMLTestFile`). The latter contains a list of test-cases to play. Each test-case has a class of actions to perform with optional arguments. The XML syntax is described in the next section.
|
|
|
|
|
|
A complete HTML report (`test_results.html`) is generated at the start of the mode and appended after each testcase with a table row that contains:
|
|
|
|
|
|
- Testcase ID
|
|
|
- Testcase Class
|
|
|
- Testcase optional arguments
|
|
|
- Status on each used UE
|
|
|
|
|
|
An example is visible [here](../documents/test_results-CI-Sandbox-Ran.html).
|
|
|
|
|
|
The UE-based classes like ('Initialize_UE', 'Attach_UE', 'Detach_UE', 'Ping', 'Iperf' or 'Reboot_UE') are executed following the same skeleton:
|
|
|
|
|
|
1. Connect to the ADB server and retrieve the list of connected devices
|
|
|
2. For each UE in the devices list do
|
|
|
1. Start a process on the associated method with device ID as parameter
|
|
|
2. The associated method performs the class actions in a child process in parallel of all child processes.
|
|
|
3. The parent process waits that all child processes are finished.
|
|
|
4. It generates a HTML row to append to the HTML file.
|
|
|
1. In case of available statistics per UE, each cell per UE in the row is populated.
|
|
|
|
|
|
# 5. XML test case file #
|
|
|
|
|
|
## 5.1. Ordered list of executed tests ##
|
|
|
|
|
|
The xml tag is **`TestCaseRequestedList`**. The list is ordered and test-case ID can be used several times.
|
|
|
|
|
|
Important: if a test-case ID is described in the rest of the XML, but not present in the requested list, it won't be executed.
|
|
|
|
|
|
## 5.2. Build_eNB class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Build_eNB</class>
|
|
|
<desc>Build eNB (USRP)</desc>
|
|
|
<Build_eNB_args>-w USRP -c --eNB</Build_eNB_args>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
The **`Build_eNB_args`** are usual build_oai options.
|
|
|
|
|
|
## 5.3. Initialize_eNB class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Initialize_eNB</class>
|
|
|
<desc>Initialize eNB (FDD/Band7/5MHz)</desc>
|
|
|
<Initialize_eNB_args>-O ci-scripts/conf_files/enb.band7.tm1.25PRB.usrpb210.conf</Initialize_eNB_args>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
The **`Initialize_eNB_args`** are usual lte-softmodem options. Note that the path to configuration file is relative to the root of the GIT repository. The python script will append the absolute path at runtime.
|
|
|
|
|
|
## 5.4. TerminateeNB class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Terminate_eNB</class>
|
|
|
<desc>Terminate eNB</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. But this test-case can be used several times in the ordered list.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.5. Initialize_HSS class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Initialize_HSS</class>
|
|
|
<desc>Initialize HSS</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.6. TerminateHSS class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>TerminateHSS</class>
|
|
|
<desc>Terminate HSS</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.7. Initialize_MME class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Initialize_MME</class>
|
|
|
<desc>Initialize MME</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.8. TerminateMME class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>TerminateMME</class>
|
|
|
<desc>Terminate MME</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.9. Initialize_SPGW class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Initialize_SPGW</class>
|
|
|
<desc>Initialize MME</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.10. TerminateSPGW class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>TerminateSPGW</class>
|
|
|
<desc>Terminate SPGW</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.11. Initialize_UE class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Initialize_UE</class>
|
|
|
<desc>Initialize UEs</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. But this test-case can be used several times in the ordered list.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.12. Terminate_UE class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Terminate_UE</class>
|
|
|
<desc>Terminate UEs</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. But this test-case can be used several times in the ordered list.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.13. Attach_UE class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Attach_UE</class>
|
|
|
<desc>Attach UEs</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. But this test-case can be used several times in the ordered list.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.14. Detach_UE class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Detach_UE</class>
|
|
|
<desc>Detach UEs</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. But this test-case can be used several times in the ordered list.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.15. Reboot_UE class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Reboot_UE</class>
|
|
|
<desc>Reboot UEs</desc>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. But this test-case can be used several times in the ordered list.
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
## 5.16. Ping class ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Ping</class>
|
|
|
<desc>ping (5MHz - 20 sec)</desc>
|
|
|
<ping_args>-c 20</ping_args>
|
|
|
<ping_packetloss_threshold>5</ping_packetloss_threshold>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. **It is important to duplicate this test-case if ran multiple times with different eNB configurations but the same ping options. Each execution will generate different log file to archive.**
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
The **`ping_args`** tag provides typical linux ping options.
|
|
|
|
|
|
The **`ping_packetloss_threshold`** tag provides a percentage threshold on lost ping packet. Above that number, the test-case will be considered a failure and the python will not execute the following test-cases in the list.
|
|
|
|
|
|
## 5.17. Iperfclass ##
|
|
|
|
|
|
```xml
|
|
|
<testCase id="xyz">
|
|
|
<class>Iperf</class>
|
|
|
<desc>iperf (5MHz - DL/6Mbps/UDP)(60 sec)</desc>
|
|
|
<iperf_args>-u -b 6M -t 60 -i 1</iperf_args>
|
|
|
<iperf_packetloss_threshold>50</iperf_packetloss_threshold>
|
|
|
<iperf_profile>single-ue</iperf_profile>
|
|
|
</testCase>
|
|
|
```
|
|
|
|
|
|
The **`id`** tag SHALL be unique. **It is important to duplicate this test-case if ran multiple times with different eNB configurations but the same iperf options. Each execution will generate different log file to archive.**
|
|
|
|
|
|
The **`desc`** tag provides a description of the test-case. No syntax required.
|
|
|
|
|
|
The **`iperf_args`** tag provides a list of usual iperf options. `-u` is quite mandatory and `-i 1` as well. Adding `-R` option will perform a uplink test.
|
|
|
|
|
|
The **`iperf_packetloss_threshold`** tag provides a percentage threshold on lost iperf packet. Above that number, the test-case will be considered a failure and the python will not execute the following test-cases in the list.
|
|
|
|
|
|
The **`iperf_profile`** tag provides a profile on bandwidth requirement per UE device. It is optional. If not present, the default value will be used. The supported values are:
|
|
|
|
|
|
- 'balanced'. The default. The requested bandwidth will equally divided by the number of connected UE devices.
|
|
|
- 'single-ue'. The requested bandwidth will tested on a single UE device.
|
|
|
- 'unbalanced'. The requested bandwidth will be reparted like this:
|
|
|
- 2% of the requested bandwidth on (N-1) UE devices
|
|
|
- The rest (= 100 - 2*(N-1)) on the last device
|
|
|
|