From c832d42eb46e94722156fdd7455a79bf2f34ca72 Mon Sep 17 00:00:00 2001
From: Raphael Defosseux <raphael.defosseux@eurecom.fr>
Date: Wed, 30 Mar 2022 16:15:31 +0200
Subject: [PATCH] doc(l2sim): adding a light tutorial for L2 sim + proxy using
containers
Signed-off-by: Raphael Defosseux <raphael.defosseux@eurecom.fr>
---
ci-scripts/yaml_files/5g_l2sim_tdd/README.md | 355 +++++++++++++++++++
doc/L2NFAPI.md | 2 +
2 files changed, 357 insertions(+)
create mode 100644 ci-scripts/yaml_files/5g_l2sim_tdd/README.md
diff --git a/ci-scripts/yaml_files/5g_l2sim_tdd/README.md b/ci-scripts/yaml_files/5g_l2sim_tdd/README.md
new file mode 100644
index 00000000000..531f2406ef8
--- /dev/null
+++ b/ci-scripts/yaml_files/5g_l2sim_tdd/README.md
@@ -0,0 +1,355 @@
+<table style="border-collapse: collapse; border: none;">
+ <tr style="border-collapse: collapse; border: none;">
+ <td style="border-collapse: collapse; border: none;">
+ <a href="http://www.openairinterface.org/">
+ <img src="../../../doc/images/oai_final_logo.png" alt="" border=3 height=50 width=150>
+ </img>
+ </a>
+ </td>
+ <td style="border-collapse: collapse; border: none; vertical-align: center;">
+ <b><font size = "5">OAI Full Stack 5G-NR L2 simulation with containers and a proxy</font></b>
+ </td>
+ </tr>
+</table>
+
+This page is only valid for an `Ubuntu18` host.
+
+This tutorial is only valid once this file is merged into the `develop` branch.
+
+# 1. Retrieving the images on Docker-Hub #
+
+Currently the images are hosted under the user account `rdefosseoai`.
+
+This may change in the future.
+
+Once again you may need to log on [docker-hub](https://hub.docker.com/) if your organization has reached pulling limit as `anonymous`.
+
+```bash
+$ docker login
+Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
+Username:
+Password:
+```
+
+Now pull images.
+
+```bash
+$ docker pull mysql:5.7
+$ docker pull rdefosseoai/oai-amf:latest
+$ docker pull rdefosseoai/oai-nrf:latest
+$ docker pull rdefosseoai/oai-smf:latest
+$ docker pull rdefosseoai/oai-spgwu-tiny:latest
+
+$ docker pull rdefosseoai/oai-gnb:develop
+$ docker pull rdefosseoai/oai-nr-ue:develop
+$ docker pull rdefosseoai/proxy:latest
+```
+
+And **re-tag** them for tutorials' docker-compose file to work.
+
+```bash
+$ docker image tag rdefosseoai/oai-amf:latest oai-amf:latest
+$ docker image tag rdefosseoai/oai-nrf:latest oai-nrf:latest
+$ docker image tag rdefosseoai/oai-smf:latest oai-smf:latest
+$ docker image tag rdefosseoai/oai-spgwu-tiny:latest oai-spgwu-tiny:latest
+
+$ docker image tag rdefosseoai/oai-gnb:develop oai-gnb:develop
+$ docker image tag rdefosseoai/oai-nr-ue:develop oai-nr-ue:develop
+$ docker image tag rdefosseoai/proxy:latest oai-lte-multi-ue-proxy:latest
+```
+
+```bash
+$ docker logout
+```
+
+Note that the proxy image is based on the source available at [https://github.com/EpiSci/oai-lte-5g-multi-ue-proxy](https://github.com/EpiSci/oai-lte-5g-multi-ue-proxy).
+
+At time of writing, the `latest` tag corresponded to `56cfdc046a5f96d5e67d42a2fc2bf6ba2fe58b41` commit.
+
+# 2. Deploy containers #
+
+**CAUTION: this SHALL be done in multiple steps.**
+
+**Just `docker-compose up -d` WILL NOT WORK!**
+
+All the following commands **SHALL** be run from the `ci-scripts/yaml_files/5g_l2sim_tdd` folder.
+
+The `gNB`, `proxy` and `NR-UE` containers will be deployed in `host`-mode. It will use the host loopback interface to connect.
+
+```bash
+sudo ifconfig lo: 127.0.0.2 netmask 255.0.0.0 up
+```
+
+## 2.1. Deploy OAI 5G Core Network ##
+
+```bash
+$ cd ci-scripts/yaml_files/5g_l2sim_tdd
+$ docker-compose up -d mysql oai-nrf oai-amf oai-smf oai-spgwu oai-ext-dn
+Creating network "l2sim-oai-public-net" with driver "bridge"
+Creating network "l2sim-oai-traffic_net-net" with driver "bridge"
+Creating l2sim-oai-nrf ... done
+Creating l2sim-mysql ... done
+Creating l2sim-oai-spgwu ... done
+Creating l2sim-oai-amf ... done
+Creating l2sim-oai-smf ... done
+Creating l2sim-oai-ext-dn ... done
+```
+
+Wait for a bit.
+
+```bash
+$ docker-compose ps -a
+ Name Command State Ports
+-------------------------------------------------------------------------------------------------
+l2sim-mysql docker-entrypoint.sh mysqld Up (healthy) 3306/tcp, 33060/tcp
+l2sim-oai-amf /bin/bash /openair-amf/bin ... Up (healthy) 38412/sctp, 80/tcp, 9090/tcp
+l2sim-oai-ext-dn /bin/bash -c apt update; ... Up (healthy)
+l2sim-oai-nrf /bin/bash /openair-nrf/bin ... Up (healthy) 80/tcp, 9090/tcp
+l2sim-oai-smf /bin/bash /openair-smf/bin ... Up (healthy) 80/tcp, 8805/udp, 9090/tcp
+l2sim-oai-spgwu /openair-spgwu-tiny/bin/en ... Up (healthy) 2152/udp, 8805/udp
+```
+
+At this point, you can prepare a capture on the newly-created public docker bridges:
+
+```bash
+$ ifconfig
+...
+l2sim-public: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
+ inet 192.168.71.129 netmask 255.255.255.192 broadcast 192.168.71.191
+ inet6 fe80::42:c4ff:fe2b:3d38 prefixlen 64 scopeid 0x20<link>
+ ether 02:42:c4:2b:3d:38 txqueuelen 0 (Ethernet)
+ RX packets 4 bytes 112 (112.0 B)
+ RX errors 0 dropped 0 overruns 0 frame 0
+ TX packets 7 bytes 626 (626.0 B)
+ TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
+
+l2sim-traffic: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
+ inet 192.168.72.129 netmask 255.255.255.192 broadcast 192.168.72.191
+ inet6 fe80::42:b5ff:fed3:e732 prefixlen 64 scopeid 0x20<link>
+ ether 02:42:b5:d3:e7:32 txqueuelen 0 (Ethernet)
+ RX packets 2652 bytes 142335 (142.3 KB)
+ RX errors 0 dropped 0 overruns 0 frame 0
+ TX packets 3999 bytes 23367972 (23.3 MB)
+ TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
+...
+
+$ sudo nohup tshark -f "(host 192.168.72.135 and icmp) or (not host 192.168.72.135 and not arp and not port 53 and not port 2152 and not port 2153)" -i l2sim-public -i l2sim-traffic -w /tmp/capture_5g_l2sim_tdd.pcap > /tmp/tshark.log 2>&1 &
+```
+
+## 2.2. Deploy OAI gNB in Standalone Mode as a VNF ##
+
+**CAUTION: To execute this 2nd step, the whole `CN5G` SHALL be in `healthy` state (especially the `mysql` container).**
+
+```bash
+$ docker-compose up -d oai-gnb
+l2sim-oai-nrf is up-to-date
+l2sim-oai-spgwu is up-to-date
+l2sim-oai-ext-dn is up-to-date
+Creating l2sim-oai-gnb ... done
+```
+
+Wait for a bit.
+
+```bash
+$ docker-compose ps -a
+ Name Command State Ports
+-------------------------------------------------------------------------------------------------
+l2sim-mysql docker-entrypoint.sh mysqld Up (healthy) 3306/tcp, 33060/tcp
+l2sim-oai-amf /bin/bash /openair-amf/bin ... Up (healthy) 38412/sctp, 80/tcp, 9090/tcp
+l2sim-oai-ext-dn /bin/bash -c apt update; ... Up (healthy)
+l2sim-oai-gnb /opt/oai-gnb/bin/entrypoin ... Up (healthy)
+l2sim-oai-nrf /bin/bash /openair-nrf/bin ... Up (healthy) 80/tcp, 9090/tcp
+l2sim-oai-smf /bin/bash /openair-smf/bin ... Up (healthy) 80/tcp, 8805/udp, 9090/tcp
+l2sim-oai-spgwu /openair-spgwu-tiny/bin/en ... Up (healthy) 2152/udp, 8805/udp
+```
+
+You can verify that the `gNB` is connected with the `AMF`:
+
+```bagh
+$ docker logs rfsim5g-oai-amf
+...
+[AMF] [amf_app] [info ] |----------------------------------------------------gNBs' information-------------------------------------------|
+[AMF] [amf_app] [info ] | Index | Status | Global ID | gNB Name | PLMN |
+[AMF] [amf_app] [info ] | 1 | Connected | 0x0 | gnb-l2sim-vnf | 208, 99 |
+[AMF] [amf_app] [info ] |----------------------------------------------------------------------------------------------------------------|
+...
+```
+
+## 2.3. Deploy OAI NR-UE and proxy
+
+```bash
+$ docker-compose up -d proxy oai-nr-ue0
+l2sim-mysql is up-to-date
+l2sim-oai-nrf is up-to-date
+l2sim-oai-amf is up-to-date
+l2sim-oai-smf is up-to-date
+l2sim-oai-spgwu is up-to-date
+l2sim-oai-ext-dn is up-to-date
+l2sim-oai-gnb is up-to-date
+Creating l2sim-oai-nr-ue ... done
+Creating l2sim-proxy ... done
+```
+
+Wait for a bit.
+
+```bash
+$ docker-compose ps -a
+ Name Command State Ports
+-------------------------------------------------------------------------------------------------
+l2sim-mysql docker-entrypoint.sh mysqld Up (healthy) 3306/tcp, 33060/tcp
+l2sim-oai-amf /bin/bash /openair-amf/bin ... Up (healthy) 38412/sctp, 80/tcp, 9090/tcp
+l2sim-oai-ext-dn /bin/bash -c apt update; ... Up (healthy)
+l2sim-oai-gnb /opt/oai-gnb/bin/entrypoin ... Up (healthy)
+l2sim-oai-nr-ue0 /opt/oai-nr-ue/bin/entrypo ... Up (healthy)
+l2sim-oai-nrf /bin/bash /openair-nrf/bin ... Up (healthy) 80/tcp, 9090/tcp
+l2sim-oai-smf /bin/bash /openair-smf/bin ... Up (healthy) 80/tcp, 8805/udp, 9090/tcp
+l2sim-oai-spgwu /openair-spgwu-tiny/bin/en ... Up (healthy) 2152/udp, 8805/udp
+l2sim-proxy /oai-lte-multi-ue-proxy/bi ... Up (healthy)
+
+$ docker stats --no-stream --format "table {{.Container}}\t{{.CPUPerc}} {{.MemUsage}}\t{{.MemPerc}}" l2sim-mysql l2sim-oai-amf l2sim-oai-ext-dn l2sim-oai-gnb l2sim-oai-nr-ue0 l2sim-oai-nrf l2sim-oai-smf l2sim-oai-spgwu l2sim-proxy
+CONTAINER CPU % MEM USAGE / LIMIT MEM %
+l2sim-mysql 0.03% 206.7MiB / 62.54GiB 0.32%
+l2sim-oai-amf 4.05% 29.49MiB / 62.54GiB 0.05%
+l2sim-oai-ext-dn 0.00% 31.27MiB / 62.54GiB 0.05%
+l2sim-oai-gnb 1.29% 1.853GiB / 62.54GiB 2.96%
+l2sim-oai-nr-ue0 1.43% 350.8MiB / 62.54GiB 0.55%
+l2sim-oai-nrf 0.21% 9.105MiB / 62.54GiB 0.01%
+l2sim-oai-smf 3.24% 30.23MiB / 62.54GiB 0.05%
+l2sim-oai-spgwu 0.00% 11.78MiB / 62.54GiB 0.02%
+l2sim-proxy 6.97% 290.4MiB / 62.54GiB 0.45%
+```
+
+**CAUTION: As you can see the CPU usage is not that important compared to a 5G RF simulator equivalent.**
+
+But the CPU speed matters; I am running on a fast server:
+
+```bash
+$ lscpu
+Architecture: x86_64
+CPU(s): 16
+...
+Model name: Intel(R) Xeon(R) Silver 4215R CPU @ 3.20GHz
+```
+
+I tried on a slower server with more CPUs and it did not work:
+
+```bash
+oaici@orion:~$ lscpu
+Architecture: x86_64
+CPU(s): 48
+...
+Model name: Intel(R) Xeon(R) CPU E5-2658 v3 @ 2.20GHz
+```
+
+We will work on this issue.
+
+
+Making sure the OAI UE is connected:
+
+```bash
+$ docker exec -it l2sim-oai-nr-ue /bin/bash
+root@bb4d400a832d:/opt/oai-nr-ue# ifconfig
+eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
+ inet 192.168.71.137 netmask 255.255.255.192 broadcast 192.168.71.191
+ ether 02:42:c0:a8:47:89 txqueuelen 0 (Ethernet)
+ RX packets 224259 bytes 5821372018 (5.8 GB)
+ RX errors 0 dropped 0 overruns 0 frame 0
+ TX packets 235916 bytes 7848786376 (7.8 GB)
+ TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
+
+lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
+ inet 127.0.0.1 netmask 255.0.0.0
+ loop txqueuelen 1000 (Local Loopback)
+ RX packets 0 bytes 0 (0.0 B)
+ RX errors 0 dropped 0 overruns 0 frame 0
+ TX packets 0 bytes 0 (0.0 B)
+ TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
+
+oaitun_ue1: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
+ inet 12.1.1.2 netmask 255.255.255.0 destination 12.1.1.2
+ unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 500 (UNSPEC)
+ RX packets 0 bytes 0 (0.0 B)
+ RX errors 0 dropped 0 overruns 0 frame 0
+ TX packets 0 bytes 0 (0.0 B)
+ TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
+```
+
+# 3. Check traffic #
+
+## 3.1. Check your Internet connectivity ##
+
+You can also check with the `ext-dn` container (IP address is `192.168.72.135` in docker-compose)
+
+```bash
+$ docker exec -it l2sim-oai-nr-ue /bin/bash
+root@bb4d400a832d# ping -I oaitun_ue1 -c 20 192.168.72.135
+PING 192.168.72.135 (192.168.72.135) from 12.1.1.2 oaitun_ue1: 56(84) bytes of data.
+64 bytes from 192.168.72.135: icmp_seq=1 ttl=63 time=65.1 ms
+64 bytes from 192.168.72.135: icmp_seq=2 ttl=63 time=74.2 ms
+...
+64 bytes from 192.168.72.135: icmp_seq=19 ttl=63 time=25.0 ms
+64 bytes from 192.168.72.135: icmp_seq=20 ttl=63 time=34.7 ms
+
+--- 192.168.72.135 ping statistics ---
+20 packets transmitted, 20 received, 0% packet loss, time 19025ms
+rtt min/avg/max/mdev = 16.413/120.639/209.673/57.159 ms
+```
+
+# 4. Un-deployment #
+
+```bash
+$ docker-compose down
+Stopping l2sim-oai-nr-ue2 ... done
+Stopping l2sim-oai-nr-ue ... done
+Stopping l2sim-oai-gnb ... done
+Stopping l2sim-oai-ext-dn ... done
+Stopping l2sim-oai-spgwu ... done
+Stopping l2sim-oai-smf ... done
+Stopping l2sim-oai-amf ... done
+Stopping l2sim-oai-nrf ... done
+Stopping l2sim-mysql ... done
+Removing l2sim-oai-nr-ue2 ... done
+Removing l2sim-oai-nr-ue ... done
+Removing l2sim-oai-gnb ... done
+Removing l2sim-oai-ext-dn ... done
+Removing l2sim-oai-spgwu ... done
+Removing l2sim-oai-smf ... done
+Removing l2sim-oai-amf ... done
+Removing l2sim-oai-nrf ... done
+Removing l2sim-mysql ... done
+Removing network l2sim-oai-public-net
+Removing network l2sim-oai-traffic-net
+```
+
+# 5. Adapt the `docker-compose` to your environment #
+
+In the `SMF` section, provide your own DNS IP address:
+
+```yaml
+ DEFAULT_DNS_IPV4_ADDRESS=192.168.18.129
+```
+
+In the `gNB` section, provide your docker-host primary IP address and interface name: in our case `192.168.18.193` and `eno1`.
+
+```yaml
+ GNB_NGA_IF_NAME: eno1
+ GNB_NGA_IP_ADDRESS: 192.168.18.193
+ GNB_NGU_IF_NAME: eno1
+ GNB_NGU_IP_ADDRESS: 192.168.18.193
+```
+
+Same thing in the `nr-ue` section:
+
+```yaml
+ NR_UE_NFAPI_IF_NAME: eno1
+```
+
+This tutorial is a first draft. This nFAPI feature and the proxy are still under development.
+
+At time of writing, we were able to run in `host-mode`, 1 `NR-UE` and just ping traffic.
+
+Later development will include:
+
+ - deploying `gNB-VNF`, `proxy` and `UE` in isolated containers (with their own IP address)
+ - more UEs
+ - more traffic (`UDP` and `TCP`)
diff --git a/doc/L2NFAPI.md b/doc/L2NFAPI.md
index 5b8c0f546cf..77f9e5cb783 100644
--- a/doc/L2NFAPI.md
+++ b/doc/L2NFAPI.md
@@ -46,6 +46,8 @@ This proxy allows to perform L2 nFAPI simulator for:
* 5G-NSA
* 5G-SA
+Another tutorial for 5G SA mode with 1 User is available [here](../ci-scripts/yaml_files/5g_l2sim_tdd/README.md).
+
----
[oai wiki home](https://gitlab.eurecom.fr/oai/openairinterface5g/wikis/home)
--
GitLab