References
- Sopenode Inria Documentation - SLICES Project
- SD-Fabric documentation
- Repository for omec-upf
- Repository for UP4
- OAI 5G CN Tutorials
Hardware information
Spine switch - Wedge100BF-32QS
Leaf switch - Wedge100BF-32X
OAI CN integration with SD-Fabric
SD-Fabric setup at Eurecom
Step 1: Provision Switch (ONL & P4 Pipeline)
Install ONL on switch
Refer to the following tutorial for installing ONL on an Edgecore Switch
P4 pipeline compile
- Clone oai sdfabric repo
git clone https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-upf-sdfabric.git- Download Intel p4-studio SDE from Intel community portal and copy to docker folder
scp -r bf-sde-9.7.0.tgz leaf2:/root/oai-cn5g-upf-sdfabric/docker- Build p4-studio
docker build -t p4-studio -f Dockerfile .- Compile p4 pipeline with
fabric-upf&fabricprofile
git clone https://github.com/stratum/fabric-tna.git
cd fabric-tna/
export SDE_VERSION=9.7.0
export SDE_DOCKER_IMG=p4-studio
make fabric-upf fabric
make pipeconf
make build PROFILES="fabric-upf fabric"Successful build
*** ONOS pipeconf .oar package created succesfully
/root/fabric-tna/target/fabric-tna-1.2.0-SNAPSHOT.oarStep 2: Configure switch as Kubernetes nodes and join cluster (Optional)
//ToDo
Step 3: Prepare ONOS network configuration (UP4-APP, Fabric-TNA pipeconf app)
- 3.1 Deploy Onos
Deploy Onos docker service with network mode host
docker run --network host --rm --tty --detach --publish 8181:8181 --publish 8101:8101 --publish 5005:5005 --publish 51001:51001 --publish 830:830 --name onos onosproject/onos:2.5.8- 3.2 Activate APPs at Onos
#!/bin/sh
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.drivers.barefoot/active
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.drivers.bmv2/active
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.lldpprovider/active
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.hostprovider/active
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.proxyarp/active
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.segmentrouting/active
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.route-service/active
curl --user onos:rocks -X POST http://127.0.0.1:8181/onos/v1/applications/org.onosproject.fpm/active- 3.3 Install Pipeconf
curl --fail -sSL --user onos:rocks --noproxy localhost -X POST -H Content-Type:application/octet-stream http://127.0.0.1:8181/onos/v1/applications?activate=true --data-binary @./oai-cn5g-upf-sdfabric/apps/fabric-tna-1.2.0-SNAPSHOT.oar- 3.3 Trellis APP
curl --fail -sSL --user onos:rocks --noproxy localhost -X POST -H Content-Type:application/octet-stream http://127.0.0.1:8181/onos/v1/applications?activate=true --data-binary @./oai-cn5g-upf-sdfabric/apps/segmentrouting-app-3.3.0-SNAPSHOT.oar- 3.4 Install UP4 app
curl --fail -sSL --user onos:rocks --noproxy localhost -X POST -H Content-Type:application/octet-stream http://127.0.0.1:8181/onos/v1/applications?activate=true --data-binary @./oai-cn5g-upf-sdfabric/apps/up4-app-1.2.0-SNAPSHOT.oar - 3.5 Enable port forwarding for remote Onos GUI access [OPTIONAL] Here tolix is a jumphost machine for Onos host
ssh -A -i ~/.ssh/id_rsa_silecs -L 8181:172.21.20.110:8181 tolix- 3.6 Access Onos Console
ssh -p 8101 onos@10.0.0.1Step 4: Prepare Stratum chassis configuration for each switch
- 4.1 Run stratum on switch
root@spine:~#
git clone https://github.com/stratum/stratum.git
export SDE_VERSION=9.7.0
export CHASSIS_CONFIG=/root/stratum/chassis_config.pb.txt
~/stratum/stratum/hal/bin/barefoot/docker/start-stratum-container.sh -enable_onlp=false -bf_switchd_background=false -experimental_enable_p4runtime_translation- 4.2 Push tofino config Onos
curl --user onos:rocks -X POST -H Content-Type:application/json -H Accept:application/json http://127.0.0.1:8181/onos/v1/network/configuration/ -d@./oai-cn5g-upf-sdfabric/netcfg/tofino-netcfg.json - 4.3 Push up4 config Onos
curl --user onos:rocks -X POST -H Content-Type:application/json -H Accept:application/json http://127.0.0.1:8181/onos/v1/network/configuration/ -d@./oai-cn5g-upf-sdfabric/netcfg/up4-netcfg.jsonOnos logs
20:10:23.345 INFO [Up4DeviceManager] Setup UPF physical device: device:spine
20:10:23.360 INFO [FabricUpfProgrammable] UpfProgrammable initialized for appId DefaultApplicationId{id=177, name=org.stratumproject.fabric-tna.upf} and deviceId device:spine
20:10:23.361 INFO [Up4DeviceManager] UPF physical device device:spine setup successful!
20:10:23.362 INFO [Up4DeviceManager] Ensuring all interfaces present in app config are present on the leader device.
20:10:23.368 WARN [Up4DeviceManager] UpfInterface(type=ACCESS, prefix=172.21.23.20/32, slice_id=0) is missing from leader device! Installing
20:10:23.378 INFO [FabricUpfProgrammable] Installing UpfInterface(type=ACCESS, prefix=172.21.23.20/32, slice_id=0)
20:10:23.385 INFO [Up4DeviceManager] UPF data plane setup successful!
20:10:23.385 INFO [Up4DeviceManager] Up4Config updated
20:10:23.386 INFO [Up4DeviceManager] Up4Config updated- 4.4 Add routes for N3, N6 and UE
route-add 0.0.0.0/0 10.0.1.2
route-add 12.2.1.0/24 10.0.1.1
route-add 172.21.16.0/22 172.21.23.254Step 5: Deploy OAI-5G-CN
Clone OAI 5G CN federated repo
git clone https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-fed.git -b sdfabricUpdate oai-cn5g-fed/docker-compose/docker-compose-basic-sdfabric-nrf.yaml
Note 1: PFCP Agent is running in network mode host since it will communicate to Onos (UP4 APP) remotely over management network
Note 2: Update following two public ip addresses accordingly
- UPF N3 address in service
sdfabric-upfin docker-compose file
- N3_IPV4_ADDRESS_LOCAL=172.21.23.12- ONOS (UP4) address and UPF N3 address in
upf.json
"access_ip": "172.21.23.12/32",
"p4rtc_server": "172.21.16.102"Deploy OAI 5G CN
[OPTIONAL] Build PFCP agenet with NF registration feature
git clone https://github.com/orion-belt/omec-upf.git
cd omec-upf && DOCKER_TARGETS=pfcpiface make docker-buildOR use pre-built image for PFCP agenet with NF registration feature: rohankharade/upf-epc-pfcpiface:v0.4.0-dev
Deploy OAI 5G CN
cd oai-cn5g-fed/docker-compose
oai-cn5g-fed/docker-compose$ docker-compose -f docker-compose-basic-sdfabric-nrf.yaml up -d
Creating network "demo-oai-public-net" with driver "bridge"
Creating mysql ... done
Creating oai-nrf ... done
Creating oai-ext-dn ... done
Creating oai-udr ... done
Creating oai-udm ... done
Creating oai-ausf ... done
Creating oai-amf ... done
Creating oai-smf ... done
Creating oai-spgwu ... done
Creating sdfabric-upf ... done- Sample output
oai-cn5g-fed/docker-compose$ docker-compose -f docker-compose-basic-sdfabric-nrf.yaml ps -a
Name Command State Ports
-------------------------------------------------------------------------------------------
mysql docker-entrypoint.sh mysqld Up (healthy) 3306/tcp, 33060/tcp
oai-amf python3 /openair-amf/bin/e ... Up (healthy) 38412/sctp, 80/tcp, 9090/tcp
oai-ausf python3 /openair-ausf/bin/ ... Up (healthy) 80/tcp
oai-ext-dn /bin/bash -c ip route add ... Up (healthy)
oai-nrf python3 /openair-nrf/bin/e ... Up (healthy) 80/tcp, 9090/tcp
oai-smf python3 /openair-smf/bin/e ... Up (healthy) 80/tcp, 8805/udp, 9090/tcp
oai-spgwu python3 /openair-spgwu-tin ... Up (healthy) 2152/udp, 8805/udp
oai-udm python3 /openair-udm/bin/e ... Up (healthy) 80/tcp
oai-udr python3 /openair-udr/bin/e ... Up (healthy) 80/tcp
sdfabric-upf /conf/entrypoint.sh UpStep 6.1: Deploy test with OAI-RFSIM
- Pull docker images
docker pull oaisoftwarealliance/oai-gnb:develop
docker pull oaisoftwarealliance/oai-nr-ue:develop- Add route if needed for AMF [Optional]
ip route add <AMF_Subnet> via <GW_IP_for_AMF> dev <Interface>
e.g.
ip route add 192.168.70.0/24 via 172.21.20.110 dev eno1- Run Gnb [Update
GNB_NGA_IF_NAME(AMF) andGNB_NGU_IF_NAME(UPF) accordingly]
docker-compose -f docker-compose-rfsim.yaml up -d oai-gnb- Run UE
docker-compose -f docker-compose-rfsim.yaml up -d oai-nr-ue1- Sample wireshark packet captures Pcap for test with OAI RFSIM
Step 6.2: Deploy test with OAI-GNB and COTS UE
6.2.1 Build OAI gNB
# Get openairinterface5g source code
git clone https://gitlab.eurecom.fr/oai/openairinterface5g.git ~/openairinterface5g -b develop
cd ~/openairinterface5g
source oaienv
cd cmake_targets
# Install OAI dependencies
./build_oai -I
# Build OAI gNB
./build_oai -w USRP --ninja --gNB --build-lib all -c6.2.2 Run OAI gNB
cd ~/openairinterface5g/cmake_targets/ran_build/build
sudo ./nr-softmodem -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/gnb.sa.band78.fr1.106PRB.usrpb210.conf --sa -E --continuous-txMake sure NGSetup procedure is successful before testing with UE.
6.2.3 Test with COTS UE
We tested the setup using Quectel RM500Q-GL modem. For more details regarding configuring SIM card and testing UE with core network, please check RAN tutorial for COTS UE. Also here are some useful AT Commands that can be used to configure Quectel module.
- Sample wireshark packet captures
Pcap for NAS/NGAP (N1N2)
Pcap for PFCP (N4)
Pcap for GTPU (N3)