Performance Issues in MIMO Implementation with X410 USRPs
We are attempting to create a 2x2 MIMO system with two X410 USRPs (FPGA Image: X4_200). Initially, we want to observe throughput metrics for SISO, SIMO, MISO and MIMO systems. These X410s are connected to two separate computers using 100Gbps (However, due to the FPGA image, it operates at 10Gbps.) Ethernet cables. The specifications of the computers are as follows:
gNB Computer:
- Memory: 62.6 GiB
- Processor: 12th Intel Core i7-12700KFx20
- OS Name: Ubuntu 20.04.6 LTS
- OS Type: 64-bit
UE Computer:
- Memory: 62.5 GiB
- Processor: 13th Intel Core i9-13900 Kx32
- OS Name: Ubuntu 20.04.5 LTS
- OS Type: 64-bit
The config file we use is below:
gnb.sa.band78.fr1.106PRB.2x2.usrpn300.conf
We run the following command for gNodeB and UE:
- gNodeB Command
sudo ./nr-softmodem -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/gnb.sa.band78.fr1.106PRB. 2x2.usrpn300.conf --sa --usrp-tx-thread-config 1 --thread-pool 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18 -A 45
- UE Command
sudo ./nr-uesoftmodem --usrp-args "addr=192.168.10.2, clock_source=internal,time_source=internal" -r 106 --numerology 1 --band 78 -C 3319680000 --sa --ue-rxgain 30 --ue-txgain 30 --ue-fo-compensation --uicc0.imsi 001010000000001 -A 45 --ue-nb-ant-rx 2 --ue-nb-ant-tx 2 --thread-pool 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31 -d
Initially, we were getting Late Packet Error (L) as seen in the image below.
To solve this error, we wrote the commands on these pages "https://kb.ettus.com/Getting_Started_with_4G_LTE_using_Eurecom_OpenAirInterface_(OAI)_on_the_USRP_2974" and "https://files.ettus.com/manual/page_usrp_x3x0_config.html" and the Late Packet Error was fixed.
We made the performance settings on the site "https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/doc/NR_SA_Tutorial_COTS_UE.md?ref_type=heads" as follows:
for ((i=0;i<$(nproc);i++)); do sudo cpufreq-set -c $i -r -g performance; done
sudo sysctl -w net.core.wmem_max=62500000
sudo sysctl -w net.core.rmem_max=62500000
sudo sysctl -w net.core.wmem_default=62500000
sudo sysctl -w net.core.rmem_default=62500000
sudo ethtool -G ens3f1np1 tx 8192 rx 8192
Also we increase the system stack size limit: sudo sh -c "ulimit -s 8192"
And we set set the MTU to 9000: sudo ifconfig <interface> mtu 9000 # For 10 GigE
Later, we wanted to use the iperf commands given on the "https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/doc/NR_SA_Tutorial_COTS_UE.md?ref_type=heads" site to examine throughput metrics, but we encountered some problems. Although in iperf we can achieve high data sending sizes (1Gbps - 2Gbps) for SISO, in other systems the PDSCH disconnects at much smaller sizes (10Mbps - 20Mbps) and the PDSCH constellation diagram on the nrscope screen remains constant. (PBCH and PDCCH diagrams continue to work.) As far as we understand (it may be a wrong inference) only the PDSCH connection is lost. Other PBCH and PDCCH channels continue to communicate. Below are the commands and terminal output for iperf:
iperf command:
- CN5G host:
docker exec -it oai-ext-dn ping 12.1.1.2
- UE host:
ping 192.168.70.135 -t -S 12.1.1.2
The terminal output for the UE is as follows:
[HW] allocating graph of 19008 points
[HW] allocating graph of 5544 points
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 42 dB, CQI = 15
[HW] allocating graph of 28512 points
[NR_PHY] ============================================
[NR_PHY] Harq round stats for Downlink: 1635/0/0
[NR_PHY] ============================================
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 43 dB, CQI = 15
[HW] allocating graph of 26784 points
[HW] allocating graph of 5208 points
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 43 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 0, SINR = 40 dB, CQI = 15
[HW] allocating graph of 48384 points
[HW] allocating graph of 12600 points
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 1, SINR = 41 dB, CQI = 15
[HW] allocating graph of 183168 points
[HW] allocating graph of 17808 points
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 1, SINR = 39 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 1, SINR = 40 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 0, SINR = 40 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 42 dB, CQI = 15
[NR_PHY] ============================================
[NR_PHY] Harq round stats for Downlink: 1968/126/120/108
[NR_PHY] ============================================
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 42 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 43 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 41 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 42 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 42 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 40 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 42 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 1 i1 = 0.0.0, i2 = 0, SINR = 42 dB, CQI = 15
[NR_PHY] ============================================
[NR_PHY] Harq round stats for Downlink: 1968/126/120/108
[NR_PHY] ============================================
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 0, SINR = 41 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 0, SINR = 38 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 1, SINR = 40 dB, CQI = 15
[NR_PHY] RSRP = -64 dBm, RI = 2 i1 = 0.0.0, i2 = 1, SINR = 40 dB, CQI = 15
[PHY] [UE 0] frame 542, nr_slot_rx 0, Error decoding PBCH!
[PHY] [UE 0] frame 544, nr_slot_rx 0, Error decoding PBCH!
[NR_PHY] RSRP = -109 dBm, RI = 2 i1 = 0.0.0, i2 = 0, SINR = 90 dB, CQI = 15
[PHY] [UE 0] frame 546, nr_slot_rx 0, Error decoding PBCH!
[PHY] [UE 0] frame 548, nr_slot_rx 0, Error decoding PBCH!
[PHY] [UE 0] frame 550, nr_slot_rx 0, Error decoding PBCH!
[PHY] [UE 0] frame 552, nr_slot_rx 0, Error decoding PBCH!
[NR_PHY] RSRP = -109 dBm, RI = 2 i1 = 0.0.0, i2 = 1, SINR = 90 dB, CQI = 15
[PHY] [UE 0] frame 554, nr_slot_rx 0, Error decoding PBCH!
[PHY] [UE 0] frame 556, nr_slot_rx 0, Error decoding PBCH!
[PHY] [UE 0] frame 558, nr_slot_rx 0, Error decoding PBCH!
[PHY] [UE 0] frame 560, nr_slot_rx 0, Error decoding PBCH!
For gNB, the terminal output is as follows:
NR_MAC] Frame.Slot 0.0
UE RNTI dc76 CU-UE-ID 1 in-sync PH 0 dB PCMAX 0 dBm, average RSRP -64 (16 meas)
UE dc76: CQI 15, RI 1, PMI (0,3)
UE dc76: UL-RI 1, TPMI 0
UE dc76: dlsch_rounds 14400/1693/4/0, dlsch_errors 0, pucch0_DTX 0, BLER 0.00038 MCS (0) 9
UE dc76: ulsch_rounds 11312/0/0/0, ulsch_errors 0, ulsch_DTX 0, BLER 0.00000 MCS (0) 9
UE dc76: MAC: TX 36581747 RX 964932 bytes
UE dc76: LCID 1: TX 1046 RX 282 bytes
UE dc76: LCID 2: TX 0 RX 0 bytes
UE dc76: LCID 4: TX 35565217 RX 4102 bytes
[NR_MAC] Frame.Slot 128.0
UE RNTI dc76 CU-UE-ID 1 in-sync PH 0 dB PCMAX 0 dBm, average RSRP -64 (16 meas)
UE dc76: CQI 15, RI 1, PMI (0,3)
UE dc76: UL-RI 1, TPMI 0
UE dc76: dlsch_rounds 14413/1693/4/0, dlsch_errors 0, pucch0_DTX 0, BLER 0.00010 MCS (0) 9
UE dc76: ulsch_rounds 11696/0/0/0, ulsch_errors 0, ulsch_DTX 0, BLER 0.00000 MCS (0) 9
UE dc76: MAC: TX 36583310 RX 997700 bytes
UE dc76: LCID 1: TX 1046 RX 282 bytes
UE dc76: LCID 2: TX 0 RX 0 bytes
UE dc76: LCID 4: TX 35565217 RX 4102 bytes
[NR_MAC] Frame.Slot 256.0
UE RNTI dc76 CU-UE-ID 1 in-sync PH 0 dB PCMAX 0 dBm, average RSRP -64 (16 meas)
UE dc76: CQI 15, RI 2, PMI (0,0)
UE dc76: UL-RI 1, TPMI 0
UE dc76: dlsch_rounds 15653/1717/28/16, dlsch_errors 10, pucch0_DTX 0, BLER 0.00003 MCS (0) 19
UE dc76: ulsch_rounds 12089/0/0/0, ulsch_errors 0, ulsch_DTX 0, BLER 0.00000 MCS (0) 9
UE dc76: MAC: TX 40984429 RX 1030684 bytes
UE dc76: LCID 1: TX 1046 RX 282 bytes
UE dc76: LCID 2: TX 0 RX 0 bytes
UE dc76: LCID 4: TX 39953632 RX 4327 bytes
[RLC] ../../../openair2/LAYER2/nr_rlc/nr_rlc_entity_am.c:1769:nr_rlc_entity_am_recv_sdu: warning: 1 SDU rejected, SDU buffer full
[NR_MAC] Frame.Slot 384.0
UE RNTI dc76 CU-UE-ID 1 in-sync PH 0 dB PCMAX 0 dBm, average RSRP -64 (16 meas)
UE dc76: CQI 15, RI 2, PMI (0,0)
UE dc76: UL-RI 1, TPMI 0
UE dc76: dlsch_rounds 15776/1837/148/144, dlsch_errors 129, pucch0_DTX 0, BLER 0.38916 MCS (0) 17
UE dc76: ulsch_rounds 12485/0/0/0, ulsch_errors 0, ulsch_DTX 0, BLER 0.00000 MCS (0) 9
UE dc76: MAC: TX 42288792 RX 1063740 bytes
UE dc76: LCID 1: TX 1046 RX 282 bytes
UE dc76: LCID 2: TX 0 RX 0 bytes
UE dc76: LCID 4: TX 41254531 RX 5048 bytes
As can be seen, on the UE side:
- [PHY] [UE 0] frame 542, nr_slot_rx 0, Error decoding PBCH!
On the gNB side:
- [RLC] ../../../openair2/LAYER2/nr_rlc/nr_rlc_entity_am.c:1769:nr_rlc_entity_am_recv_sdu: warning: 1 SDU rejected, SDU buffer full
We get two errors like this. While performing tests on the SISO system, while iperf works without any problems, why does the connection fail in other systems? What could it be caused by? How we can solve this problem?
Thank you in advance for your interest.
Kind regards,