UPF Refactoring & Multi-CN Interoperability

Franck MESSAOUDIrequested to merge
feature-refactoring-split-into-components into develop

Overview

This merge request introduces a major refactoring of the OAI UPF codebase, splitting monolithic components into modular, maintainable units while ensuring full N4 interoperability with multiple 5G Core Networks (Open5GS v2.7.6, OAI-CN5G, free5gc).

1. Architecture Refactoring

1.1 Component Separation

The monolithic UPF application has been split into dedicated, single-responsibility components:

Component File(s) Responsibility
UserPlaneComponent UserPlaneComponent.cpp/h Main orchestrator for UPF control plane, manages BPF/XDP programs, PFCP sessions, and network interfaces (3GPP TS 23.501)
SessionManager SessionManager.cpp/h Complete CRUD operations for PFCP sessions, PDRs, FARs, QERs (3GPP TS 29.244)
SessionProgramManager SessionProgramManager.cpp/h BPF program management and PFCP IE to BPF structure conversion
SessionPrograms SessionPrograms.cpp/h Session-specific BPF program state management
BPFProgram BPFProgram.cpp/h Base class for BPF program lifecycle management
SignalHandler SignalHandler.cpp/h Graceful shutdown and signal handling
Configuration Configuration.cpp/h Configuration management and validation

1.2 Design Patterns Implemented

  • Singleton Pattern: UserPlaneComponent::GetInstance(), SessionProgramManager::GetInstance()
  • Observer Pattern: ISessionObserver interface for session event notifications
  • Factory Pattern: BPF program creation via SessionProgramManager
  • RAII: Proper resource management for BPF maps and programs

2. N4 Interface (PFCP) Improvements

2.1 Complete CRUD Operations for PFCP Rules

Implemented full Create/Read/Update/Delete operations per 3GPP TS 29.244:

// Session-level operations


SessionOperationResult CreateSession(std::shared_ptr<pfcp::pfcp_session>);
SessionOperationResult UpdateSession(std::shared_ptr<pfcp::pfcp_session>);
SessionOperationResult DeleteSession(uint64_t seid);

// PDR operations (Section 5.2.1)


bool AddPdr(uint64_t seid, std::shared_ptr<pfcp::pfcp_pdr>);
bool UpdatePdr(uint64_t seid, std::shared_ptr<pfcp::pfcp_pdr>);
bool RemovePdr(uint64_t seid, uint16_t pdr_id);

// FAR operations (Section 5.2.2)


bool AddFar(uint64_t seid, std::shared_ptr<pfcp::pfcp_far>);
bool UpdateFar(uint64_t seid, std::shared_ptr<pfcp::pfcp_far>);
bool RemoveFar(uint64_t seid, uint32_t far_id);

// QER operations (Section 5.2.3)


bool AddQer(uint64_t seid, std::shared_ptr<pfcp::pfcp_qer>);
bool UpdateQer(uint64_t seid, std::shared_ptr<pfcp::pfcp_qer>);
bool RemoveQer(uint64_t seid, uint32_t qer_id);

2.2 N4 Message Handlers

Full implementation of PFCP message handling per 3GPP TS 29.244 Section 6:

  • Session Establishment (Section 7.5.2): EstablishSession()
  • Session Modification (Section 7.5.4): ModifySession() with incremental update support
  • Session Deletion (Section 7.5.5): RemoveSession()

2.3 Modification Request Handlers

Granular handling of PFCP Session Modification requests:

size_t HandlePdrUpdates(session, mod_req);  // Section 8.2.9
size_t HandleFarUpdates(session, mod_req);  // Section 8.2.10
size_t HandleQerUpdates(session, mod_req);  // Section 8.2.11
size_t HandlePdrRemoval(session, mod_req);  // Section 8.2.16
size_t HandleFarRemoval(session, mod_req);  // Section 8.2.17
size_t HandleQerRemoval(session, mod_req);  // Section 8.2.18

3. Multi-CN Interoperability

3.1 Open5GS v2.7.6 Compatibility Fix

Problem: Open5GS (3GPP compliant) does not include UE IP address in uplink PDRs. Our BPF code incorrectly required UE IP for all PDR matching.

Root Cause in upf_xdp_kern.c:

// OLD (broken)


if (ipaddr != pkt_ue_ip) {
  continue;  // Always fails for Open5GS uplink PDRs (ipaddr=0)
}

Fix: Made UE IP check optional per 3GPP TS 29.244:

// NEW (3GPP compliant)


if ((ipaddr != 0) && (ipaddr != pkt_ue_ip)) {
  continue;  // Only check UE IP if present in PDR
}

3.2 TEID Extraction from Packet

Problem: The BPF code was using session->teid_ul (from map) instead of the actual packet's TEID for PDR matching.

Fix in lookup_session_n3():

// Added new output parameter to extract packet TEID


static __always_inline struct session_id* lookup_session_n3(
    void* data, void* data_end, struct ethhdr* eth, u32* ue_ip_out,
    u8* qfi_out, u32* pkt_teid_out) {  // NEW PARAMETER
  
  // Extract TEID from incoming GTP-U packet
  *pkt_teid_out = gtpu->teid;
}

Fix in xdp_uplink():

u32 pkt_teid = 0;
struct session_id* session =
    lookup_session_n3(data, data_end, eth, &ue_ip, &qfi, &pkt_teid);

// Use packet TEID, NOT session->teid_ul


struct pfcp_pdr* pdr = match_pdr_n3(seid, pkt_teid, ue_ip, qfi);

3.3 Compatibility Matrix

Core Network Version UE IP in UL PDR Status
Open5GS v2.7.6 No (3GPP compliant) ✅ Fixed
OAI-CN5G Latest Yes ✅ Works
free5gc Latest Variable ✅ Works

4. XDP/eBPF Datapath Improvements

4.1 PDR Matching Logic

Improved PDR matching per 3GPP TS 29.244 Section 5.2.1:

Uplink PDR Matching (match_pdr_n3):

  1. Primary criterion: F-TEID match (mandatory)
  2. Optional: UE IP match (only if present in PDR)
  3. Optional: QFI match
  4. Selection: Lowest precedence value wins

Downlink PDR Matching (match_pdr_n6):

  1. Primary criterion: UE IP match (mandatory)
  2. Optional: SDF filter match with specificity scoring
  3. Selection: Highest specificity, then lowest precedence

4.2 SDF Filter Specificity Scoring

New algorithm for selecting most specific SDF filter:

static __always_inline u32 calc_sdf_specificity(
    const struct sdf_filtr* sdf, u8 pkt_proto) {
  u32 score = 0;
  
  // Protocol specificity (most important)
  if (sdf->protocol == 0) score += 100;       // Any protocol
  else if (sdf->protocol == pkt_proto) score += 300;  // Exact match
  
  // IP address specificity (count mask bits)
  score += __builtin_popcount(src_mask);
  score += __builtin_popcount(dst_mask);
  
  // Port range specificity (reward narrow ranges)
  score += (65535 - src_port_range) / 1000;
  score += (65535 - dst_port_range) / 1000;
  
  return score;
}

4.3 GTP-U Processing

  • Encapsulation (gtpu_encap_ipv4): Downlink packets N6→N3
  • Decapsulation (gtpu_decap_ipv4): Uplink packets N3→N6
  • QFI Handling: PDU Session Container extension support

5. TC (Traffic Control) QoS Layer

5.1 QER Program Implementation

New TC-BPF program for QoS enforcement per 3GPP TS 23.501:

class QERProgram : public BPFProgram {
public:
  void Setup(uint64_t seid, 
             std::vector<std::shared_ptr<pfcp::pfcp_qer>>& qers,
             std::vector<std::shared_ptr<pfcp::pfcp_pdr>>& pdrs);
  void TearDown();
  
private:
  void ConfigureQerMaps(struct qer_tc_kern_c* skel, const upf_config& cfg);
  void SetupHTBQdisc(const std::string& interface);
  void AddQosClass(uint32_t class_id, uint64_t rate, uint64_t ceil);
};

5.2 QoS Features

  • Gate Status: UL/DL gate control per QER
  • MBR (Maximum Bitrate): Rate limiting per QoS flow
  • GBR (Guaranteed Bitrate): Minimum bandwidth guarantee
  • QFI Mapping: Per-flow QoS classification

6. BPF Maps Configuration

6.1 Dynamic Map Resizing

Implemented runtime map size configuration based on deployment scale:

void ConfigureXdpMaps(struct upf_xdp_kern_c* skel, const upf_config& cfg) {
  // Session maps
  ConfigureMapMaxEntries(skel->maps.session_by_ue_ip_map, 
                         cfg.max_pdu_sessions);
  ConfigureMapMaxEntries(skel->maps.pdrs_per_session_map,
                         cfg.max_pdu_sessions);
  
  // Interface maps  
  ConfigureMapMaxEntries(skel->maps.redirect_interfaces_map,
                         cfg.max_upf_redirect_interfaces);
  
  // ARP table
  ConfigureMapMaxEntries(skel->maps.arp_table_map,
                         cfg.max_arp_entries);
}

6.2 Map Validation

Added pre-flight validation for map configurations:

if (cfg.max_upf_redirect_interfaces > cfg.max_upf_interfaces) {
  throw std::runtime_error(
      "Invalid config: redirect interfaces > total interfaces");
}

int num_ifaces = CountAvailableInterfaces();
if (cfg.max_upf_interfaces > num_ifaces) {
  Logger::upf_app().warn("Clamping max_upf_interfaces to %d", num_ifaces);
}

7. PFCP IE to BPF Conversion

7.1 Conversion Functions

Complete conversion of PFCP IEs to BPF-compatible structures:

// 3GPP TS 29.244 Section 8.2.2 - PDR


struct pfcp_pdr ConvertPdr(std::shared_ptr<pfcp::pfcp_pdr>) const;

// 3GPP TS 29.244 Section 8.2.3 - FAR


struct pfcp_far ConvertFar(std::shared_ptr<pfcp::pfcp_far>) const;

// 3GPP TS 29.244 Section 8.2.4 - QER


struct pfcp_qer ConvertQer(std::shared_ptr<pfcp::pfcp_qer>) const;

7.2 TEID Extraction Helpers

New helper functions for proper TEID handling:

// Extract uplink TEID from PDR's local F-TEID (UPF listens on N3)


static uint32_t GetUplinkTeidFromPdr(std::shared_ptr<pfcp::pfcp_pdr>);

// Extract downlink TEID from FAR's outer header creation (send to gNB)


static uint32_t GetDownlinkTeidFromFar(std::shared_ptr<pfcp::pfcp_far>);

8. Logging Improvements

8.1 Structured Logging Format

Consistent log format with context prefixes:

[N4] Create Session: seid 0x1 - Validating session context
[N4] Create Session: seid 0x1 - Creating eBPF data-path pipeline
  → Uplink PDRs (2 rules):
    • PDR 2: Local F-TEID 0x1 (UPF listens on N3)
    • PDR 4: Local F-TEID 0x3 (UPF listens on N3)
  → Downlink PDRs (1 rule):
    • PDR 1 → FAR 1: Remote F-TEID 0x1 (send to gNB on N3)
[eBPF] Create Pipeline - Pipeline created for session 0x1 with 3 PDRs

8.2 Log Format Macros 

#define SEID_FMT "0x%lx"
#define TEID_FMT "0x%x"

8.3 ARP Logging 

ARP: 192.168.70.134 dev n3 lladdr 02:42:c0:a8:46:86 [SEID=0x1]

9. Configuration Improvements

9.1 New Configuration Parameters 

upf:
  max_pdu_sessions: 1024
  max_pdrs_per_session: 16
  max_upf_interfaces: 8
  max_upf_redirect_interfaces: 4
  max_arp_entries: 256
  enable_bpf_datapath: true
  enable_qos: true

9.2 Startup Banner

New visual startup display showing:

  • UPF configuration summary
  • Network interface details
  • Data plane status (XDP mode, QoS)
  • BPF map statistics

10. Bug Fixes

10.1 Uplink PDR Matching Fix

  • Issue: Uplink traffic not redirected with Open5GS
  • Cause: BPF code required UE IP in uplink PDRs (not 3GPP compliant)
  • Fix: Made UE IP check optional, use packet TEID for matching

10.2 TEID Handling Fix

  • Issue: Wrong TEID used for PDR matching
  • Cause: Using session->teid_ul from map instead of packet TEID
  • Fix: Extract TEID from GTP-U header in lookup_session_n3()

10.3 Multiple TEID Support

  • Issue: Only primary TEID stored in PDU session map
  • Cause: Map structure only supported one TEID per direction
  • Fix: Added warning logging, PDR array now stores all TEIDs

10.4 Memory Safety

  • Added null pointer checks throughout
  • RAII pattern for BPF resource management
  • Exception handling with proper cleanup

11. 3GPP Compliance

References

  • TS 23.501: 5G System Architecture
  • TS 29.244: PFCP Protocol (N4 Interface)
  • TS 29.281: GTP-U Protocol
  • TS 23.502: 5G Procedures

Key Compliance Points

  1. UE IP is optional in uplink PDRs (TS 29.244 Section 8.2.62)
  2. F-TEID is primary match criterion for uplink (TS 29.244 Section 5.2.1)
  3. Precedence-based PDR selection (TS 29.244 Section 8.2.29)
  4. QFI handling in QER (TS 29.244 Section 8.2.89)

12. Testing

Validated Configurations

  • OAI-CN5G (Docker deployment)
  • Open5GS v2.7.6 (Docker deployment)
  • UERANSIM (UE/RAN simulator)
  • oai-ext-dn (External data network)

Test Scenarios

  1. PDU Session Establishment
  2. Uplink traffic (UE → DN)
  3. Downlink traffic (DN → UE)
  4. Session Modification (add/update/remove PDRs)
  5. Session Deletion
  6. Multiple QoS flows

13. Files Changed

New Files

  • SessionManager.cpp/h
  • SessionProgramManager.cpp/h
  • SessionPrograms.cpp/h
  • UserPlaneComponent.cpp/h
  • BPFProgram.cpp/h
  • SignalHandler.cpp/h
  • Configuration.cpp/h

Modified Files

  • upf_xdp_kern.c (PDR matching fix)
  • upf_xdp_user.cpp/h (Map configuration)
  • qer_tc_user.cpp/h (QoS implementation)
  • pfcp_session.cpp/hpp
  • upf_config.hpp

14. Breaking Changes

None. Full backward compatibility maintained with existing deployments.

15. Future Work

  • IPv6 support in PDR matching
  • URR (Usage Reporting Rules) implementation
  • BAR (Buffering Action Rules) support
  • Multiple PDN connections per UE
  • N9 interface support (UPF-to-UPF)

Merge request reports