DEPLOY_SA5G_HC.md 10.8 KB
Newer Older
arora's avatar
arora committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14
<table style="border-collapse: collapse; border: none;">
  <tr style="border-collapse: collapse; border: none;">
    <td style="border-collapse: collapse; border: none;">
      <a href="http://www.openairinterface.org/">
         <img src="./images/oai_final_logo.png" alt="" border=3 height=50 width=150>
         </img>
      </a>
    </td>
    <td style="border-collapse: collapse; border: none; vertical-align: center;">
      <b><font size = "5">OpenAirInterface 5G Core Network Deployment using Helm Charts</font></b>
    </td>
  </tr>
</table>

arora's avatar
arora committed
15 16 17 18 19 20 21


The cloud native network functions in production will be deployed using a production grade container orchestrator like kubernetes or Openshift. OAI 5g Core network is being designed to keep up with the latest cloud native deployment scenarios. This release of helm charts is focused on deploying each Cloud-native Network Function(CNF) individually.

![Helm Chart Deployment](./images/helm_diag.png)


arora's avatar
arora committed
22 23 24 25 26
**Reading time: ~40mins**

**Tutorial replication time: ~1h30mins**


arora's avatar
arora committed
27 28 29
**TABLE OF CONTENTS**

1.  [Description](#1-description)
arora's avatar
arora committed
30
2.  [Building Images](#2-building-images)
arora's avatar
arora committed
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
3.  [Configuring Helm Charts](#3-configuring-helm-charts)
4.  [Deploying Helm Charts](#4-deploying-helm-charts)


## 1. Description

The helm charts can be used on any production grade kubernetes cluster. Currently they are only tested on our inhouse Openshift cluster the cluster information can be found below.

| Software                 | Version                             |
|:------------------------ |:----------------------------------- |
| Openshift Client Version | 4.4.10                              |
| Kubernetes Version       | Kubernetes Version: v1.17.1+45f8ddb |
| helm                     | v3.5.3                              |

### Pre-requisite

arora's avatar
arora committed
47
The cluster on which these helm charts will be deployed should have RBAC and [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni). Multus is necessary to provide multiple interfaces to AMF and UPF/SPGWU. 
arora's avatar
arora committed
48 49 50

## 2. Building Images

arora's avatar
arora committed
51
The base image used by network function is dependent on the operating system it will be running on. If it is a debian (ubuntu) based cluster then base image will be ubuntu. If it is a RPM (core-os) based cluster then base images will UBI8. Follow the tutorial on [how to build images](./BUILD_IMAGES.md) depending on the cluster/worker-node operating system. In case of Ubuntu based worker nodes, the images can be pulled from [docker-hub](./RETRIEVE_OFFICIAL_IMAGES.md).
arora's avatar
arora committed
52 53 54

## 3. Configuring Helm Charts

arora's avatar
arora committed
55
Clone the helm chart repository from gitlab repository
arora's avatar
arora committed
56 57

```
arora's avatar
arora committed
58
$ git clone -b v1.1.0 https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-fed.git
arora's avatar
arora committed
59 60 61 62 63
$ cd charts
$ ls charts
mysql  oai-amf  oai-nrf  oai-smf  oai-spgwu-tiny
```

arora's avatar
arora committed
64
Helm chart of every network function looks similar and has the below structure. Only the chart of mysql database is different and the NRF helm chart has an extra pvc.yaml to create a presistant volume for storing tcpdump.
arora's avatar
arora committed
65 66 67 68 69

```
Network_function/
├── Chart.yaml
├── templates
arora's avatar
arora committed
70 71 72 73 74 75 76 77
│             ├── configmap.yaml
│             ├── deployment.yaml
│             ├── _helpers.tpl
│             ├── multus.yaml
│             ├── NOTES.txt
│             ├── rbac.yaml
│             ├── serviceaccount.yaml
|             └── service.yaml
arora's avatar
arora committed
78 79 80 81 82
└── values.yaml 

1 directory, 10 files
```

arora's avatar
arora committed
83
All the configurable parameters for a particular commit/release are mentioned in the `values.yaml` file. These parameters will keep on changing in the future depending on the nature of development and features. 
arora's avatar
arora committed
84

arora's avatar
arora committed
85
**NOTE**: If there is a need to edit a specific configuration parameter that is not configurable via the helm-chart values.yaml file then it has to be changed at the time of building images.
arora's avatar
arora committed
86

arora's avatar
arora committed
87
Depending on the namespace where these charts will be instantiated change the value of `namespace` parameter in `values.yaml`. All the network function related configurable parameters are in the sections `config` of the `values.yaml`. To understand the usage and description of each network function configuration parameter refer their [wiki page](https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-amf/-/wikis/home).
arora's avatar
arora committed
88

arora's avatar
arora committed
89
Create a namespace where the helm-charts will be deployed, in our environment we deploy them in `oai` namespace. To create a namespace use the below command on your cluster, 
arora's avatar
arora committed
90

arora's avatar
arora committed
91 92 93 94 95 96
```bash
# needs a user which has the right to create namespaces
$ kubectl create ns oai
or 
$ oc new-project oai
```
arora's avatar
arora committed
97

arora's avatar
arora committed
98

arora's avatar
arora committed
99
### 3.1 Networking related information
arora's avatar
arora committed
100

arora's avatar
arora committed
101
Network functions communicate based on *kubernetes service concept*, the network functions are using FQDN of other network functions to use their service. For example AMF registers with NRF using NRF ip-address. This way we can get rid of any static ip-address configuration. Except for the interfaces which are supposed to be reached from outside the cluster environment. GNB will not be included in the cluster. Thus N1/N2/NGAP and N3/GTP-U interface are provided using multus CNI to AMF and UPF pod. 
arora's avatar
arora committed
102

arora's avatar
arora committed
103
In our environment to reduce complexity and reduce static ip-address allocation we are providing only one interface to each pod (default Kubernetes CNI) except AMF and UPF where we provide two interfaces because gNB or gNB emulator is not running in the cluster environment. For example SMF can have different interfaces/ip-addresses for N4, Nsmf but in our case we are providing only one interface/ip-address to take benifit of *Kubernetes service concept*. But using multus there can be multiple interfaces of SMF too. 
arora's avatar
arora committed
104 105 106

### 3.2 Configuring AMF

arora's avatar
arora committed
107
Open the [values.yaml](../charts/oai-amf/values.yaml) to configure the required parameters. There are many parameters which can be configured. Below are some important parameters,  
arora's avatar
arora committed
108
```
arora's avatar
arora committed
109 110 111 112 113 114 115 116
namespace: "oai" # namespace where AMF will be deployed 

image:
  repository: oai-amf # image name either locally present or in a public/private repository
  version: v1.1.0 # image tag

# configure these values based on gNB/emulator configuration
config: 
arora's avatar
arora committed
117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133
  mcc: "208"
  mnc: "95"
  regionId: "128"
  amfsetId: "1"
  servedGuamiMcc0: "208"
  servedGuamiMnc0: "95"
  servedGuamiRegionId0: "128"
  servedGuamiAmfSetId0: "1"
  servedGuamiMcc1: "460"
  servedGuamiMnc1: "11"
  servedGuamiRegionId1: "10"
  servedGuamiAmfSetId1: "1"
  plmnSupportMcc: "208"
  plmnSupportMnc: "95"
  plmnSupportTac: "0xa000"
  sst0: "222"
  sd0: "123"
arora's avatar
arora committed
134 135
  sst1: "111"
  sd1: "124"
arora's avatar
arora committed
136 137 138 139 140 141 142 143 144 145 146 147
```

The mysql database is pre-configured with some subscriber information. If there is new subscriber information then it should be configured in the mysql database. The subscriber PLMN information should match with gNB and AMF. 

```
  mySqlServer: "mysql"
  mySqlUser: "root"
  mySqlPass: "linux"
  mySqlDb: "oai_db"
  operatorKey: "63bfa50ee6523365ff14c1f45f88737d" (should be the same in mysql)
```

arora's avatar
arora committed
148 149
There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-amf/values.yaml). Infront of every parameter there is a comment. 

arora's avatar
arora committed
150 151
### 3.3 Configuring SMF

arora's avatar
arora committed
152
Open [values.yaml](../charts/oai-smf/values.yaml) to configure SMF configuration. DNS configuration which will be shared with the UE based on the DNS used in your environment, it can be 8.8.8.8 or 4.4.4.4 if you don't know your DNS. 
arora's avatar
arora committed
153 154

```
arora's avatar
arora committed
155 156 157 158 159 160 161
namespace: "oai" # namespace where SMF will be deployed 

image:
  repository: oai-smf # image name either locally present or in a public/private repository
  version: v1.1.0 # image tag

config:
arora's avatar
arora committed
162 163 164 165
  dnsIpv4Address: ""
  dnsSecIpv4Address: ""
```

arora's avatar
arora committed
166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212
Currenly DNN related information and UE network related information can only be changed at the time of building the SMF network function. Refer [Configure the Containers](./CONFIGURE_CONTAINERS.md) to understand how it can be done. 

There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-smf/values.yaml). Infront of every parameter there is a comment. 

### 3.4 Configuring UPF

Open [values.yaml](../charts/oai-spgwu-tiny/values.yaml) to configure UPF/SPGWU configuration

```
namespace: "oai" # namespace where SMF will be deployed 

image:
  repository: oai-spgwu-tiny # image name either locally present or in a public/private repository
  version: v1.1.2 # image tag

config:
  gwId: 1
  mnc: 208 # should match with AMF and SMF and gNB information
  mcc: 95 # should match with AMF and SMF and gNB information
  realm: "3gpp.org" 
  pidDirectory: "/var/run"
```

There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-spgwu-tiny/values.yaml). Infront of every parameter there is a comment. 

### 3.5 Configuring NRF

NRF configuration is straight forward, most of configurable parameters have comment infront of it for explaination. Refer the [values.yaml](../charts/oai-nrf/values.yaml)

### 3.6 Configuring subscriber data in MYSQL

Currently the MYSQL is configured with below subscriber data,

```
IMSI - 208950000000030-34
IMEI - 55000000000001
Secret Key (K) - 0x0C0A34601D4F07677303652C0462535B
OPc - 0x63bfa50ee6523365ff14c1f45f88737d
```

To configure new subscriber add a new entry in [values.yaml](../charts/mysql/values.yaml) line 345, add a new entry after 

```
INSERT INTO `users` VALUES (imsi,'380561234567',imei,NULL,'PURGED',50,40000000,100000000,47,0000000000,1,0xkey,0,0,0x40,'ebd07771ace8677a',0xOpC);
```

In above statement change `imsi`, `imei`, `key` and `opc` with appropriate value according to your environment. 
arora's avatar
arora committed
213 214 215 216 217

## 4. Deploying Helm Charts

Helm charts have an order of deployment for the proper configuration of core network. 

arora's avatar
arora committed
218
`mysql --> nrf --> amf --> smf --> upf(spgwu)`
arora's avatar
arora committed
219 220 221 222

Once the configuration is finished the charts can be deployed with a user who has the rights to

1. Create RBAC
arora's avatar
arora committed
223 224
2. Run pod with privileged and anyuid scc
3. Create multus binds
arora's avatar
arora committed
225 226 227 228 229 230 231 232 233 234

```
$ helm install mysql mysql/
# wait for the pod to come up
$ helm install amf oai-amf/
# wait for the pod to come up
$ helm install nrf oai-nrf/
# wait for the pod to come up
$ helm install smf oai-smf/
# wait for the pod to come up
arora's avatar
arora committed
235
$ helm install upf oai-spgwu-tiny/
arora's avatar
arora committed
236 237 238
# wait for the pod to come up
$ helm list
NAME  NAMESPACE       REVISION  UPDATED                                   STATUS    CHART                 APP VERSION
arora's avatar
arora committed
239 240 241 242 243
mysql oai-5g-develop  1         2021-07-29 14:20:34.010881045 +0200 CEST  deployed  mysql-1.6.9           5.7.30
amf   oai-5g-develop  1         2021-07-29 14:25:34.010881045 +0200 CEST  deployed  oai-amf-1.1.0         1.1.0      
nrf   oai-5g-develop  1         2021-07-29 14:16:11.530949242 +0200 CEST  deployed  oai-nrf-1.1.0         1.1.0      
smf   oai-5g-develop  1         2021-07-29 15:34:15.62785988 +0200 CEST   deployed  oai-smf-1.1.0         1.1.0      
upf   oai-5g-develop  1         2021-07-29 15:28:04.773480519 +0200 CEST  deployed  oai-spgwu-tiny-1.1.2  1.1.2  
Rohan's avatar
Rohan committed
244
```
arora's avatar
arora committed
245 246

Now go ahead and use OAI-gNB/dsTest/gNBSIM or any other gNB or emulator to test the deployed core network.