DEPLOY_SA5G_HC.md 11.3 KB
Newer Older
arora's avatar
arora committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14
<table style="border-collapse: collapse; border: none;">
  <tr style="border-collapse: collapse; border: none;">
    <td style="border-collapse: collapse; border: none;">
      <a href="http://www.openairinterface.org/">
         <img src="./images/oai_final_logo.png" alt="" border=3 height=50 width=150>
         </img>
      </a>
    </td>
    <td style="border-collapse: collapse; border: none; vertical-align: center;">
      <b><font size = "5">OpenAirInterface 5G Core Network Deployment using Helm Charts</font></b>
    </td>
  </tr>
</table>

arora's avatar
arora committed
15 16 17 18 19 20 21


The cloud native network functions in production will be deployed using a production grade container orchestrator like kubernetes or Openshift. OAI 5g Core network is being designed to keep up with the latest cloud native deployment scenarios. This release of helm charts is focused on deploying each Cloud-native Network Function(CNF) individually.

![Helm Chart Deployment](./images/helm_diag.png)


arora's avatar
arora committed
22 23 24 25 26
**Reading time: ~40mins**

**Tutorial replication time: ~1h30mins**


arora's avatar
arora committed
27 28 29
**TABLE OF CONTENTS**

1.  [Description](#1-description)
arora's avatar
arora committed
30
2.  [Building Images](#2-building-images)
arora's avatar
arora committed
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
3.  [Configuring Helm Charts](#3-configuring-helm-charts)
4.  [Deploying Helm Charts](#4-deploying-helm-charts)


## 1. Description

The helm charts can be used on any production grade kubernetes cluster. Currently they are only tested on our inhouse Openshift cluster the cluster information can be found below.

| Software                 | Version                             |
|:------------------------ |:----------------------------------- |
| Openshift Client Version | 4.4.10                              |
| Kubernetes Version       | Kubernetes Version: v1.17.1+45f8ddb |
| helm                     | v3.5.3                              |

### Pre-requisite

arora's avatar
arora committed
47
The cluster on which these helm charts will be deployed should have RBAC and [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni). Multus is necessary to provide multiple interfaces to AMF and UPF/SPGWU. 
arora's avatar
arora committed
48 49 50

## 2. Building Images

arora's avatar
arora committed
51
The base image used by network function is dependent on the operating system it will be running on. If it is a debian (ubuntu) based cluster then base image will be ubuntu. If it is a RPM (core-os) based cluster then base images will UBI8. Follow the tutorial on [how to build images](./BUILD_IMAGES.md) depending on the cluster/worker-node operating system. In case of Ubuntu based worker nodes, the images can be pulled from [docker-hub](./RETRIEVE_OFFICIAL_IMAGES.md).
arora's avatar
arora committed
52 53 54

## 3. Configuring Helm Charts

arora's avatar
arora committed
55
Clone the helm chart repository from gitlab repository
arora's avatar
arora committed
56 57

```
arora's avatar
arora committed
58
$ git clone -b v1.1.0 https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-fed.git
arora's avatar
arora committed
59 60 61 62 63
$ cd charts
$ ls charts
mysql  oai-amf  oai-nrf  oai-smf  oai-spgwu-tiny
```

arora's avatar
arora committed
64
Helm chart of every network function looks similar and has the below structure. Only the chart of mysql database is different and the NRF helm chart has an extra pvc.yaml to create a presistant volume for storing tcpdump.
arora's avatar
arora committed
65 66 67 68 69

```
Network_function/
├── Chart.yaml
├── templates
arora's avatar
arora committed
70 71 72 73 74 75 76 77
│             ├── configmap.yaml
│             ├── deployment.yaml
│             ├── _helpers.tpl
│             ├── multus.yaml
│             ├── NOTES.txt
│             ├── rbac.yaml
│             ├── serviceaccount.yaml
|             └── service.yaml
arora's avatar
arora committed
78 79 80 81 82
└── values.yaml 

1 directory, 10 files
```

arora's avatar
arora committed
83
All the configurable parameters for a particular commit/release are mentioned in the `values.yaml` file. These parameters will keep on changing in the future depending on the nature of development and features. 
arora's avatar
arora committed
84

arora's avatar
arora committed
85
**NOTE**: If there is a need to edit a specific configuration parameter that is not configurable via the helm-chart values.yaml file then it has to be changed at the time of building images.
arora's avatar
arora committed
86

arora's avatar
arora committed
87
Depending on the namespace where these charts will be instantiated change the value of `namespace` parameter in `values.yaml`. All the network function related configurable parameters are in the sections `config` of the `values.yaml`. To understand the usage and description of each network function configuration parameter refer their [wiki page](https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-amf/-/wikis/home).
arora's avatar
arora committed
88

arora's avatar
arora committed
89
Create a namespace where the helm-charts will be deployed, in our environment we deploy them in `oai` namespace. To create a namespace use the below command on your cluster, 
arora's avatar
arora committed
90

arora's avatar
arora committed
91 92 93 94 95 96
```bash
# needs a user which has the right to create namespaces
$ kubectl create ns oai
or 
$ oc new-project oai
```
arora's avatar
arora committed
97

arora's avatar
arora committed
98

arora's avatar
arora committed
99
### 3.1 Networking related information
arora's avatar
arora committed
100

arora's avatar
arora committed
101
Network functions communicate based on *kubernetes service concept*, the network functions are using FQDN of other network functions to use their service. For example AMF registers with NRF using NRF ip-address. This way we can get rid of any static ip-address configuration. Except for the interfaces which are supposed to be reached from outside the cluster environment. GNB will not be included in the cluster. Thus N1/N2/NGAP and N3/GTP-U interface are provided using multus CNI to AMF and UPF pod. 
arora's avatar
arora committed
102

arora's avatar
arora committed
103
In our environment to reduce complexity and reduce static ip-address allocation we are providing only one interface to each pod (default Kubernetes CNI) except AMF and UPF where we provide two interfaces because gNB or gNB emulator is not running in the cluster environment. For example SMF can have different interfaces/ip-addresses for N4, Nsmf but in our case we are providing only one interface/ip-address to take benifit of *Kubernetes service concept*. But using multus there can be multiple interfaces of SMF too. 
arora's avatar
arora committed
104 105 106

### 3.2 Configuring AMF

arora's avatar
arora committed
107
Open the [values.yaml](../charts/oai-amf/values.yaml) to configure the required parameters. There are many parameters which can be configured. Below are some important parameters,  
arora's avatar
arora committed
108
```
arora's avatar
arora committed
109 110 111 112 113 114 115 116
namespace: "oai" # namespace where AMF will be deployed 

image:
  repository: oai-amf # image name either locally present or in a public/private repository
  version: v1.1.0 # image tag

# configure these values based on gNB/emulator configuration
config: 
arora's avatar
arora committed
117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133
  mcc: "208"
  mnc: "95"
  regionId: "128"
  amfsetId: "1"
  servedGuamiMcc0: "208"
  servedGuamiMnc0: "95"
  servedGuamiRegionId0: "128"
  servedGuamiAmfSetId0: "1"
  servedGuamiMcc1: "460"
  servedGuamiMnc1: "11"
  servedGuamiRegionId1: "10"
  servedGuamiAmfSetId1: "1"
  plmnSupportMcc: "208"
  plmnSupportMnc: "95"
  plmnSupportTac: "0xa000"
  sst0: "222"
  sd0: "123"
arora's avatar
arora committed
134 135
  sst1: "111"
  sd1: "124"
arora's avatar
arora committed
136 137 138 139 140 141 142 143 144 145 146 147
```

The mysql database is pre-configured with some subscriber information. If there is new subscriber information then it should be configured in the mysql database. The subscriber PLMN information should match with gNB and AMF. 

```
  mySqlServer: "mysql"
  mySqlUser: "root"
  mySqlPass: "linux"
  mySqlDb: "oai_db"
  operatorKey: "63bfa50ee6523365ff14c1f45f88737d" (should be the same in mysql)
```

arora's avatar
arora committed
148 149 150

Configure `readinessProbe` and `livenessProbe` by default they are `true` to switch them off change the value with `false`

arora's avatar
arora committed
151 152
There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-amf/values.yaml). Infront of every parameter there is a comment. 

arora's avatar
arora committed
153 154
### 3.3 Configuring SMF

arora's avatar
arora committed
155
Open [values.yaml](../charts/oai-smf/values.yaml) to configure SMF configuration. DNS configuration which will be shared with the UE based on the DNS used in your environment, it can be 8.8.8.8 or 4.4.4.4 if you don't know your DNS. 
arora's avatar
arora committed
156 157

```
arora's avatar
arora committed
158 159 160 161 162 163 164
namespace: "oai" # namespace where SMF will be deployed 

image:
  repository: oai-smf # image name either locally present or in a public/private repository
  version: v1.1.0 # image tag

config:
arora's avatar
arora committed
165 166 167 168
  dnsIpv4Address: ""
  dnsSecIpv4Address: ""
```

arora's avatar
arora committed
169 170
Currenly DNN related information and UE network related information can only be changed at the time of building the SMF network function. Refer [Configure the Containers](./CONFIGURE_CONTAINERS.md) to understand how it can be done. 

arora's avatar
arora committed
171 172
Configure `readinessProbe` and `livenessProbe` by default they are `true` to switch them off change the value with `false`

arora's avatar
arora committed
173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193
There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-smf/values.yaml). Infront of every parameter there is a comment. 

### 3.4 Configuring UPF

Open [values.yaml](../charts/oai-spgwu-tiny/values.yaml) to configure UPF/SPGWU configuration

```
namespace: "oai" # namespace where SMF will be deployed 

image:
  repository: oai-spgwu-tiny # image name either locally present or in a public/private repository
  version: v1.1.2 # image tag

config:
  gwId: 1
  mnc: 208 # should match with AMF and SMF and gNB information
  mcc: 95 # should match with AMF and SMF and gNB information
  realm: "3gpp.org" 
  pidDirectory: "/var/run"
```

arora's avatar
arora committed
194 195
Configure `readinessProbe` and `livenessProbe` by default they are `true` to switch them off change the value with `false`

arora's avatar
arora committed
196 197 198 199 200 201
There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-spgwu-tiny/values.yaml). Infront of every parameter there is a comment. 

### 3.5 Configuring NRF

NRF configuration is straight forward, most of configurable parameters have comment infront of it for explaination. Refer the [values.yaml](../charts/oai-nrf/values.yaml)

arora's avatar
arora committed
202 203
Configure `readinessProbe` and `livenessProbe` by default they are `true` to switch them off change the value with `false`

arora's avatar
arora committed
204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221
### 3.6 Configuring subscriber data in MYSQL

Currently the MYSQL is configured with below subscriber data,

```
IMSI - 208950000000030-34
IMEI - 55000000000001
Secret Key (K) - 0x0C0A34601D4F07677303652C0462535B
OPc - 0x63bfa50ee6523365ff14c1f45f88737d
```

To configure new subscriber add a new entry in [values.yaml](../charts/mysql/values.yaml) line 345, add a new entry after 

```
INSERT INTO `users` VALUES (imsi,'380561234567',imei,NULL,'PURGED',50,40000000,100000000,47,0000000000,1,0xkey,0,0,0x40,'ebd07771ace8677a',0xOpC);
```

In above statement change `imsi`, `imei`, `key` and `opc` with appropriate value according to your environment. 
arora's avatar
arora committed
222 223 224 225 226

## 4. Deploying Helm Charts

Helm charts have an order of deployment for the proper configuration of core network. 

arora's avatar
arora committed
227
`mysql --> nrf --> amf --> smf --> upf(spgwu)`
arora's avatar
arora committed
228 229 230 231

Once the configuration is finished the charts can be deployed with a user who has the rights to

1. Create RBAC
arora's avatar
arora committed
232 233
2. Run pod with privileged and anyuid scc
3. Create multus binds
arora's avatar
arora committed
234 235 236

```
$ helm install mysql mysql/
arora's avatar
arora committed
237
# wait for the pod to be ready 
arora's avatar
arora committed
238
$ helm install amf oai-amf/
arora's avatar
arora committed
239
# wait for the pod to be ready 
arora's avatar
arora committed
240
$ helm install nrf oai-nrf/
arora's avatar
arora committed
241
# wait for the pod to be ready 
arora's avatar
arora committed
242
$ helm install smf oai-smf/
arora's avatar
arora committed
243
# wait for the pod to be ready 
arora's avatar
arora committed
244
$ helm install upf oai-spgwu-tiny/
arora's avatar
arora committed
245
# wait for the pod to be ready 
arora's avatar
arora committed
246 247
$ helm list
NAME  NAMESPACE       REVISION  UPDATED                                   STATUS    CHART                 APP VERSION
arora's avatar
arora committed
248 249 250 251 252
mysql oai-5g-develop  1         2021-07-29 14:20:34.010881045 +0200 CEST  deployed  mysql-1.6.9           5.7.30
amf   oai-5g-develop  1         2021-07-29 14:25:34.010881045 +0200 CEST  deployed  oai-amf-1.1.0         1.1.0      
nrf   oai-5g-develop  1         2021-07-29 14:16:11.530949242 +0200 CEST  deployed  oai-nrf-1.1.0         1.1.0      
smf   oai-5g-develop  1         2021-07-29 15:34:15.62785988 +0200 CEST   deployed  oai-smf-1.1.0         1.1.0      
upf   oai-5g-develop  1         2021-07-29 15:28:04.773480519 +0200 CEST  deployed  oai-spgwu-tiny-1.1.2  1.1.2  
Rohan's avatar
Rohan committed
253
```
arora's avatar
arora committed
254 255

Now go ahead and use OAI-gNB/dsTest/gNBSIM or any other gNB or emulator to test the deployed core network.