DEPLOY_SA5G_HC.md 10.8 KB
Newer Older
arora's avatar
arora committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14
<table style="border-collapse: collapse; border: none;">
  <tr style="border-collapse: collapse; border: none;">
    <td style="border-collapse: collapse; border: none;">
      <a href="http://www.openairinterface.org/">
         <img src="./images/oai_final_logo.png" alt="" border=3 height=50 width=150>
         </img>
      </a>
    </td>
    <td style="border-collapse: collapse; border: none; vertical-align: center;">
      <b><font size = "5">OpenAirInterface 5G Core Network Deployment using Helm Charts</font></b>
    </td>
  </tr>
</table>

arora's avatar
arora committed
15 16 17 18 19 20 21


The cloud native network functions in production will be deployed using a production grade container orchestrator like kubernetes or Openshift. OAI 5g Core network is being designed to keep up with the latest cloud native deployment scenarios. This release of helm charts is focused on deploying each Cloud-native Network Function(CNF) individually.

![Helm Chart Deployment](./images/helm_diag.png)


arora's avatar
arora committed
22 23 24
**TABLE OF CONTENTS**

1.  [Description](#1-description)
arora's avatar
arora committed
25
2.  [Building Images](#2-building-images)
arora's avatar
arora committed
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
3.  [Configuring Helm Charts](#3-configuring-helm-charts)
4.  [Deploying Helm Charts](#4-deploying-helm-charts)


## 1. Description

The helm charts can be used on any production grade kubernetes cluster. Currently they are only tested on our inhouse Openshift cluster the cluster information can be found below.

| Software                 | Version                             |
|:------------------------ |:----------------------------------- |
| Openshift Client Version | 4.4.10                              |
| Kubernetes Version       | Kubernetes Version: v1.17.1+45f8ddb |
| helm                     | v3.5.3                              |

### Pre-requisite

arora's avatar
arora committed
42
The cluster on which these helm charts will be deployed should have RBAC and [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni). Multus is necessary to provide multiple interfaces to AMF and UPF/SPGWU. 
arora's avatar
arora committed
43 44 45

## 2. Building Images

arora's avatar
arora committed
46
The base image used by network function is dependent on the operating system it will be running on. If it is a debian (ubuntu) based cluster then base image will be ubuntu. If it is a RPM (core-os) based cluster then base images will UBI8. Follow the tutorial on [how to build images](./BUILD_IMAGES.md) depending on the cluster/worker-node operating system. In case of Ubuntu based worker nodes, the images can be pulled from [docker-hub](./RETRIEVE_OFFICIAL_IMAGES.md).
arora's avatar
arora committed
47 48 49

## 3. Configuring Helm Charts

arora's avatar
arora committed
50
Clone the helm chart repository from gitlab repository
arora's avatar
arora committed
51 52

```
arora's avatar
arora committed
53
$ git clone -b v1.1.0 https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-fed.git
arora's avatar
arora committed
54 55 56 57 58
$ cd charts
$ ls charts
mysql  oai-amf  oai-nrf  oai-smf  oai-spgwu-tiny
```

arora's avatar
arora committed
59
Helm chart of every network function looks similar and has the below structure. Only the chart of mysql database is different and the NRF helm chart has an extra pvc.yaml to create a presistant volume for storing tcpdump.
arora's avatar
arora committed
60 61 62 63 64

```
Network_function/
├── Chart.yaml
├── templates
arora's avatar
arora committed
65 66 67 68 69 70 71 72
│             ├── configmap.yaml
│             ├── deployment.yaml
│             ├── _helpers.tpl
│             ├── multus.yaml
│             ├── NOTES.txt
│             ├── rbac.yaml
│             ├── serviceaccount.yaml
|             └── service.yaml
arora's avatar
arora committed
73 74 75 76 77
└── values.yaml 

1 directory, 10 files
```

arora's avatar
arora committed
78
All the configurable parameters for a particular commit/release are mentioned in the `values.yaml` file. These parameters will keep on changing in the future depending on the nature of development and features. 
arora's avatar
arora committed
79

arora's avatar
arora committed
80
**NOTE**: If there is a need to edit a specific configuration parameter that is not configurable via the helm-chart values.yaml file then it has to be changed at the time of building images.
arora's avatar
arora committed
81

arora's avatar
arora committed
82
Depending on the namespace where these charts will be instantiated change the value of `namespace` parameter in `values.yaml`. All the network function related configurable parameters are in the sections `config` of the `values.yaml`. To understand the usage and description of each network function configuration parameter refer their [wiki page](https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-amf/-/wikis/home).
arora's avatar
arora committed
83

arora's avatar
arora committed
84
Create a namespace where the helm-charts will be deployed, in our environment we deploy them in `oai` namespace. To create a namespace use the below command on your cluster, 
arora's avatar
arora committed
85

arora's avatar
arora committed
86 87 88 89 90 91
```bash
# needs a user which has the right to create namespaces
$ kubectl create ns oai
or 
$ oc new-project oai
```
arora's avatar
arora committed
92

arora's avatar
arora committed
93

arora's avatar
arora committed
94
### 3.1 Networking related information
arora's avatar
arora committed
95

arora's avatar
arora committed
96
Network functions communicate based on *kubernetes service concept*, the network functions are using FQDN of other network functions to use their service. For example AMF registers with NRF using NRF ip-address. This way we can get rid of any static ip-address configuration. Except for the interfaces which are supposed to be reached from outside the cluster environment. GNB will not be included in the cluster. Thus N1/N2/NGAP and N3/GTP-U interface are provided using multus CNI to AMF and UPF pod. 
arora's avatar
arora committed
97

arora's avatar
arora committed
98
In our environment to reduce complexity and reduce static ip-address allocation we are providing only one interface to each pod (default Kubernetes CNI) except AMF and UPF where we provide two interfaces because gNB or gNB emulator is not running in the cluster environment. For example SMF can have different interfaces/ip-addresses for N4, Nsmf but in our case we are providing only one interface/ip-address to take benifit of *Kubernetes service concept*. But using multus there can be multiple interfaces of SMF too. 
arora's avatar
arora committed
99 100 101

### 3.2 Configuring AMF

arora's avatar
arora committed
102
Open the [values.yaml](../charts/oai-amf/values.yaml) to configure the required parameters. There are many parameters which can be configured. Below are some important parameters,  
arora's avatar
arora committed
103
```
arora's avatar
arora committed
104 105 106 107 108 109 110 111
namespace: "oai" # namespace where AMF will be deployed 

image:
  repository: oai-amf # image name either locally present or in a public/private repository
  version: v1.1.0 # image tag

# configure these values based on gNB/emulator configuration
config: 
arora's avatar
arora committed
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128
  mcc: "208"
  mnc: "95"
  regionId: "128"
  amfsetId: "1"
  servedGuamiMcc0: "208"
  servedGuamiMnc0: "95"
  servedGuamiRegionId0: "128"
  servedGuamiAmfSetId0: "1"
  servedGuamiMcc1: "460"
  servedGuamiMnc1: "11"
  servedGuamiRegionId1: "10"
  servedGuamiAmfSetId1: "1"
  plmnSupportMcc: "208"
  plmnSupportMnc: "95"
  plmnSupportTac: "0xa000"
  sst0: "222"
  sd0: "123"
arora's avatar
arora committed
129 130
  sst1: "111"
  sd1: "124"
arora's avatar
arora committed
131 132 133 134 135 136 137 138 139 140 141 142
```

The mysql database is pre-configured with some subscriber information. If there is new subscriber information then it should be configured in the mysql database. The subscriber PLMN information should match with gNB and AMF. 

```
  mySqlServer: "mysql"
  mySqlUser: "root"
  mySqlPass: "linux"
  mySqlDb: "oai_db"
  operatorKey: "63bfa50ee6523365ff14c1f45f88737d" (should be the same in mysql)
```

arora's avatar
arora committed
143 144
There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-amf/values.yaml). Infront of every parameter there is a comment. 

arora's avatar
arora committed
145 146
### 3.3 Configuring SMF

arora's avatar
arora committed
147
Open [values.yaml](../charts/oai-smf/values.yaml) to configure SMF configuration. DNS configuration which will be shared with the UE based on the DNS used in your environment, it can be 8.8.8.8 or 4.4.4.4 if you don't know your DNS. 
arora's avatar
arora committed
148 149

```
arora's avatar
arora committed
150 151 152 153 154 155 156
namespace: "oai" # namespace where SMF will be deployed 

image:
  repository: oai-smf # image name either locally present or in a public/private repository
  version: v1.1.0 # image tag

config:
arora's avatar
arora committed
157 158 159 160
  dnsIpv4Address: ""
  dnsSecIpv4Address: ""
```

arora's avatar
arora committed
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207
Currenly DNN related information and UE network related information can only be changed at the time of building the SMF network function. Refer [Configure the Containers](./CONFIGURE_CONTAINERS.md) to understand how it can be done. 

There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-smf/values.yaml). Infront of every parameter there is a comment. 

### 3.4 Configuring UPF

Open [values.yaml](../charts/oai-spgwu-tiny/values.yaml) to configure UPF/SPGWU configuration

```
namespace: "oai" # namespace where SMF will be deployed 

image:
  repository: oai-spgwu-tiny # image name either locally present or in a public/private repository
  version: v1.1.2 # image tag

config:
  gwId: 1
  mnc: 208 # should match with AMF and SMF and gNB information
  mcc: 95 # should match with AMF and SMF and gNB information
  realm: "3gpp.org" 
  pidDirectory: "/var/run"
```

There are more parametes which can be configured like extra interfaces and resource usage by network function, please refer [values.yaml](../charts/oai-spgwu-tiny/values.yaml). Infront of every parameter there is a comment. 

### 3.5 Configuring NRF

NRF configuration is straight forward, most of configurable parameters have comment infront of it for explaination. Refer the [values.yaml](../charts/oai-nrf/values.yaml)

### 3.6 Configuring subscriber data in MYSQL

Currently the MYSQL is configured with below subscriber data,

```
IMSI - 208950000000030-34
IMEI - 55000000000001
Secret Key (K) - 0x0C0A34601D4F07677303652C0462535B
OPc - 0x63bfa50ee6523365ff14c1f45f88737d
```

To configure new subscriber add a new entry in [values.yaml](../charts/mysql/values.yaml) line 345, add a new entry after 

```
INSERT INTO `users` VALUES (imsi,'380561234567',imei,NULL,'PURGED',50,40000000,100000000,47,0000000000,1,0xkey,0,0,0x40,'ebd07771ace8677a',0xOpC);
```

In above statement change `imsi`, `imei`, `key` and `opc` with appropriate value according to your environment. 
arora's avatar
arora committed
208 209 210 211 212

## 4. Deploying Helm Charts

Helm charts have an order of deployment for the proper configuration of core network. 

arora's avatar
arora committed
213
`mysql --> nrf --> amf --> smf --> upf(spgwu)`
arora's avatar
arora committed
214 215 216 217

Once the configuration is finished the charts can be deployed with a user who has the rights to

1. Create RBAC
arora's avatar
arora committed
218 219
2. Run pod with privileged and anyuid scc
3. Create multus binds
arora's avatar
arora committed
220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238

```
$ helm install mysql mysql/
# wait for the pod to come up
$ helm install amf oai-amf/
# wait for the pod to come up
$ helm install nrf oai-nrf/
# wait for the pod to come up
$ helm install smf oai-smf/
# wait for the pod to come up
$ helm install spgwu oai-spgwu-tiny/
# wait for the pod to come up
$ helm list
NAME  NAMESPACE       REVISION  UPDATED                                   STATUS    CHART                 APP VERSION
amf   oai-5g-develop  1         2021-05-12 12:07:36.345877418 +0200 CEST  deployed  oai-amf-0.1.0         0.1.0 
mysql oai-5g-develop  1         2021-05-12 11:09:32.597525506 +0200 CEST  deployed  mysql-1.6.9           5.7.30     
nrf   oai-5g-develop  1         2021-05-12 11:32:43.648706741 +0200 CEST  deployed  oai-nrf-0.1.0         0.1.0 
smf   oai-5g-develop  1         2021-05-12 12:05:38.251220635 +0200 CEST  deployed  oai-smf-0.1.0         0.1.0 
spgwu oai-5g-develop  1         2021-05-12 12:08:31.408369994 +0200 CEST  deployed  oai-spgwu-tiny-0.1.1  0.1.1 
Rohan's avatar
Rohan committed
239
```
arora's avatar
arora committed
240 241

Now go ahead and use OAI-gNB/dsTest/gNBSIM or any other gNB or emulator to test the deployed core network.