|
|
|
|
|
|
|
|
# Prerequisites
|
|
|
|
|
|
ElasticMon was developed and tested with Ubuntu server edition x86_64 GNU/Linux Ubuntu 16.04, kernel version 4.10.0-37-generic. ElasticMon requires one open port for ElasticSearch(tested with 6+) instance to run with the support of http methods and RESTful APIs.
|
|
|
|
|
|
**Note to readers:** _Please, read ElasticMon's **[terms of use](elasticmon-manual#Fundamentals)** first before you continue with this tutorial._
|
|
|
|
|
|
|
|
|
# Download and Install
|
|
|
|
|
|
## ElasticSearch Server
|
|
|
|
|
|
Visit [ElasticSearch Download page](https://www.elastic.co/downloads/elasticsearch) and download the server for your platform. For portable setup, follow the commands below by replacing the version of 6.x.x to any 6+ desired version.(i.e 6.3.1, 6.5.1 tested)
|
|
|
```bash
|
|
|
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.x.x.tar.gz
|
|
|
tar -xvf elasticsearch-6.x.x.tar.gz
|
|
|
rm elasticsearch-6.x.x.tar.gz
|
|
|
```
|
|
|
### Configurations
|
|
|
Update the file located at `elasticsearch-6.x.x/config/elasticsearch.yml` with your clustering and endpoint configurations.
|
|
|
_(Default configurations run elasticsearch on localhost, port 9200 for HTTP interface and 9300 for node connections)_
|
|
|
|
|
|
Configurations below will cause ElasticSearch to bind to every network interface it possibly can, including public ip addresses. A public endpoint is required for applications from remote networks to interact with _ElasticMON_.
|
|
|
```
|
|
|
network.host: 0.0.0.0
|
|
|
http.port: 9200
|
|
|
```
|
|
|
Update the file located at `elasticsearch-6.x.x/config/jvm.options` for configuring Elasticsearch JVM settings to ensure enough Java heap space.
|
|
|
|
|
|
For more information about configurations visit [ElasticSearch Configurations](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html)
|
|
|
|
|
|
### Run
|
|
|
Run the executable to start the server. Observe the terminal log to detect the binded endpoints(i.e `publish_address {193.xx.xx.xx:PORT}`)
|
|
|
```bash
|
|
|
elasticsearch-6.x.x/bin/elasticsearch
|
|
|
```
|
|
|
|
|
|
## Visualisation Tools
|
|
|
Visualisation tools are useful to analyse and plot metrics of recorded data.
|
|
|
### Kibana
|
|
|
|
|
|
Visit [Kibana Download page](https://www.elastic.co/downloads/kibana) and download it. For portable setup, follow the commands below by replacing the version of 6.x.x to any 6+ desired version.(i.e 6.3.1, 6.5.1 tested)
|
|
|
|
|
|
```bash
|
|
|
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.x.x-linux-x86_64.tar.gz
|
|
|
shasum -a 512 kibana-6.x.x-linux-x86_64.tar.gz
|
|
|
tar -xzf kibana-6.x.x-linux-x86_64.tar.gz
|
|
|
rm kibana-6.x.x-linux-x86_64.tar.gz
|
|
|
```
|
|
|
### Configurations
|
|
|
Update the file located at `kibana-6.x.x/config/kibana.yml` with your own configurations.
|
|
|
_(Default configurations run kibana on localhost, port 5601 and expect ElasticSearch to be on localhost:9200)_
|
|
|
|
|
|
Edit `kibana.yml` with your own ElasticSearch endpoint:
|
|
|
```
|
|
|
elasticsearch.url: "http:193.xxx.xxx.xxx:PORT" #ElasticSearch endpoint
|
|
|
server.port: 5601 #Kibana local execution port
|
|
|
```
|
|
|
Set elasticsearch authentication, **ONLY** if XPACK Security API is enabled on your elasticsearch server:
|
|
|
```
|
|
|
elasticsearch.username: "user"
|
|
|
elasticsearch.password: "pass"
|
|
|
```
|
|
|
|
|
|
### Run
|
|
|
Run the executable to start the tool:
|
|
|
```bash
|
|
|
kibana-6.x.x/bin/kibana
|
|
|
```
|
|
|
|
|
|
### Grafana
|
|
|
Visit [Grafana Download page](https://grafana.com/grafana/download) and download it.
|
|
|
_NOTE: ElaticSearch support on Grafana only exist on versions starting from v5.2+_
|
|
|
|
|
|
```bash
|
|
|
wget https://dl.grafana.com/oss/release/grafana-5.4.3.linux-amd64.tar.gz
|
|
|
tar -zxvf grafana-5.4.3.linux-amd64.tar.gz
|
|
|
```
|
|
|
|
|
|
### Run
|
|
|
Run the executable to start the tool:
|
|
|
|
|
|
By default grafana will bind to localhost, port 8080. Visit [Grafana Configurations](http://docs.grafana.org/installation/configuration/) to edit configurations.
|
|
|
```bash
|
|
|
grafana-5.x.x/bin/grafana-server
|
|
|
```
|
|
|
|
|
|
Add your ElasticSearch as a DataFrame and click test to confirm.
|
|
|
|
|
|
|
|
|
# Use Case examples
|
|
|
|
|
|
There are more than one way to interract with an ElasticSearch server. Visit [ElasticSearch Clients](https://www.elastic.co/guide/en/elasticsearch/client/index.html) or [ElasticSearch Community Contributed Clients](https://www.elastic.co/guide/en/elasticsearch/client/community/current/index.html) to see the list of clients they support.
|
|
|
In here we will simply show one using REST API, and one with Python client.
|
|
|
|
|
|
## RestAPI Example:
|
|
|
|
|
|
Command below sends a GET request to ElasticSearch instance used to store FlexRAN configs and stats.
|
|
|
This search query is directed to index `enb_config` and will sort data in date_time descending order, filter only the ones monitored in the last 5 minutes, and return date_time of measurement, agent_id, ul and dl slice percentages.
|
|
|
|
|
|
```bash
|
|
|
curl -XGET "http://ElasticSearchURL:PORT/enb_config/_search" -H 'Content-Type: application/json' -d' { "_source": {"includes": [ "eNB_config.agent_info.agent_id","eNB_config.eNB.cellConfig.sliceConfig.dl.percentage","eNB_config.eNB.cellConfig.sliceConfig.ul.percentage"]}, "sort": [ { "date_time": { "order": "desc" } } ], "query": { "range": { "date_time": { "gt": "now-5m" } } } }'
|
|
|
```
|
|
|
|
|
|
Output for only one:
|
|
|
```json
|
|
|
"date_time": "2019-02-13T13:58:38.189",
|
|
|
"eNB_config": [
|
|
|
{
|
|
|
"agent_info": [{"agent_id": 1}],
|
|
|
"eNB": {"cellConfig": [{"sliceConfig": {"ul": [{"percentage": 100}],"dl": [{"percentage": 100}]}}]}
|
|
|
}
|
|
|
```
|
|
|
|
|
|
Visit [ElasticQuery CheatSheet](http://elasticsearch-cheatsheet.jolicode.com/) to learn more properties about queries.
|
|
|
|
|
|
## Python Client Example:
|
|
|
Install elasticsearch Python libraries, check the [manual page](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/index.html):
|
|
|
```bash
|
|
|
pip install elasticsearch
|
|
|
```
|
|
|
|
|
|
Create a connection to ElasticSearch server:
|
|
|
```python
|
|
|
es = Elasticsearch(
|
|
|
[ElasticMON_Url],
|
|
|
#http_auth=('elastic', 'password'),
|
|
|
port=ElasticMON_Port,
|
|
|
#use_ssl=True,
|
|
|
#verify_certs=True,
|
|
|
#ca_certs=certifi.where(),
|
|
|
)
|
|
|
```
|
|
|
|
|
|
Send read/write/delete commands using the connection:
|
|
|
|
|
|
Code block below will send a search query, which will return only 1 resulting JSON where the date_time is the most recent and the rrcMeasurements were activated for rsrp and rsrq measurements to be active.
|
|
|
```python
|
|
|
response = es.search(index="mac_stats", body={"size" : 1,"sort": [{"date_time": {"order": "desc"}}],"query":{"bool":{"must":[{"range":{"mac_stats.ue_mac_stats.mac_stats.rrcMeasurements.measid":{"gt":0}}}]}}})
|
|
|
|
|
|
#extract response from the returned JSON _source field
|
|
|
mac_stats = json.loads(response['hits']['hits'][0]['_source'])
|
|
|
|
|
|
#extract fields from the returned object
|
|
|
measurement_datetime = mac_stats['date_time']
|
|
|
wbcqi = actual_wbcqi=mac_stats['mac_stats'][0][u'ue_mac_stats'][0][u'mac_stats']['dlCqiReport'][u'csiReport'][0][u'p10csi'][u'wbCqi']
|
|
|
rsrp = mac_stats['mac_stats'][0][u'ue_mac_stats'][0][u'mac_stats'][u'rrcMeasurements'][u'pcellRsrp']
|
|
|
.
|
|
|
.
|
|
|
.
|
|
|
print("Wbcqi =",wbcqi)
|
|
|
print("measurement_datetime =",measurement_datetime)
|
|
|
print("rsrp =",rsrp)
|
|
|
.
|
|
|
.
|
|
|
#Create data frames from the returned list of json objects if necessary for Machine Learning or other purposes
|
|
|
```
|
|
|
|
|
|
Output:
|
|
|
```
|
|
|
Wbcqi = 11
|
|
|
measurement_datetime = 2018-12-03T16:24:04.879
|
|
|
rsrp = -105
|
|
|
``` |
|
|
\ No newline at end of file |