Currently it is hosted on the CI Jenkins server on this job/item.
A build is automatically triggered on this job by actions on the OAI eNB GitLab repository. See the repository integrations page for more details.
The eNB-CI master job is configured to start when:
a push is made to develop branch
a merge-request to the develop branch is opened.
when extra commits are pushed to the source branch of an already opened merge request
The last one allows developers to fix issues on their working branch when requested by CI administrator or integrators.
The eNB-CI master job may be configured without any parameter. In that case no social media notifications (such as Slack, Mattermost, ...) is performed.
At time of writing, the eNB-CI master job hosted at Eurecom is using the following parameters:
pipelineUsesSlack : the master job will send notifications to OAI Slack workspace
RedHatWorkingPath : temporaly CentOS build is done on a real server and not a virtual machine. These parameters will be removed once VM CentOS build is available.
2. Source Control of the Jenkins pipeline script
IMPORTANT: the Jenkins Declarative Pipeline script file is retrieved from Source Control Management (SCM) under GIT w/ the develop branch.
It means that a temporary clone of the openairinterface5g repository is made on the latest version of develop branch to retrieve one file: ci-scripts/Jenkinsfile-gitlab.
This file will be loaded by the Jenkins master job and executed.
During a Merge-Request, if modifications to this particular file are made, they WON'T be taken into account.
Last important point: on the Jenkins web interface, there is a replay button for a job. This mechanism works when it is a push event to develop. It does not work when it is a merge request event. The best way to replay a Merge-Request build is to:
Go to the GitLab Integration page of the repository
Select the Merge Request Hook event that triggered the job
Click on View Details
Make sure it is the correct event by examining the payload
Click on the Resend Request
Another way is to push a new commit to the source branch.
3. Declarative Pipeline Script
The script is written in Groovy, a Java-derivative language, using the Declarative Pipeline syntax. It does not prevent us to use some Scripted Pipeline syntax elements when needed.
The first few blocks are written to:
Specify the node to work on
Specify some generic options such as timestamping, using AnsiColor, preparing the connection with the GitLab server.
Using gitlabBuilds keyword with a set of stages sends a message to the GitLab server. The latter performs the following steps:
Create a GitLab pipeline
Associate this pipeline with the corresponding commit or Merge-Request that triggered the Jenkins job
Associate this GitLab pipeline with the Jenkins master job
Create an external job for each stage specified and put them in a pending stage ().
Stages are the following:
3.1. Verify Parameters stage
In this stage we are verifying if the parameters are consistent.
At the time of writing:
Redhat - CentOS parameter check: if all 3 needed parameters are present in the Jenkins Master job configuration, CentOS build will be performed on a remote CentOS server. If not, no CentOS build will be scheduled.
3.2. Verify Guidelines stage
In this stage we are verifying several guidelines:
The Coding Formatting Rules
[TBD] The "Non-CI-Admin" modifications rules:
CI and Configuration files SHALL NOT be modified by an non-CI expert
Also in this stage, in case of a Merge-Request we are performing a temporary merge between the source branch and the target branch to work on.
In case of merge conflicts, the master job will stop immediately and fail the Merge-Request.
3.3. Start Virtual Machine stages
The different needed VMs are started sequentially. We tried to start them in parallel but it was too much stress on the host HW. We were seeing failures.
**[TBD]**We could start them 2 by 2 to speed up a bit the processing.
Note if a VM start is forgotten for a given variant, the corresponding VM will be created and started during the build stage.
The created VMs are brand new systems with a minimal package configuration.
3.4. Build stages
Each variant build is performed in parallel on its corresponding VM. OAI build script will install all the needed packages. Each stage should be surrounded with the gitlabCommitStatus keyword associated with the stage name.
When entering the stage, the status on the external job on the GitLab pipeline will pass from pending () to running ().
Log files will be used to determine the pass/fail status of each build.
When entering the stage, the status on the external job on the GitLab pipeline will pass from running () to pass or fail ().
A dedicated VM performs static code analysis using the cppcheck tool.
At the time of writing, errors and warnings detected by this method are not a blocking stage.
When all variant builds, an HTML report is generated (like this one) and added to the job artifacts with all compilation logs.
Also in case of a Merge-Request, a script is analysing if the contributor did not introduce new compilation warnings. In that case, a comment in the Merge-Request discussion is added with the list of impacted files.
3.5. Test stages
Within the VM, testing that do not require HW resources (such as RF boards and COTS-UE) is performed.
At time of writing the list of performed tests are:
When testing requires HW resources, the testing is deported to a slave job. See this page for more details.
If slave jobs share the same resources (like EPC and/or UEs), they should be run sequentially. Otherwise they can be run in parallel.
Normally an HTML report is created by each slave job. The master job should retrieve and archives them.
3.6. Destroy all VMs stage
At this stage, all build/test logs should have be retrieved from the Virtual Machines. They can be stopped and their images removed from the file system.
This stage could be done in case of failure also at the end of the pipeline in the final post block.
3.7. Final post block
Based on the status of all stages, a pass/fail message is added to the comment of the merge request.
If social media messaging is enabled, a message can be sent to such a platform (like Slack, Mattermost, ...)