Skip to content

Parallelize ASN.1 generation and build

Robert Schmidt requested to merge parallel-asn1 into develop

Ok, I give it a second try after !1087 (closed).

This parallelizes the generation and build of ASN.1 structures. It uses add_subdirectory() to recursively descent into the directories that contain the ASN.1 files (LTE RRC, NR RRC, S1AP, NGAP, M2AP, M3AP, X2AP, F1AP, LPP; E1AP has already been done this way). Using add_custom_command(), we trigger the generation of ASN.1 files during build time (i.e., when running make/ninja, and not cmake). This is possible, by defining, for each target, a list of the generated files. This is important for cmake to be able to recognize if/which files have changed/created, so that it can build them after generation in a second step. Note that I don't consider this a disadvantage, as it (a) recommended by cmake to list all files independently instead of globbing (b) the same files are always generated anyway. If somebody wants to upgrade ASN.1, they simply have to provide a new list of files, which is trivial to do.

There is a README.md in the LTE RRC

time ./build_oai --ninja --gNB --eNB --nrUE --UE -P --build-lib all -c

On my machine, the total compilation time, by this parallelization, goes from

real	11m9,426s
user	70m7,410s
sys	12m4,164s

to

real	8m59,358s
user	72m20,434s
sys	12m30,563s

-> 20% speed up

I measured on a faster machine (however, I cannot build the ue_ip module, thus it stops there), the time shrinks from

real	4m22.323s
user	22m12.106s
sys	7m25.644s

to

real	2m37.118s
user	22m13.403s
sys	7m43.479s

-> 40% speedup

Open Points:

  • there is some manual patching of some generated files in cmake_targets/tools/make_asn1c_includes.sh which I did not do yet. According to @Cedric.Roux, there should be warnings but I see none
  • there is a custom target asn1 which forces the generation and compilation of all ASN.1 targets -- for many of our targets this is necessary, but it might not always be the case. We could group all asn1 targets (X2AP, ...) and only have some top-level targets depend on then, e.g., nr-softmodem, such that the compilation of this one will trigger all asn1, but others not (e.g., certain utilities don't depend on ASN.1)
  • ue_ip is broken is broken on some kernels
  • !1828 (merged): the new asn1c has some changed command line options that we need to fix here as well
  • I put the 4G/5G labels to check that the small code changes I made don't have an effect. Otherwise BUILD-ONLY should be enough in theory
  • there is a helper script test_compile_targets.sh to compile many targets (that I deemed "important", see targets.working) individually, to verify that I don't break anything. You can use it if you want to make tests
Edited by Robert Schmidt

Merge request reports