Parallelize ASN.1 generation and build
Ok, I give it a second try after !1087 (closed).
This parallelizes the generation and build of ASN.1 structures. It uses add_subdirectory()
to recursively descent into the directories that contain the ASN.1 files (LTE RRC, NR RRC, S1AP, NGAP, M2AP, M3AP, X2AP, F1AP, LPP; E1AP has already been done this way). Using add_custom_command()
, we trigger the generation of ASN.1 files during build time (i.e., when running make/ninja, and not cmake). This is possible, by defining, for each target, a list of the generated files. This is important for cmake to be able to recognize if/which files have changed/created, so that it can build them after generation in a second step. Note that I don't consider this a disadvantage, as it (a) recommended by cmake to list all files independently instead of globbing (b) the same files are always generated anyway. If somebody wants to upgrade ASN.1, they simply have to provide a new list of files, which is trivial to do.
There is a README.md in the LTE RRC
time ./build_oai --ninja --gNB --eNB --nrUE --UE -P --build-lib all -c
On my machine, the total compilation time, by this parallelization, goes from
real 11m9,426s
user 70m7,410s
sys 12m4,164s
to
real 8m59,358s
user 72m20,434s
sys 12m30,563s
-> 20% speed up
I measured on a faster machine (however, I cannot build the ue_ip module, thus it stops there), the time shrinks from
real 4m22.323s
user 22m12.106s
sys 7m25.644s
to
real 2m37.118s
user 22m13.403s
sys 7m43.479s
-> 40% speedup
Open Points:
- there is some manual patching of some generated files in cmake_targets/tools/make_asn1c_includes.sh which I did not do yet. According to @Cedric.Roux, there should be warnings but I see none
- there is a custom target
asn1
which forces the generation and compilation of all ASN.1 targets -- for many of our targets this is necessary, but it might not always be the case. We could group all asn1 targets (X2AP, ...) and only have some top-level targets depend on then, e.g.,nr-softmodem
, such that the compilation of this one will trigger allasn1
, but others not (e.g., certain utilities don't depend on ASN.1) - ue_ip is broken is broken on some kernels
- !1828 (merged): the new asn1c has some changed command line options that we need to fix here as well
- I put the 4G/5G labels to check that the small code changes I made don't have an effect. Otherwise BUILD-ONLY should be enough in theory
- there is a helper script
test_compile_targets.sh
to compile many targets (that I deemed "important", seetargets.working
) individually, to verify that I don't break anything. You can use it if you want to make tests