CI: rework log analysis to simplify and allow new use cases

The CI has some log analysis, but it is unmaintainable (huge functions that mix eNB/gNB or lteUE/nrUE, are not tested, most of it is not used or useful...) or limited to specific use cases (the timing-phytest time analysis is hardcoded to source builds instead of also handling analysis when running from docker images). This MR reimplements both functions. For the log analysis, the CI team decided to throw away the old analysis, and reimplement a mechanism that checks the couple of things we care for.

  • Allow to analyze phy-test timing analysis from docker files, and add a unit test
  • Rework existing log analysis to remove useless checks, and make it more composable
    • The undeployment steps (Undeploy_Object(), Terminate_eNB()) take an <analysis> XML object which then either has a <services> subelement with check that are whitespace delimited (similar to how other xml task steps have <services>). For checks that contain whitespace, they can use <service> (singular) that encodes a single check, such that e.g., arguments could include spaces.
    • Each check is of form service=check[=arguments], where service is the docker services, check is a checking function (implemented as a class with run(file, options) function inside cls_loganalysis.py, and options arguments after the second = that is to be read by run() (basically, options will contain arguments that can be freeform)
    • these definitions can occur multiple times, e.g., it could be serviceA=Check1 and serviceA=Check2=Args and it will execute both Check1 and Check2
    • if a service is declared, it will execute alse the Default check which currently checks for an assertion
    • Implemented (and inspired from the existing CI analysis) are:
      • Default check to check for assertions (will be extended, to e.g., address/undefined behavior sanitizer analysis)
      • RetxCheck: retransmission check as used in the CI
      • LastLineContains to check the last line containes a specific string
      • ContainsString to check a specific string appears in logs
      • EndsWithBye: check that towards the end (but not necessarily the last line), there is a Bye. string (a workaround since we don't always terminate with 0)
  • There is a plan to use return codes after stopping in docker, but most executables do not end with 0 most of the time, such that we needed to abandon this plan.
  • Additional fixes (see commit messages) and delete unused code
Edited by Robert Schmidt

Merge request reports

Loading