Fix problems detected by UBsan

ldpc decoder breaks in error if we use 'gcc --sanitize' because of variable arrays that have null size
these arrays are not used later in the code, but still it is not good

the casts in the middle are not correct: a tab[x][y] can be in C either pointer of pointers, or actual 2D array that is not a pointer to pointer but a multiplication of the dimensions to find the right offset

digging in the code, i found a lot of repeated code, that i had to fix as it is deeply linked in these invalid variable arrays

note of something more that may be done the special cases "BN3" could be removed, if the const arrays for this BN are created and initialized with proper value (0)