Skip to content

Fix several issues in ulsch_mmse_2layers

Bartosz Podrygajlo requested to merge fix-type-remove-assert-2layer-mmse into develop

Fix incorrect array type of determ_fin which was set to int32_t while the array holds the results of simde_mm_abs_epi32 which produces unsigned values.

Remove assert on right-shifting negative integers as it is impossible for determ_fin to hold negative values.

Change type of sum_det to uint32_t to prevent possible overflow.

Fix the type cast (int -> uint32_t) to prevent casting before shift in nr_ulsch_comp_muli_sum which might have resulted in right shifting negative integers.

Closes #753 (closed)

Edited by Bartosz Podrygajlo

Merge request reports

Loading