[lib] Move type cast in Fletcher checksum

The int type cast should be on the whole expression
passed to the mod operator. Otherwise it won't work
when/if c0/c1 is unsigned.
Making c0/c1 unsigned makes it possible to use 5802
as MODX value.
This commit is contained in:
Joakim Tjernlund 2008-11-17 11:22:25 +01:00 committed by Paul Jakma
parent 4768061ad5
commit 6e907dd4ab

View File

@ -91,7 +91,7 @@ fletcher_checksum(u_char * buffer, const size_t len, const uint16_t offset)
}
/* The cast is important, to ensure the mod is taken as a signed value. */
x = ((int)(len - offset - 1) * c0 - c1) % 255;
x = (int)((len - offset - 1) * c0 - c1) % 255;
if (x <= 0)
x += 255;