0

I need to calculate a CRC for a data array that is identical to the result of the _crc16_update function in avr-libc.

I understand the _crc16_update function to be:

uint16_t crc16_update(uint16_t crc, uint8_t a)
{
    int i;

    crc ^= a;
    for (i = 0; i < 8; ++i)
    {
        if (crc & 1)
            crc = (crc >> 1) ^ 0xA001;
        else
            crc = (crc >> 1);
    }

    return crc;
}

My current code in an Android app to calculate the same CRC:

private short crc16Update( short crc, byte a )
{
    short i;

    crc ^= a;
    for( i = 0; i < 8; ++i )
    {
        if ( (crc & 1) > 0 )
            crc = (crc >>> 1) ^ 0xA001;
        else
            crc = (crc >>> 1);
     }

    return crc;
}

I call that method with the following loop (_hexBytes is the array containing the data, _crc is supposed to hold the value identical to the CRC calculated by the avr-libc function):

for( Byte b : _hexBytes )
    _crc = crc16Update( _crc, b );

I'm not getting the same result on both ends. Can anybody see an error in my code to calculate the CRC?

Here is some test data I've used:

Data: [0x0c, 0x94, 0x39]
CRC from my Java code: 0x0827
CRC from the avr-libc code: 0xd16e

Maybe it's related to the Java short being signed?

Khantahr
  • 8,156
  • 4
  • 37
  • 60
  • What are you getting and what do you expect to get? `int` is a 32-bit type in Java. Try using `short`. – ajb Jun 05 '17 at 00:18
  • It's about 30kB of data being checksumed, though I just realized I could replace it with a few bytes for testing purposes. I'll try that. – Khantahr Jun 05 '17 at 00:20
  • Using `int` instead of `short` was definitely part, if not all, of the problem, still testing...thanks. – Khantahr Jun 05 '17 at 00:40
  • Still doesn't work, I added an edit with some data and resulting CRC values. – Khantahr Jun 05 '17 at 01:03
  • You should look up the table-driven version. It is eight times as fast, at least. – user207421 Jun 05 '17 at 01:04
  • Would using the table driven version in Java produce the same result as the function in avr-libc? I can't change that end, I have to make the Java side match. – Khantahr Jun 05 '17 at 01:06
  • Yes, of course it does, otherwise it would be pointless. I have a Java version I can post here as an answer, do you want? – user207421 Jun 05 '17 at 01:25
  • If you can post it, it would be much appreciated. – Khantahr Jun 05 '17 at 01:27
  • @ajb replacing an `uint16_t` variable from C with a `short`in Java is actually a very bad advice. Java doesn't know any unsigned types, so they have to be "emulated" using the next bigger integer type and masking out the unneeded bits. Otherwise, you get really unpredictable behaviour on underflow and overflow or when shifting the bits around. – tofro Jun 05 '17 at 12:21
  • @tofro I think as long as you use `>>>` for right shift, which the OP did, it doesn't matter. The bit patterns will be the same. And Java's behavior on overflow is clearly defined, so I don't think your last comment is correct. Using a larger integer type means you have to be concerned with 1 bits in the upper half of the integer which could then get shifted in with a right-shift, unless you are careful to `&` all your results with `0xFFFF`. If you think I'm wrong, please show me a concrete example of how using `short` would fail. – ajb Jun 05 '17 at 18:37
  • @tofro is right, I only got it to work after changing everything to int and using bitmasks. Using a table method similar to the accepted answer. – Khantahr Jun 05 '17 at 22:06
  • OK, it looks like tofro is right but for all the wrong reasons. I was wrong, because I forgot that Java automatically promotes the `short` to an `int` (which sign-extends) before applying `>>>`, which defeats the purpose of `>>>`. (Which seems wrong, but I guess they had their reasons.) If `x` is a `short`, `(x & 0xFFFF) >>> 1` should work fine (because `x` is promoted to `int` before the `&` operator). tofro is still wrong about the behavior being unpredictable. – ajb Jun 06 '17 at 04:21

1 Answers1

0

Throw it away and use the table-driven version, which is at least 8 times as fast. You have to adjust the parameters of course, e.g. your polynomial is 0xA001, which looks like POLYNOMIAL_IBM below.

/**
 * CRC16. An implementation of {@link java.util.zip.Checksum} for 16-bit cyclic redundancy checks.
 *
 */

import java.util.zip.Checksum;

/**
 * CRC16. An implementation of {@link java.util.zip.Checksum} for 16-bit cyclic redundancy checks.
 * 
 * @author Esmond Pitt.
 */
public class CRC16 implements Checksum
{
    public enum Algorithm
    {
        CRC_CCIT(POLYNOMIAL_CCITT, INIT_CCITT),
        XMODEM(POLYNOMIAL_XMODEM, (short)0),
        ARC(POLYNOMIAL_ARC, (short)0),
        IBM(POLYNOMIAL_IBM, (short)0),
        ANSI(POLYNOMIAL_IBM, (short)0);

        Algorithm(int polynomial, short initialValue)
        {
            this.polynomial = polynomial;
            this.initialValue = initialValue;
        }

        int polynomial;
        short initialValue;
    };

    /**
     * CRC-CCITT polynomial, used by X.25, V.41, Bluetooth, PPP, IrDA, BACnet; known as CRC-CCITT
     */
    public static final int POLYNOMIAL_CCITT = 0x1021;
    public static final short INIT_CCITT = (short)0xffff;
    /**
     * CRC-16 polynomial, used by XMODEM, USB, many others; also known as CRC-16
     */
    public static final int POLYNOMIAL_IBM = 0xa001;
    public static final int POLYNOMIAL_XMODEM   = 0x8408;
    public static final int POLYNOMIAL_ARC = 0x8005;

//  private final int   polynomial;
    private final short init;
    private final short[]   crcTable = new short[256];
    private short   value;

    /**
     * Construct a CRC16-CCIT, with polynomial=0x1021 and initial value=0xffff.
     */
    public CRC16()
    {
        this(Algorithm.CRC_CCIT);
    }

    /**
     *  COnstruct a polynomial specifying the algorithm.
     *
     * @param algorithm Algorithm
     */
    public CRC16(Algorithm algorithm)
    {
        this(algorithm.polynomial, algorithm.initialValue);
    }

    /**
     * Construct a CRC16 specifying the polynomial and initial value.
     * @param polynomial Polynomial, typically one of the POLYNOMIAL_* constants.
     * @param init Initial value, typically either 0xffff or zero.
     */
    public CRC16(int polynomial, short init)
    {
//      this.polynomial = polynomial;
        this.value = this.init = init;
        for (int dividend = 0; dividend < 256; dividend++)
        {
            int remainder = dividend << 8;
            for (int bit = 8; bit > 0; --bit)
                if ((remainder & 0x8000) != 0)
                    remainder = (remainder << 1) ^ polynomial;
                else
                    remainder <<= 1;
            crcTable[dividend] = (short)remainder;
        }
    }

    @Override
    public void update(byte[] buffer, int offset, int len)
    {
        for (int i = 0; i < len; i++)
        {
            int data = buffer[offset+i] ^ (value >>> 8);
            value = (short)(crcTable[data & 0xff] ^ (value << 8));
        }
    }

    /**
     * Updates the current checksum with the specified array of bytes.
     * Equivalent to calling <code>update(buffer, 0, buffer.length)</code>.
     * @param buffer the byte array to update the checksum with
     */
    public void update(byte[] buffer)
    {
        update(buffer, 0, buffer.length);
    }

    @Override
    public void update(int b)
    {
        update(new byte[]{(byte)b}, 0, 1);
    }

    @Override
    public long getValue()
    {
        return value;
    }

    @Override
    public void reset()
    {
        value = init;
    }

    /** Tester */
    public static void  main(String[] args)
    {
        byte[]  data = new byte[]{0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39};
        CRC16   crc16 = new CRC16();
        crc16.update(data,0,data.length);
        System.out.println("CCITT:\t\t\tinit=0xffff poly=0x1021 CRC16(\"123456789\")=0x29b1");
        System.out.println("Calculated:\t\tinit=0x"+Integer.toHexString(INIT_CCITT & 0xffff)+" poly=0x"+Integer.toHexString(POLYNOMIAL_CCITT)+" CRC16(\"123456789\")=0x"+Long.toHexString(crc16.getValue()));
        System.out.println("Test successful:\t"+(crc16.getValue() == 0x29b1));

        String  s = "hello";

        crc16 = new CRC16(POLYNOMIAL_IBM, INIT_CCITT);
        data = new byte[]{0x02, 0x14, 0x14, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
        crc16.update(data, 0, data.length);
        System.out.println("CRC16=0x"+Integer.toHexString((int)crc16.getValue()  & 0xffff));
    }
}
user207421
  • 305,947
  • 44
  • 307
  • 483
  • Thanks for this, it doesn't produce the correct result either, but it got me on the right track I think. – Khantahr Jun 05 '17 at 02:33
  • Well you need to adjust the parameters, as I said, maybe provide your own instead of using the built-in ones. You have to get the right polynomial and initial value, as per the C code you started with. – user207421 Jun 05 '17 at 02:43
  • I'm using the correct parameters, it led me to https://stackoverflow.com/a/28661073/1730307 which shifts the other direction and gives me the correct result for the first two bytes at least... – Khantahr Jun 05 '17 at 02:48
  • Upon further investigation, I've learned that the polynomial used depends on the direction of the shifts. In the avr function, the shifts are to the right, while in this class, they are to the left. Therefore, this implementation must use the reciprocal of the polynomial that the avr function uses. avr uses 0xa001, so this implementation must use 0x8005. – Khantahr Jun 05 '17 at 05:36
  • Finally got it working, the problem was Java treating everything as signed. I changed everything to int and applied the appropriate bitmasks everywhere and it works! – Khantahr Jun 05 '17 at 22:05
  • @Ralgha Not the reciprocal, the byte-swapped value. – user207421 Mar 23 '20 at 09:23