Building AIS Messages Decoder

Question

I used to decode AIS messages with theis package (Python) https://github.com/schwehr/noaadata/tree/master/ais until I started getting a new format of the messages. As you may know, AIS messages come in two types mostly. one part (one message) or two parts (multi message). Message#5 is always comes in two parts. example:

!AIVDM,2,1,1,A,55?MbV02;H;s<HtKR20EHE:address@hidden@Dn2222222216L961O5Gf0NSQEp6ClRp8,0*1C
!AIVDM,2,2,1,A,88888888880,2*25

I used to decode this just fine using the following piece of code:

   nmeamsg = fields.split(',')
   if nmeamsg[0] != '!AIVDM': 
    return
   total = eval(nmeamsg[1])
   part = eval(nmeamsg[2])
   aismsg = nmeamsg[5]
   nmeastring = string.join(nmeamsg[0:-1],',')


   bv = binary.ais6tobitvec(aismsg)
   msgnum = int(bv[0:6])

--

elif (total>1):
     # Multi Slot Messages: 5,6,8,12,14,17,19,20?,21,24,26
     global multimsg
     if total==2:
       if msgnum==5:
         if nmeastring.count('!AIVDM')==2 and len(nmeamsg)==13: # make sure there are two parts concatenated together
           aismsg = nmeamsg[5]+nmeamsg[11]
           bv = binary.ais6tobitvec(aismsg)    

           msg5 = ais_msg_5.decode(bv)
           print "message5 :",msg5
           return msg5

Now I'm getting a new format of the messages:

!SAVDM,2,1,7,A,55@0hd01sq`pQ3W?O81L5@E:1=0U8U@000000016000006H0004m8523k@Dp,0*2A,1410825672
!SAVDM,2,2,7,A,4hC`2U@C`40,2*76,1410825672,1410825673

Note. the number at the last index is the time in epoch format

I tried to adjust my code to decode this new format. I succeed in decoding messages with one part. My problem is multi message type.

   nmeamsg = fields.split(',')
   if nmeamsg[0] != '!AIVDM' and nmeamsg[0] != '!SAVDM': 
    return
   total = eval(nmeamsg[1])
   part = eval(nmeamsg[2])
   aismsg = nmeamsg[5]
   nmeastring = string.join(nmeamsg[0:-1],',')
   dbtimestring = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(float(nmeamsg[7])))

   bv = binary.ais6tobitvec(aismsg)
   msgnum = int(bv[0:6])

Decoder can't bring the two lines as one. So decoding fails because message#5 should contain two strings not one. The error i get is in these lines:

if nmeastring.count('!SAVDM')==2 and len(nmeamsg)==13: 
aismsg = nmeamsg[5]+nmeamsg[11]

Where len(nmeamsg) is always 8 (second line) and nmeastring.count('!SAVDM') is always 1

I hope I explained this clearly so someone can let me know what I'm missing here.

UPDATE

Okay I think I found the reason. I pass messages from file to script line by line:

for line in file:
    i=i+1

    try:
        doais(line)

Where message#5 should be passed as two lines. Any idea on how can I accomplish that?

UPDATE

I did it by modifying the code a little bit:

for line in file:
    i=i+1

try:
        nmeamsg = line.split(',')
        aismsg = nmeamsg[5]
        bv = binary.ais6tobitvec(aismsg)
        msgnum = int(bv[0:6])
        print msgnum
        if nmeamsg[0] != '!AIVDM' and nmeamsg[0] != '!SAVDM': 
         print "wrong format"
        total = eval(nmeamsg[1])
        if total == 1:
         dbtimestring = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(float(nmeamsg[8])))
         doais(line,msgnum,dbtimestring,aismsg)
        if total == 2: #Multi-line messages
         lines= line+file.next()
         nmeamsg = lines.split(',')
         dbtimestring = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(float(nmeamsg[15])))
         aismsg = nmeamsg[5]+nmeamsg[12]
         doais(lines,msgnum,dbtimestring,aismsg)

Answer 1

Be aware that noaadata is my old research code. libais is my production library thst is in use for NOAA's ERMA and WhaleAlert.

I usually make decoding a two pass process. First join multi-line messages. I refer to this as normalization (ais_normalize.py). You have several issues in this step. First the two component lines have different timestamps on the right of the second string. By the USCG old metadata standard, the last one matters. So my code will assume that these two lines are not related. Second, you don't have the required station id field.

Where are you getting the SA from in SAVDM? What device ("talker" in the NMEA vocab) is receiving these messages?

Answer 2

If you're in Ruby, I can recommend the NMEA and AIS decoder ruby gem that I wrote, available on github. It's based on the unofficial AIS spec at catb.org which is maintained by one of Kurt's colleagues.

It handles combining of multipart messages, reads from streams, and supports a large of NMEA and AIS messages. Decoding the 50 binary subtypes of AIS messages 6 and 8 is presently in development.

To handle the nonstandard lines you posted:

!SAVDM,2,1,7,A,55@0hd01sq`pQ3W?O81L5@E:1=0U8U@000000016000006H0004m8523k@Dp,0*2A,1410825672
!SAVDM,2,2,7,A,4hC`2U@C`40,2*76,1410825672,1410825673

It would be necessary to add a new parse rule that accepts fields after the checksum, but aside from that it should go smoothly. In other words, you'd copy the parser line here:

    | BANG DATA CSUM  { result = NMEAPlus::AISMessageFactory.create(val[0], val[1], val[2]) }

and have something like

    | BANG DATA CSUM COMMA DATA { result = NMEAPlus::AISMessageFactory.create(val[0], val[1], val[2], val[4]) }

What do you do with those extra timestamp(s)? It almost looks like they've been appended by whatever software is doing the logging, rather than being part of the actual message.