Decoding endian(?) data from a sensor

Question

I am not sure I even understand what is going on here with this data, but I am trying to replicate functionality like here, here or here to decode the data I am receiving over UART from my Plantower PMS5003 sensor (see datasheet) sensor in Elixir.

It's delimited by 0x42 and 0x4d and starts like this:

iex(mygadget@nerves.local)4> {:ok, data} = Circuits.UART.read(pid, 60000)
{:ok,
 <<66, 77, 0, 28, 0, 23, 0, 32, 0, 32, 0, 22, 0, 31, 0, 32, 17, 124, 4, 211, 0,
   171, 0, 8, 0, 0, 0, 0, 151, 0, 4, 5, 66, 77, 0, 28, 0, 23, 0, 32, 0, 32, 0,
   22, 0, 31, 0, 32, ...>>}

I then base16 encode it:

iex(mygadget@nerves.local)5> Base.encode16(data)
"424D001C0017002000200016001F0020117C04D300AB00080000000097000405424D001C0017002000200016001F0020117C04D300AB00080000000097000405424D001C0017001F001F0016001E001F115804BE0098000800
000000970003B5424D001C0018002000200016001F002011BB04D8009F0008000000009700043E424D001C0016001F001F0015001E001F11DC04C3009300080000000097000437424D001C0017001E001E0015001D001E11E20
4C300850008000000009700042C424D001C0016001E001E0015001D001E117304B70087000600000000970003B0424D001C0016001D001D0015001D001D111F049B007B00060000000097000331424D001C0017001E001E0016
001E001E10F5048D007D00060000000097000400424D001C0017001E001E0016001E001E10FB0496008B0004000000009700041B424D001C0016001E001E0015001E001E10B304810089000400000000970003BA424D001C001
5001C001C0014001C001C104A045E008000020000000097000319424D001C0016001C001

And split by 424D

decoded |> String.split("424D")
["", "001C0017002000200016001F0020117C04D300AB00080000000097000405",
 "001C0017002000200016001F0020117C04D300AB00080000000097000405",
 "001C0017001F001F0016001E001F115804BE0098000800000000970003B5",
 "001C0018002000200016001F002011BB04D8009F0008000000009700043E",

Then break it into chunks of 2

iex(mygadget@nerves.local)10> "001C0017002000200016001F0020117C04D300AB00080000000097000405" |> String.codepoints |> Enum.chunk(2) |> Enum.map(&Enum.join/1)
["00", "1C", "00", "17", "00", "20", "00", "20", "00", "16", "00", "1F", "00",
 "20", "11", "7C", "04", "D3", "00", "AB", "00", "08", "00", "00", "00", "00",
 "97", "00", "04", "05"]

I am fairly at a loss as to where to go from here. I found this discussion about how to do it in Java but I don't really understand what is going on there with the framebuffers.

Any insight appreciated

EDIT: tags

Answer 1

So, erlang/elixir are great languages for deconstructing raw packets in binary format--which is what Circuits.UART.read() returns. You deconstruct a binary, <<...>>, with binary pattern matching, and your datasheet contains the spec for the pattern you will use. There's no need for base16 encoding, splitting at 424D, nor breaking into chunks of 2:

defmodule My do

  def match(<<     
                66, 77, 
                _fr_len::big-integer-size(16),
                data1::big-integer-size(16),
                data2::big-integer-size(16),
                data3::big-integer-size(16),
                data4::big-integer-size(16),
                data5::big-integer-size(16),
                data6::big-integer-size(16),
                data7::big-integer-size(16),
                data8::big-integer-size(16),
                data9::big-integer-size(16),
                data10::big-integer-size(16),
                data11::big-integer-size(16),
                data12::big-integer-size(16),
                _reserved::big-integer-size(16),
                _check_code::big-integer-size(16),
                rest::binary
           >>) do

    IO.puts "pm 1.0 cf: #{data1} ug/m^3"
    IO.puts "pm 2.5 atmospheric: #{data5} ug/m^3"
    match(rest)
  end

  def match(partial_frame) do
    IO.puts "partial_frame:"
    IO.inspect partial_frame 
  end

  def go() do
    match(<<66, 77, 0, 28, 0, 23, 0, 32, 0, 32, 0, 22, 0, 31, 0, 32,  
           17, 124, 4, 211, 0, 171, 0, 8, 0, 0, 0, 0, 151, 0, 4, 5,  
           66, 77, 0, 28, 0, 23, 0, 32, 0, 32, 0, 22, 0, 31, 0, 32>>)

    :ok
  end


end

In iex:

iex(1)> My.go
pm 1.0 cf: 23 ug/m^3
pm 2.5 atmospheric: 31 ug/m^3
partial_frame:
<<66, 77, 0, 28, 0, 23, 0, 32, 0, 32, 0, 22, 0, 31, 0, 32>>
:ok

You could write 0x42, 0x4D inside the pattern to exactly match the datasheet spec, however I think it's clearer to use the decimal equivalents, 66, 77, because elixir doesn't output hex codes when outputting a binary, rather elixir outputs decimals--as can be seen in your data (Or, sometimes elixir outputs a double quoted string for a binary, which is really confusing and stupid.) With 66, 77 in the pattern, you can easily look at the data and see where it matches.

Note that the last segment rest::binary is like writing .* in a regex.

Before trusting any of the data assigned to the variables, you should probably check that the frame length is 28 and verify the check code. Unfortunately, I can't figure out what the check code represents. I get 1029 for the check code.

==========

Can you post an example of what you expect the data to look like?

A hex string like "1C" is equivalent to decimal 28. You can get all the decimal equivalents like this:

data = [ "00", "1C", "00", "17", "00", "20", "00", "20", "00", "16", "00",   
        "1F", "00", "20", "11", "7C", "04", "D3", "00", "AB", "00", "08",  
        "00", "00", "00", "00", "97", "00", "04", "05"]

for str <- data do
  Integer.parse(str, 16)
end
|> IO.inspect
|> Enum.map(fn {a, _} -> a end)

Integer.parse() returns a tuple, where the first element is an integer and the second element is the "the remainder of the string", i.e. anything that couldn't be interpreted as an integer.

Output:

[
  {0, ""},
  {28, ""},
  {0, ""},
  {23, ""},
  {0, ""},
  {32, ""},
  {0, ""},
  {32, ""},
  {0, ""},
  {22, ""},
  {0, ""},
  {31, ""},
  {0, ""},
  {32, ""},
  {17, ""},
  {124, ""},
  {4, ""},
  {211, ""},
  {0, ""},
  {171, ""},
  {0, ""},
  {8, ""},
  {0, ""},
  {0, ""},
  {0, ""},
  {0, ""},
  {151, ""},
  {0, ""},
  {4, ""},
  {5, ""}
]
[0, 28, 0, 23, 0, 32, 0, 32, 0, 22, 0, 31, 0, 32, 17, 124,  
 4, 211, 0, 171, 0, 8, 0, 0, 0, 0, 151, 0, 4, 5]

Is that what your data is supposed to look like?

It looks to me like the java code:

...forEach[b | bts.append(Integer.toHexString(b)]

does some bit twiddling with the java's bitwise OR operator: |, which makes no sense to me in that code snippet. But, in elixir you would do that with Bitwise.bor(a, b). I really think that java code should look like this:

...forEach(b -> bts.append(Integer.toHexString(b))

In other words, forEach() takes a lambda as an argument. Heck, that has me so curious, I'm going to ask the guy what that means.

Edit:

Alright, the guy responded to me, and that isn't Java--it is the syntax for a lambda though--in some DSL language.