0

I recently started using this UART library to transfer data. The test case works, but the library is not exactly a breeze to read through (at least for me).

My goal is to store data in a struct on the receiving micro controller from a struct on the transferring controller. I thought I was close on my last implementation, but it was going nowhere.

I'd like to avoid making the same mistake as last time. Here's the pseudo-code for my protocol. Does it look like I am neglecting anything?

Transfer

    1) Instance struct
    2) Point at its memory location
    3) For each variable in struct, write its data to the register

Receive

    1) Instance struct
    2) Point at its memory location
    3) When the register buffer is full, read register and store to struct by index
Community
  • 1
  • 1
Womble
  • 345
  • 3
  • 13
  • 1
    Do the structs contain pointers? – Ian Abbott Apr 15 '16 at 15:44
  • 3
    Other possible snags: the struct might not be the same size on each machine; and endianness. – Weather Vane Apr 15 '16 at 15:46
  • @IanAbbott The structs themselves do not, should they? – Womble Apr 15 '16 at 15:57
  • @WeatherVane It is the same MC and I will be flashing them with the same struct - so I expect this wont be an issue, correct? – Womble Apr 15 '16 at 15:57
  • @SensationSama are you asking a new question, or directing people to answer better the previous one? If the library test case works, why not rewind and build on that, tiny step by tiny step? – Weather Vane Apr 15 '16 at 16:00
  • @WeatherVane This is a new question because I am dealing with an entirely different library. I wish to avoid confusion with old content, and I don't expect a response in the previous thread. With that being said many of the same questions remain, don't they? I am trying to build on this library, and I am here seeking advice in doing so. – Womble Apr 15 '16 at 16:07
  • Sorry, but I am not sure what the pseudo code means by "iterate through struct" and "when bits are set". It is not clear what you are asking, or whether the code is the same as in your previous question link. – Weather Vane Apr 15 '16 at 16:10
  • 2
    Build up a string representation of your struct instances and send them through RS232 (or whatever). Parse the string representation on the receiving site and rebuild your struct instance. – knivil Apr 15 '16 at 16:17
  • Go to useraproperserialprotocol.com – Martin James Apr 15 '16 at 16:20
  • @WeatherVane I've attempted to clarify the pseudo code above, but yes – the code would be similar, as that is how I tried to build it previously. Of course it would include whatever is necessary to work with this library, but I'm trying to identify the proper steps to build it. – Womble Apr 15 '16 at 16:20
  • @knivil There's a thought. I would like to avoid this if possible, though. I might be wrong but this solution might be a bit memory wasteful and needlessly complex? – Womble Apr 15 '16 at 16:27
  • @MartinJames Pretty sure this library is proper, please elaborate. Also if you're trying to be funny, rude. – Womble Apr 15 '16 at 16:28
  • @SensationSama early on in my career I asked a client how he wanted the data filed. Without hesitation, he said "as text". If you want data transferred to another system, it has to be portable. – Weather Vane Apr 15 '16 at 16:31
  • 1
    @SensationSama I'm trying to stop you doing anyting like 'For each variable in struct, write its data to the register', as is Weather Vane. You are going to transfer data from one system to another over a byte stream. Byte stream, not struct stream or string stream. You NEED to precisely define what you are going to send at the level of single bytes, including an unambiguous message start and message data, eg. [4 bytes of int] will not work because of endianness issues. – Martin James Apr 15 '16 at 17:15

1 Answers1

4

Serialization is something you would think is quite simple at first glance, but as some of the commenters have noted there are quite a few details that need to be taken into consideration. Endianness is important, but you also need to know how the compiler is organizing the data inside of your structs. Compilers tend to try to align your data in memory in a way that minimizes the time it takes to read/write the data. 16/32/64-bit processors will have machine instructions that access several bytes of data at once (such as for adding two 32-bit ints, etc), and these will require that the memory being accessed is aligned in a way that makes sense. Instructions accessing bytes can use any address, instructions accessing 16-bit words require that they are located at even addresses, instructions accessing 32-bit double-words require that they are located at addresses divisible by 4, and so on. Add to this the fact that many of the standard types in C can have different widths on different platforms and you can begin to appreciate the breadth of this topic.

A the very least you're really going to need to define a rudimentary protocol that details precisely how the struct in question is going to be serialized. Different micros and/or compilers can store data in different ways, and if you want your code to be even slightly portable you will want to avoid just shifting your data into a buffer as is. You mentioned in a comment that the transmitting and receiving devices will be using the same microcontroller, so in principle you could get away with a naive implementation. But as I see it you will already be doing some work to implement the naive serialization, so you might as well take the time to get a well-defined serialization. Odds are it will save you from a headache later if you want to verify that your bits are all getting to their proper locations. It will also save you from transmitting a potentially large number of unused padding bits (see 2nd link below).

You should have a look at this SO question about serialization: (C - serialization techniques ). There are a few different answers, but the accepted answer is pretty much what I would suggest. As long as you know the size of your structs you could get by without the dynamic memory allocation for the buffer, but having a dedicated "serialize_...()" function for each data object is a good route to take.

I also recommend you read through this document: (http://www.catb.org/esr/structure-packing/ ). It is aimed at showing how to use struct packing to save memory, but it is also a great source for understanding how to serialize data in C (the subjects are strongly related).

A few years ago I had to roll my own tiny serial protocol to enable some 8-bit PICs talk to each other via GPIOs, and I found it to be fun to go through and define starting/stopping conditions and a simple data protocol. A fairly common approach is to have starting/stopping bytes for your packet, and then you describe your data in a consistent format, such as:

[start byte] [type0] [length0] [data0] [type1] [length1] [data1] ... [typeN] [lengthN] [dataN] [end byte]

The above format is a more general/flexible approach, and some advanced protocols allow for the description of large data structures using a similar format. But in our simple case with only one possible message we can assume that the transmitting device and receiving device agree on the order of each "object" in the message and that they also know the type and length in bytes of each "object", so you don't need to transmit this data. If you want to transfer one of several possible structs you could use a single [type] byte to communicate which struct is being sent. You really have a lot of options, so have fun with it. You get bonus points for using a CRC to make sure your data got across uncorrupted.

I apologize for the lack of fancy formatting; I'm a newbie to actually answering SO questions.

Community
  • 1
  • 1
DrEsperanto
  • 167
  • 9
  • Well, it gets an upvote from me, anyway, for explaining the pile of stuff that I could not be bothered to:) – Martin James Apr 17 '16 at 20:21
  • @MartinJames Thanks! You guys already gave Sensation the answer in the comments, but I figured he/she could use a bit more explanation. I also needed to get some rep so I can upvote and comment, so I figure it's a win-win. BTW, I like your picture :) – DrEsperanto Apr 18 '16 at 12:34