5

I am currently working on a project which requires the code to be Misra 2012 compliant. Throughout the project we have lots of required misra warnings telling us we cant convert pointer to one type to a pointer to another type. Things as simple as void *memcpy(void *to, const void *from, size_t n) produce two Misra Required warnings since both to and from need to be type-casted to void* and const void* respectively. Conversion from void* to a pointer to any other type also gives a Misra warning.

My question is how does Misra expect malloc and everything else to work without any warnings being thrown? Even converting a void* buffer to uint8_t* buffer to parse abuffer byte by byte and fill up all the elements of a structure structure will throw numerous warnings?

Instead of these warnings could it not just show use a note or info asking us to double check packing, alignment and anything else that might go wrong?

Adrian McCarthy
  • 45,555
  • 16
  • 123
  • 175
thunderbird
  • 2,715
  • 5
  • 27
  • 51
  • 2
    Does MISRA forbid implicit conversions, explicit casts, or both? (A terminology note: a cast is an an explicit operator; an implicit conversion is not a cast.) You don't need a cast to call `memcpy` or `malloc`. – Keith Thompson Feb 17 '16 at 19:22
  • What is this MISRA thing? If it doesn't allow you to provide a pointer to something when void* is expected, it is just wrong. – SergeyA Feb 17 '16 at 19:23
  • 4
    _"how does Misra expect malloc and everything else to work without any warnings being thrown?"_ Based on what I could find you're not supposed to use `malloc`, meaning the answer would be _"It doesnt"_. – Michael Feb 17 '16 at 19:26
  • 2
    You can check the official documentation at http://www.misra.org.uk/ Every rule has its rationale. – Ismael Infante Feb 17 '16 at 19:29
  • @KeithThompson both implicit and explicit casts will thorw error. And most of the times the code won't even compile for implicit cast(unless a void* is involved). – thunderbird Feb 17 '16 at 19:35
  • 6
    You should not cast `void *` to/from other pointers in C. C is not C++. And AFIK MISRA does not even allow (or at least strongly discourages) using dynamic memory allocation at all. – too honest for this site Feb 17 '16 at 19:35
  • well memory allocation was just an example. It seems like Misra completely forbids any pointer typecast. – thunderbird Feb 17 '16 at 19:37
  • While you hardly can avoid all casts, it generally is good advise only to use cast if they are really necessary. A properly designed interface does seldom need casts, especially of pointers (another issue is signed/unsigned casting). – too honest for this site Feb 17 '16 at 19:39
  • 1
    There is no such thing as an "implicit cast" in C, nor does the standard use the term "typecast". The word "cast" applies only to the explicit operator, written as a parenthesized type name. Can you update the question with a quotation of the actual requirement? (If implicit pointer conversions are forbidden, I don't see how you could use `malloc` or `memcpy`.) – Keith Thompson Feb 17 '16 at 19:39
  • `to` and `from` do NOT need to be typecast to void, to call memcpy or malloc – M.M Feb 17 '16 at 19:52
  • @M.M ofcourse not but Misra will throw warnings if you do that. – thunderbird Feb 17 '16 at 19:54
  • @KeithThompson yes it seems like malloc and memcpy are not supposed to be used – thunderbird Feb 17 '16 at 19:55
  • What *exactly* does the actual MISRA requirement say? The exact wording could be important. – Keith Thompson Feb 17 '16 at 20:00
  • Wikipedia suggests that MISRA does not allow use of malloc anyway – M.M Feb 17 '16 at 20:28
  • Even Microsoft bans memcpy. https://blogs.microsoft.com/cybertrust/2009/05/14/please-join-me-in-welcoming-memcpy-to-the-sdl-rogues-gallery/ – Adrian McCarthy Feb 18 '16 at 00:25
  • 1
    You should post the specific MISRA rule(s) that your asking about (they're numbered and any warning should provide that number) and you should post an example of the code that generates the rule violation message. Otherwise there's potential for a lot of confusion (as evidenced in the long comment trail...). – Michael Burr Feb 18 '16 at 00:52
  • MISRA-C:2012 is obviously not C++ so I'm removing that tag. – Lundin Feb 18 '16 at 07:29
  • and the reason for deleting the Lint tag was? – Veriloud Feb 19 '16 at 00:19

3 Answers3

11

I would like to go back to what the OP asked and get a few things straight. First of all, there is no problem in calling void *memcpy(void *to, const void *from, size_t n), as a conversion of a pointer to object to a void pointer does not violate any MISRA-C:2012 guideline. In other words, any tool producing violations for that is simply buggy.

Secondly, before coming to any conclusion it is important to read what Rule 11.5, the relevant MISRA-C:2012 guideline, actually says, that is:

  Rule 11.5
  A conversion should not be performed from pointer to void into
  pointer to object

  Category Advisory
  Analysis Decidable, Single Translation Unit
  Applies to C90, C99

  Rationale
  Conversion of a pointer to void into a pointer to object may result
  in a pointer that is not correctly aligned, resulting in undefined
  behaviour. It should be avoided where possible but may be necessary,
  for example when dealing with memory allocation functions. If
  conversion from a pointer to object into a pointer to void is used,
  care should be taken to ensure that any pointers produced do not
  give rise to the undefined behaviour discussed under Rule 11.3.

Observations:

  1. it is an advisory rule (i.e., neither required nor mandatory), so it can be deviated, and MISRA defined the correct deviation process;
  2. converting a pointer to object to a pointer to void is fine: it is the other way around that is problematic;
  3. the rationale explicitly mentions memory allocation functions (and, yes, a program that uses dynamic memory allocation can be made compliant to MISRA-C:2012);
  4. the rationale provides guidance on what to do when converting pointers to objects to pointers to void, perfectly in line with that the OP would like to have ("info asking us to double check packing, alignment and anything else that might go wrong").
5

This does not answer your question, which is about rationales. Rather, it points out that you should not be in this situation in the first place.

Typing "misra malloc" into your favourite search engine leads us to:

http://www.misra.org.uk/forum/viewtopic.php?t=260

which asks:

As per the rule we are not supposed to use functions like malloc(), free(), calloc() etc. But malloc() is a very common requirement. Most of the embedded system applications use their own application level memory managers so as to make the allocation and de-allocation fast. Do you have any suggestions to get around this problem ( if we can't use malloc, any other way )?

And the answer is:

We have been asked about solutions and workarounds for various things that are prohibited in both MISRA C1 and MISRA C2 such as using malloc, calloc, etc. for dynamic memory allocation. Neither MISRA or any member of the MISRA C Working Group will give any guidance or approval to any deviation or "workaround".


You have a requirement that the code comply with a certain standard. You're using mechanisms which don't comply with that standard. Either figure out a principled and robust way to comply, or come up with a clear policy for how to deal with deliberate non-compliance.

You mention memcpy, for example. It's non-compliant. So take a step back and ask "suppose I did not have any implementation of memcpy. How would I write my own memcpy that was compliant?" You're using memcpy to solve a problem; solve the problem without it.

Now do the same for malloc. There was a day when malloc didn't exist, and someone had to write it without malloc. You have a problem that is solved by malloc. If you didn't have malloc, how would you solve it? Nothing in malloc is magical; you can write your own that does a better job than malloc does, where by "better" I mean "complies with your requirements".

Eric Lippert
  • 647,829
  • 179
  • 1,238
  • 2,067
  • well we actually use dynamic allocation only once and then manage it throughout the application life-cycle so we have sort of avoided repeatedly allocating dynamic memory . There are other issues like passing generic inter task messages, parsing data received over a network interface and so on and so forth. Somehow by trying to comply with Misra, I don't get a feeling that we are actually making our code more robust and better and loss out a lot on efficiency. – thunderbird Feb 17 '16 at 19:54
  • 3
    @thunderbird: it is *frequently* the case that compliance with a standard not narrowly tailored to your specific application imposes costs in both effort and efficiency. The question of whether the costs are worth the benefit is for you and the project stakeholders to decide together, based on an informed assessment of both the costs and the benefits. Compliance isn't good for its own sake; it's because there are good outcomes. If you're not getting those good outcomes at a reasonable cost, then push back. – Eric Lippert Feb 17 '16 at 20:00
  • @thunderbird the whole idea of standards like this is to promote correctness over efficiency. Slow and steady beats fast but crashes sometimes. (I doubt you would find any measurable difference in run speed anyway though, e.g. for serializing data in a portable fashion compared to casting char buffers to structs) – M.M Feb 17 '16 at 20:29
  • @M.M, this is certainly not true. There is a **tremendous** performance difference between identity-encoded serialization and any flexible serialization. Seen it with my own eyes. – SergeyA Feb 17 '16 at 20:48
  • ok. Use `-O3`. But identity serialization is a pain anyway because when you change your struct in a later version of code then you have to write a bunch of BS to be able to load earlier versions of the save file, and then there's the case of when the struct contains pointers... – M.M Feb 17 '16 at 20:50
  • 2
    @SergeyA: Right, so there's a good example. If the performance cost exceeds the safety benefit then you can make a principled, data-driven argument that the serialization code should be *isolated* into its own walled-off subsystem that is carefully reviewed, and exempt from your requirements policy. Policies are our servants; make yours work to achieve your goals. – Eric Lippert Feb 17 '16 at 21:33
  • @M.M, well, you can always change your struct by adding fields in the end. And identity serialization has a good place for it's purposes. If you do not trust me, call NASDAQ and ask them why they have Ouch protocol. – SergeyA Feb 17 '16 at 21:36
  • I actually took some time to read the MISRA guidelines. Looks like very specific and very far away from my domain, so I can not possibly comment on those. I am not sure how one can actually write a program without using any form of memory management... but may be it is possible. – SergeyA Feb 17 '16 at 21:41
  • @SergeyA: You can certainly write sophisticated programs without heap management. There are virtual machines, or for that matter, physical machines, that do not have a heap *at all*, and they do useful work. Again: imagine you lived in a world without malloc; what would you do? Memory managers are not that hard to write; I've asked interview candidates to sketch the implementation of one on a whiteboard. – Eric Lippert Feb 17 '16 at 21:49
  • @EricLippert, you can write your own malloc, no question about that. But any version of it will return a void* - this is the only thing generalized memory manager can rerturn. And conversion from void* to the the type pointer is not allowed by Misra, judging from the comments above. Now what? – SergeyA Feb 17 '16 at 21:59
  • 3
    @SergeyA: Who said anything about writing a generalized memory manager? You need a block of ints, **write a memory manager dedicated to making you a block of ints**. Generalized memory managers are a pile of danger; the whole point of this standard is to get away from that danger, not to re-implement it. – Eric Lippert Feb 17 '16 at 22:27
  • @EricLippert, what if I need blocks of ints, doubles, short ints, floats, charcters, structs Foo, Bar and Baz, enums, unsigned ints - do I need that many specialized managers? – SergeyA Feb 17 '16 at 22:30
  • 4
    @SergeyA: Here's your choices: (1) use a language with a proper type system and runtime that ensures memory safety, pay the performance costs associated with them. (2) Use a language with a crap type system that lets you convert any reference to any other reference, write code that uses void* as a proxy for any type, pay the cost of bugs so awful they make your hair fall out, or (3) use a language with a crap type system, eschew void*, write a lot of boring code. I choose (1), but that's because such languages more than meet my needs. – Eric Lippert Feb 17 '16 at 22:34
  • MISRA guidelines are for critical systems (people's lives may be at stake), typically running with limited system resources (limited RAM, no virtual memory), that must run indefinitely. Think engine control computers, medical devices, spacecraft. The prohibition against pointer casting lets you get the most out of type checking system. The prohibition against dynamic memory allocation eliminates the possibility of fragmentation or memory leaks causing a failure while in steady state. – Adrian McCarthy Feb 18 '16 at 00:18
  • @EricLippert, if your choicew is 1), what are you doing in C++ tag? – SergeyA Feb 18 '16 at 14:20
  • 4
    @SergeyA: I freely admit that I am not a C++ expert, having never written even a single C++ compiler; I have only ever written compilers for *other* languages in C++. You really don't know a language until after you've written a couple of compilers for it. That said, I think I may be able to occasionally add some value on the subject. – Eric Lippert Feb 18 '16 at 15:29
  • @Adrian, thats true but there are people who use MISRA-C:2012 in game systems, it's not only critical systems, otherwise they should not be using lint, do you agree? – Veriloud Feb 18 '16 at 16:09
  • @Veriloud: The question is about the rationale behind some of the rules, and the rules were written for critical systems. The official title of the rules is _Guidelines for the Use of the C Language in Critical Systems_. People using these guidelines for things that aren't critical systems is irrelevant to the rationale for the rules. – Adrian McCarthy Feb 18 '16 at 21:20
  • @Adrian, regardless of its title, you do realize the original MISRA-C rules came out of the telecomm industry (a person working at BT)? They are also based on many other safer C standards (e.g. Les Hatton), and of course, Koenig's (of AT&T Bell Labs) Traps and Pitfalls, and: the C standards themselves specific to those areas in the C language that are unpredictable (undefined, unspecific and implementation specific) including folklore rules of the past MISRA stds, which have been downgraded or eliminated - i.e. no, the rationale applies to all c development. – Veriloud Feb 18 '16 at 21:30
  • @Veriloud: This is not the right forum for this conversation. The context of this question is about the rationale of particular MISRA guidelines. The guidelines were written for critical systems. Whether or not it makes sense to apply some of those guidelines in non-critical systems is off topic here. – Adrian McCarthy Feb 18 '16 at 21:38
  • @Adrian, I used to work at the company that originally submitted the guidelines, I might know. BTW, how do you know Thunderbird's context is safety critical (after all, lint is not normally a tool used in this area)? I disagree its off topic, and the considerations as SergeyA provided are also on topic. Many users apply MISRA-C that are not safety critical, this is why MISRA-C:2012 is being more widely adopted, and more of these questions from a non-critical context will be asked in the future. – Veriloud Feb 18 '16 at 21:51
3

MISRA-C:2012 is actually somewhat lax when it comes to pointer conversions. Most rules are sound, concerned with forcing you to follow the C standard, not invoking undefined behavior, or do universally bad things like casting away const qualifiers from a pointer. You shouldn't have any objections against any of that.

The only controversial rule is 11.5:

A conversion should not be performed from a pointer to void into a pointer to object.

I think this one is what causes you head ache. The rationale is alignment concerns and conversions between incompatible pointer types, which would lead to undefined behavior.

That rule does indeed indirectly ban the use of many basic library functions like memcpy. My personal recommendation to MISRA regarding this rule during the 2012 review was:

(Strongly disagree) "There are too many cases of generic C programming when void pointer casts are necessary. This rule isn't practical and will do more harm than good. Instead, make a rule banning the specific danger the rule tries to protect against, namely "pointer-to-x, cast to void*, cast to pointer-to-y"."

But they didn't listen, and there you have it: a useless rule which forces every tool to spew out a flood of false positives. Meaning that every single user of MISRA has to ignore this rule. It is Advisory so you can ignore it without raising any deviation procedure. Just block it in your static analyser and move on.

The MISRA committee has yet to realize that false positives is a safety hazard in itself, since it might lead to people starting to re-write perfectly fine code and thereby introducing bugs.

how does Misra expect malloc

MISRA expects you not to use malloc at all, it is explicitly banned by directive 4.12, for very sound reasons. But that's another topic.

Lundin
  • 195,001
  • 40
  • 254
  • 396
  • @SergeyA With static or local allocation, obviously. ([This question](http://stackoverflow.com/questions/35341893/how-do-i-create-a-module-in-misrac2012-that-follows-dir-4-12-and-4-8) is one such example.) This is nothing unique to MISRA, all safety standards ban the use of dynamic memory allocation on the heap. – Lundin Feb 18 '16 at 14:25
  • How is it different from the 'heap'? How allocating from `static char pool[gazzilion]` using your own allocator is safer than using language-provided, years tested and polished, peer-reviwed by multituted of eyes standard function? – SergeyA Feb 18 '16 at 14:32
  • 2
    @SergeyA Dynamic allocation leads to segmentation, non-deterministic allocation times, memory leaks and so on. It is a 100% superfluous, needlessly bloated feature in any small embedded system. In mission-critical software, dynamic allocation is also completely pointless, since there are no non-deterministic attributes present. Also note that `static type_t pool[n]` is only used for exactly `n` objects of type `type_t` and nothing else. While malloc is completely type/size generic and therefore must use segmentation. – Lundin Feb 18 '16 at 14:37
  • 1
    Hold on a sec. Are you also suggesting to use fixed pool for *every* different type of object I have in my program? So if I use approximately 500 **types** in my app, I end up with 500 pools? Also, how malloc is different in terms of leaks from allocating from said pool? – SergeyA Feb 18 '16 at 14:41
  • 1
    @SergeyA For every ADT you use. Whenever you need to use true private encapsulation in C, which should be the only case when the ADT needs to do the allocation, rather than the caller. `malloc` is different because those who use it for the sake of saving memory must also actively call `free` and that's where you can get bugs. But "saving memory" dynamically doesn't make any sense in single process system, since you don't share memory with anyone. Your heap must always be large enough to fit the worst-case scenario, so they you might as well statically allocate that exact amount of memory. – Lundin Feb 18 '16 at 14:50
  • @SergeyA Think of it the other way: if you have a single process on a system with a certain fixed amount of memory, why would you want to use `malloc`? There exists no reason why you would ever need to do that, [see this](http://electronics.stackexchange.com/questions/171257/realloc-wasting-lots-of-space-in-my-mcu/171581#171581). – Lundin Feb 18 '16 at 14:53
  • Lundin, if you do not free (in any way) resources from statically allocated pools, you might as well not free those you allocate dynamically. What makes it any different? – SergeyA Feb 18 '16 at 15:02
  • 1
    Lundin, I would use malloc because it was tested more times than there are stars in observable part of universe (my wild guess at estimations). And if you do not want to free your resources, do not free them, no one forces you to. `malloc()` is also a function used by other library functions, for example, `strdup`. So, you are banning `strdup` too? – SergeyA Feb 18 '16 at 15:04
  • @Lundin "all safety standards ban the use of dynamic memory allocation on the heap." The word "ban" is inappropriate for any rules within MISRA-C:2012 other than the mandatory ones, there's a reason why the language has changed since prior MISRA standards. Also the “rules” constitute are only one part of the entire document and it's vitally important the background information and process issues described in the leading sections are not ignored...instead of treating engineers like idiots to blindly follow the rules (like the previous MISRA versions). This is why MISRA has such a bad rap. – Veriloud Feb 18 '16 at 15:19
  • 1
    @SergeyA Just read the linked to post, it explains everything. To begin with, every computer in the world is not a RAM-based PC/desktop one... – Lundin Feb 18 '16 at 15:27
  • 5
    @SergeyA: The issue is not whether malloc is well tested. The issue is not whether it reliably performs according to its contract. The issue is that malloc's contract is **terrible to begin with**. Malloc moves the testing burden from making sure that malloc does its job to making sure that the program that uses it is robust **in all possible code paths that could run out of memory**. It is nigh impossible to write robust software for that scenario. – Eric Lippert Feb 18 '16 at 15:36
  • @EricLippert, how does pool allocation solve this problem? Of not running out of memory, I mean. – SergeyA Feb 18 '16 at 15:57
  • @SergeyA: Good question. First, begin by questioning whether a program even needs to dynamically allocate memory. If it doesn't, then you don't need an allocator. Suppose it does. How could we build allocators that have better contracts than malloc? You could build allocators that are much smarter. For example: allocators that know when a block of memory contains results that could be recomputed if necessary, and therefore the block can be invalidated when memory gets low. – Eric Lippert Feb 18 '16 at 16:48
  • @SergeyA: Allocators that provide services beyond merely "here's your memory, or null", like being able to query the state of memory availability, or say what portions of the system are using how many resources, or put bounds on minimum, peak and average expected resource consumption. I could go on and on; you can make allocators that are way, way more powerful and useful than malloc that give you the benefits of type safety and amenability to analysis for both performance and correctness. – Eric Lippert Feb 18 '16 at 16:51
  • 1
    @SergeyA: And to answer your earlier question, of course you get rid of strdup. strdup is horrid. The fact that you need to duplicate strings *at all* is a consequence of the horrible design decision to make strings mutable arrays. The fact that it can arbitrarily fail is awful. If you want to write reliable software then throw all that 40 year old stuff away and build *reliable, fortified subsystems whose correctness can be determined through static analysis*, and whose contracts make the analyzing the correctness of the whole programs tractable. – Eric Lippert Feb 18 '16 at 16:56
  • @EricLippert, ping me when any modern 3-d game engine is written in your glorious new programming language. Or high-frequency trading software. So far all you are saying is wishful thinking. – SergeyA Feb 18 '16 at 17:33
  • 1
    @SergeyA: As I've repeatedly said: standards exist for a reason. The reason that mission-critical reliability standards exist is to ensure reliability in mission-critical software, by definition. The C runtime was not designed to meet those needs. If those are the wrong standards to apply to video games, because those standards impose unacceptable development or performance costs, then push back on the imposition of the standards! – Eric Lippert Feb 18 '16 at 17:36
  • Again, "required" does not mean mandatory, MISRA-C:2012 can be (should be IMHO) applied to non-critical software. Game programming can be "MISRA compliant" if you follow the process activities required, which any decent company would including rigorous peer review and documenting the reasons for not blindly tools/guidelines. As I said in my answer, the standard alludes to the practical world SergyA refers to, Dir. 4.12, pg. 34, "If a decision is made to use dynamic memory, care shall be taken to ensure that the software behaves in a predictable manner..." and then provides examples. – Veriloud Feb 18 '16 at 18:34
  • "every single user of MISRA has to ignore this rule" [citation needed] I'm aware of software groups that do not ignore this rule. – Adrian McCarthy Feb 18 '16 at 21:23
  • The rule is too broad, and because of the signal to noise factor developers turn off the rule in the tool...however, if the rule was more specific (as he suggested) it would do less harm than good...this is an issue of configuring the tool, also Lundin is right, this specific rule requires no deviation, but might have if they took his suggestion. – Veriloud Feb 18 '16 at 22:04
  • @AdrianMcCarthy It is my own belief. I really don't see how it would be practical to write programs that can't contain NULL, memcpy/memset/memcmp, generic programming (functions like bsearch, qsort) etc etc. I just now looked at the previous large project I made and checked what removing `void*` would mean. Most notably, it would mean that I would have to drop my std library which contains MISRA conforming equivalents of most common standard functions (the library ignores advisory rule 11.5) and instead use the compiler's non-compliant std lib, which is neither verified, safe nor portable. – Lundin Feb 19 '16 at 07:22