include binary data files into compiled binary without converting to hex arrays

Question

I regularly want to include (compile in) binary data from external files as an uint8_t array into my programs (embedded or bare metal).

Is there a way without the extra step of converting them into a hex arrays? Because this involves extra build steps that sometimes are not that easy to integrate e.g. in case of Arduino tool chain. And doing this manually always results in forgotten hex-updates :-)

Normally I am using gcc. So I also would be happy if it is a gcc-specific pragma or another extension (although not that happy as with an generic solution).

maybe something like

static const __attribute__((rawDataFromFile("myDataFile.bin"))) uint8_t s_data[];

instead of the classical way to external
bin2hex myDataFile.bin myDataFile.hex
and

static const uint8_t s_data[] = {
  #include "myDataFile.hex"
};

Solutions that work in C and in C++ would be preferred, but suggestions that only work in one of the two are also welcome.

Not that I'm aware of. I'm interested as well, because I normally just use `xxd -i filename.bin` for my generations, which at least has the advantage of easily incorporating a dependency rule into a makefile. — WhozCraig, Nov 12 '19 at 09:58
There is a proposal to add `std::embed` for a standardised way to embed arbitrary data in source files, without having to manually transcribe them into string literals or hex: [p1040R0](http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1040r0.html) — underscore_d, Nov 12 '19 at 10:55

Evg · Answer 1 · 2019-11-12T11:12:05.627

6

An old trick, which I used some 20 years ago with Turbo Pascal, of converting a binary file into an object one, came to my mind. Here it is with C++:

Data file data.dat:
```
ABC
```
Convert into an object file:
```
ld -r -b binary data.dat -o data.o
```
As explained here, with -b binary <name> option the object file will contain raw binary data from the file <name> in the .data section, and three symbols _binary_<name>_start, _binary_<name>_end, and _binary_<name>_size will be created.

Simple program:

#include <cstdint>
#include <iostream>

extern const std::uint8_t _binary_data_dat_start[];
extern const std::uint8_t _binary_data_dat_end[];

int main()
{
    for (auto d = _binary_data_dat_start; d != _binary_data_dat_end; ++d)
        std::cout << (char)*d << ' ';
}

Compilation and linking:
```
g++ -c test.cpp
g++ test.o data.o
```
Output:
```
A B C
```

Edit 1

that also introduces external preprocessing of the data and even introduces the need to know how the symbols will be named

Unfortunately, that's true. You might want to look at this library, which uses inline assembly to embed binary data.

Edit 2 Found a possible duplicate question.

edited Nov 12 '19 at 11:12

answered Nov 12 '19 at 10:05

Evg

25,259
5
41
83

+1 however it would be valuable to at least have the comment about the auto-generation of symbols in step 2, and what they'll look like in their program in step 3 – Florian Humblot Nov 12 '19 at 10:14
@FMashiro, will do, thanks. – Evg Nov 12 '19 at 10:16
1

thanks for answer, but that also introduces external preprocessing of the data and even introduces the need to know how the symbols will be named. – vlad_tepesch Nov 12 '19 at 10:22
@vlad_tepesch, what about [this library](https://github.com/graphitemaster/incbin)? – Evg Nov 12 '19 at 10:51
great, that did not came up on my search before. that library looks good on the first glance. I will have to double check if it is compatible with different target platform, since it uses inline assembly. – vlad_tepesch Nov 12 '19 at 11:23

score 0 · Answer 2 · answered Nov 12 '19 at 10:32

0

use ld

ld -r -b binary binaryfile -o objectfile 
objcopy --rename-section .data=.yoursection objectfile

answered Nov 12 '19 at 10:32

0___________

60,014
4
34
74

include binary data files into compiled binary without converting to hex arrays

2 Answers2