0

I'm trying to make a program for chess and in most cases I would need only 4, 6 or 8(here I can use char) bytes. So can I create a type that use 4 bytes, or an array with 4 bytes per cases ? It would lead to an significant gain in memory (and in efficiency ?).

Thanks all.

moi
  • 467
  • 4
  • 19
juste_thomas
  • 1
  • 4
    _"I would need only 4, 6 or 8(here I can use char) bytes"_ - `char` is one byte, so you could use that for everything you listed here. Do you really mean byte or maybe bit? – Lukas-T Sep 11 '21 at 11:21
  • *"It would lead to an significant gain in memory"* I assume you're trying to implement the AI and store states. Using a few more bytes each of the 32 pieces is an absolutely insignificant amount for a modern PC, so it shouldn't be an issue to store a single state that takes a few more bytes. As for allocating arbitrary number of bytes: as long as the number is non-zero there's absolutely no issue with this except for the fact that effectively you'll consume 8 or more bytes effectively. – fabian Sep 11 '21 at 11:31
  • 2
    Whether you'll gain a benefit by improving memory consumption depends on many factors though: If there is sufficient main memory available optimizing the data for the cache may yield much better results, but this could be exactly the same as memory consumption. Btw: if there may be an issue with too many small memory allocations you may be able to solve this by allocating an and using parts of the array for data belonging to different entities... – fabian Sep 11 '21 at 11:34
  • Generally, for such cases, you should simply use a structure with data members in decreasing size order to usually get the smallest data with smallest alignment. For example `struct small { unsigned short a; unsigned short b; unsigned char c; unsigned char d; };` will probably be 6 or 8 bytes and realtively efficient. If more compression is need, then it is more a **case by case analysis** instead of some general rules. – Phil1970 Sep 11 '21 at 12:52
  • Where is your current data definition? When asking a question, you should almost always **provide some code** that show what you have done. – Phil1970 Sep 11 '21 at 13:17
  • Please provide enough code so others can better understand or reproduce the problem. – Community Sep 15 '21 at 09:23

1 Answers1

0

If you like, you can allocate a big buffer and manage the memory yourself.

If you do so, make sure not to use structs, or if you do, learn about "byte padding" or "byte alignment", since it will make each struct occupy more space than it "needs" to.

Once you have raw allocated memory (with malloc, std::vector or using an array, for example), you need to "tightly store" your data.

Here's some code:

char array[5000] = { 0 }; // or char* array = malloc(5000);
// Let's say you want to store several "char, int" structs

const int structSize = sizeof(char) + sizeof(int);

*(char*)&array[0 * structSize] = 'a';
*(int*)&array[0 * structSize + sizeof(char)] = 1;

*(char*)&array[1 * structSize] = 'b';
*(int*)&array[1 * structSize + sizeof(char)] = 2;

*(char*)&array[2 * structSize] = 'c';
*(int*)&array[2 * structSize + sizeof(char)] = 3;

WARNING: In the code above, if you are storing structs/classes rather than raw types (char, int...), make sure to use "placement new" on the memory location.

If, instead, you use a "struct { char, int }", its size will be 8 bytes (on most computers, not all), 3 of which are padding (unused memory).

Please note that, while you gain in memory usage, you lose in cpu efficiency (that is what the point of padding is).

Alternatively, see this article for alternatives that tell the compiler not to do the padding (using "#pragma pack").

#pragma pack(1)
struct nopadding
{
    char first;
    int second;
};

sizeof(nopadding); // <- is now equal to 5 instead of 8
ZeroZ30o
  • 375
  • 2
  • 18
  • 1
    In C++ `operator new` is generally allocating memory. Also [valgrind](https://valgrind.org/) could be a useful tool. Compile code with a recent [GCC](https://gcc.gnu.org/) invoked as `g++ -Wall -Wextra -O -g` – Basile Starynkevitch Sep 11 '21 at 11:43
  • 1
    What's the advantage of using `malloc` and `emplacement new` instead of just `std::vector foo(400)`? That's 400 instances of a 6-byte structure, with automatic construction, destruction and memory management. – Lukas-T Sep 11 '21 at 11:47
  • @churill I believe using the vector will automatically add byte aligning. I am not sure of this as I have not tested it, though. And @ Basile "placement new" does not allocate any memory, it simply calls the constructor on an existing piece of memory. – ZeroZ30o Sep 11 '21 at 11:49
  • Any memory allocation will align memory and round up allocation to some value (and might have hidden data). Only the first sentence on your anwer is somewhat useful. – Phil1970 Sep 11 '21 at 12:45
  • @Phil1970 You're right mostly right - I didn't consider that it was the struct themselves doing the byte alignment. However, more than the first sentence is useful, as placing the memory yourself (to avoid padding) is indeed more efficient memory usage (less CPU-efficient, though). I will edit the post to reflect this. – ZeroZ30o Sep 11 '21 at 13:00
  • 3
    Unaligned memory access are not portable. And if one does not care about protability, then it can probably use specific attributes, pragma or compiler flag anyway to get the desired alignment from its `struct`. I would not recommand any hack from a general question (without any code). – Phil1970 Sep 11 '21 at 13:15
  • @Phil1970 Unfortunate timing as I had just edited the question to add those options in the linked article. However I would not call this a "hack", but I fail to see how it is not portable - it is a normal usage of arrays with some casting. I do agree though, that it would add more complexity than structs with the compiler option. – ZeroZ30o Sep 11 '21 at 13:20
  • 1
    Even ignoring unaligned access, failure to call placement-new for those numbers causes undefined behavior, even if it might work in practice. (Starting with C++20 you can get away with `std::launder` instead.) – HolyBlackCat Sep 11 '21 at 13:20
  • @HolyBlackCat Can you explain why? I was under the assumption that "native" data types such as char and int do not need to be constructed. – ZeroZ30o Sep 11 '21 at 13:22
  • 2
    @ZeroZ30o Placement-new for them is indeed a no-op, but the standard still requires it, see [`[basic.life]`](http://eel.is/c++draft/basic.life) and [this cppreference article](https://en.cppreference.com/w/cpp/language/lifetime). – HolyBlackCat Sep 11 '21 at 13:31
  • @HolyBlackCat I understand using placement new for the raw byte array if you were creating it with templates or general structs in it, but I do not subscribe to the idea of following paradigms (in this case, constructors/destructors) at any benefit/cost ratio. I will, however, edit the post to mention placement new in case custom structs are used. – ZeroZ30o Sep 11 '21 at 13:36
  • 1
    Even if this violation doesn't cause problems on modern compilers (which I'm not sure about, I'd have to test it), this is somewhat of a slippery slope. Even if you're careful, from time to time you'll get elusive, hard-to-debug crashes. Even if they're not caused by this specific violation, you're not going to know it, and will have to check every single "technically-UB-but-works" trick in the problematic code. :| – HolyBlackCat Sep 11 '21 at 13:50
  • @HolyBlackCat I simply disagree with the classification. I fail to see how a char or int would EVER need a constructor / destructor on already-allocated memory - to me, it feels like trying to fit the generic paradigm to something that does not need it at all. – ZeroZ30o Sep 11 '21 at 14:09
  • 1
    @ZeroZ30o The two links I provided should explain that this is UB, regardless of whether the type is "simple" or not. You're writing to an `int` before its lifetime has started, even though the placement-new that would start the lifetime would be a no-op. Compilers can (at least in theory) optimize based on the assumption that you don't violate this rule. – HolyBlackCat Sep 11 '21 at 14:33
  • 1
    Another useful link: [What is the strict aliasing rule?](https://stackoverflow.com/questions/98650/what-is-the-strict-aliasing-rule). – HolyBlackCat Sep 11 '21 at 14:36