21

The What's New for Delphi XE2 contains the following.

Packed Now Forces Byte Alignment of Records

If you have legacy code that uses the packed record type and you want to link with an external DLL or with C++, you need to remove the word "packed" from your code. The packed keyword now forces byte alignment, whereas in the past it did not necessarily do this. The behavior change is related to C++ alignment compatibility changes in Delphi 2009.

I don't understand this. I'm struggling with this point: whereas in the past it did not necessarily do this. What I cannot reconcile is that packed has always resulted in byte alignment of records to the best of my knowledge. Can anyone give an example of a packed record that is not byte aligned? Obviously this would have to be in an earlier version.

Why do the docs say "if you want to link with an external DLL or with C++, you need to remove the the word packed from your code"? If the external code uses #pragma pack(1) then what are we to do if packed is off limits?

What about the $ALIGN directive? Are {$A1} and {$A-} equivalent to packed or is there some extra meaning with packed?

It seems that I'm missing something and would appreciate it if somebody could explain this. Or is the documentation just really poor?

Update

I'm reasonably convinced that the documentation is referring to alignment of the record itself rather than the layout of the record. Here's a little program that shows that the the use of packed on a record forces the alignment of the record to be 1.

program PackedRecords;
{$APPTYPE CONSOLE}
type
  TPackedRecord = packed record
    I: Int64;
  end;

  TPackedContainer = record
    B: Byte;
    R: TPackedRecord;
  end;

  TRecord = record
    I: Int64;
  end;

  TContainer = record
    B: Byte;
    R: TRecord;
  end;

var
  pc: TPackedContainer;
  c: TContainer;

begin
  Writeln(NativeInt(@pc.R)-NativeInt(@pc.B));//outputs 1
  Writeln(NativeInt(@c.R)-NativeInt(@c.B));//outputs 8
  Readln;
end.

This produces the same output on Delphi 6, 2010, XE and XE2 32 bit and XE 64 bit.

David Heffernan
  • 601,492
  • 42
  • 1,072
  • 1,490
  • 6
    Keep in mind that there are two alignments in play: the alignment of the fields within the record (which is what packed affects) and the alignment of the record itself, say, in an array of these records. Historically, I believe packed did not affect the alignment of the record itself and that was later changed, or the other way around. – dthorpe Dec 11 '11 at 01:02
  • Desperate googling revealed your question and the article it referred to. I don't get it either, I'm all agog for an example. Unpacked array in a packed record, or may be a variant record? On my linux box at the moment otherwise I'd be intrigued enough to experiment. – Tony Hopkinson Dec 11 '11 at 01:26
  • @dthorpe Hi Danny. I'm aware of the difference between layout and alignment. The Delphi docs nowadays document that packed records have alignment 1. But my experience is that this has always been so. It was the case in D6 for sure. So packed does affect both layout and alignment. Are you saying that if you go far enough back, i.e. D1 say, that packed only affected layout? – David Heffernan Dec 11 '11 at 07:42
  • I recall that it came up as an issue in Kylix that required quite a bit of internal discussion, but I don't recall the details of which way things went. – dthorpe Dec 12 '11 at 17:11

4 Answers4

5

The latest updates to the documentation have removed all of the text on which this question was based. My conclusion is that the original text was simply a documentation error.

David Heffernan
  • 601,492
  • 42
  • 1,072
  • 1,490
2

As I am not the Delphi compiler guy I can also mostly guess as others did: The record alignment might not be meant for aligning the members inside the record, but the record itself instead.

If you declare a record variable it is aligned to some address in memory, that is most likely aligned on a 4-byte boundary. This has been the case (as tested in D2007) for packed and unpacked records.

Now in XE2 a packed record is placed in a 1-byte boundary, while unpacked records are placed on some even boundary, which can be controlled by the align keyword. Like this:

type
  TRecAligned = record
    b1: byte;
    u1: uint64;
  end align 16;

  TRecPackedAligned = packed record
    b1: byte;
    u1: uint64;
  end align 16;

The packed record is still aligned on a 1-byte boundary while the unpacked record is aligned to a 16-byte boundary.

As I said, it is only a guess. The wording of the Embarcadero quote isn't that much clear on the suject.

Uwe Raabe
  • 45,288
  • 3
  • 82
  • 130
  • I'm fairly sure that the change concerns *alignment* of the record rather than its internal *layout*. However, packed records had alignment of 1 in previous versions too. I must say, I'm not familiar with that `align` syntax. Is that recent? – David Heffernan Dec 12 '11 at 19:56
  • The relevant documentation is here: http://docwiki.embarcadero.com/RADStudio/en/Internal_Data_Formats#Record_Types – David Heffernan Dec 12 '11 at 20:02
  • 1
    I stumbled upon it by chance, but couldn't find any documentation about it. I'm not sure if your assumption is right: the online help mentions _The $A directive controls alignment of fields in Delphi record types and class structures_ while the quote above says _Alignment of Records_ – Uwe Raabe Dec 12 '11 at 20:03
  • We all know how erratic the documentation is. By trial and error it can be found that packed and $A- and $A1 all result in the alignment of the record being 1. – David Heffernan Dec 12 '11 at 20:07
  • 1
    This behavior is acknowledged as a [defect](http://qc.embarcadero.com/wc/qcmain.aspx?d=87283), if we are to take the QC site serious. – Sertac Akyuz Dec 12 '11 at 20:09
  • @Uwe See my update to the question. Focusing on alignment, I can't see what has changed as the versions change. – David Heffernan Dec 12 '11 at 21:34
1

As far as I remember, record used to be packed since a version of the compiler around Delphi 5-6.

Then, for performance reasons, plain record fields were aligned, according to the settings in the project option. If you define a packed record there won't be any alignment within the record.

The text you are quoting seems related not specifically to XE2, but to a Delphi 2009 change. See this Blog entry for historical purpose.

I guess that "'Packed' Now Forces Byte Alignment of Records" refers to the Delphi 2009 {$OLDTYPELAYOUT ON} feature - which may have some implementation issue before XE2. I agree with you: it sounds like a documentation issue.

David Heffernan
  • 601,492
  • 42
  • 1,072
  • 1,490
Arnaud Bouchez
  • 42,305
  • 3
  • 71
  • 159
  • My understanding of the XE2 What's New is that it relates to *alignment* of the record itself rather than the record's *layout* which is what `$OLDTYPELAYOUT` controls. My best guess is that the link in the What's New to [Internal Data Formats](http://docwiki.embarcadero.com/RADStudio/en/Internal_Data_Formats) was created in anticipation of content being added to Internal Data Formats but that content has not been added yet. But I could very well be wrong. It's still a complete mystery to me! – David Heffernan Dec 11 '11 at 10:01
  • 1
    About `$Align`, `{$A1}` and `{$A-}` are equivalent to `packed` for all versions of Delphi (including XE2) - as far as I can tell from the code I compiled with this compiler. – Arnaud Bouchez Dec 11 '11 at 17:33
-1

pascal has always supported packed structures which are probably easiest explained using an example

Lets say you have two arrays x and y, and that we are using a 16 bit processor. That is, the 'word' size of the processor is 16 bits.

myrec = 
record
  b : byte;
  i : integer;
end;

x : array[1..10] of myrec;
y : packed array[1..10] of myrec;

Pascal only tells you that you have 10 elements to work with, each holding a 3 bytes of information (lets assume integer is the old 16 bit variety). It actually says nothing about how that information is stored. You may assume the array is stored in 30 consecutive bytes, however this is not necessary true and is completely compiler dependent (a really good reason to avoid pointer mathematics).

A complier may well put a dummy byte in between the field values b and i, in order to ensure that both b and i fall on word boundaries. In this instance the structure will take a total of 40 bytes.

The 'packed' reserved word instructs the compiler to optimise for size over speed, whereas leaving packed out, the compiler will generally optimise for speed over size. In this case the structure would be optimised and could take only 30 bytes.

Again I say 'could' because it is still compiler dependent. The 'packed' keyword just says to pack the data closer together, but doesn't actually say how. Some compilers therefore simply ignored the keyword, whereas others used packed to disable word alignment.

In the case of the latter, what typically could happen is that each element is aligned to the word size of the processor. You need to realise here that the word size of the processor is typically the register size, not simply '16 bits' as it has commonly come to mean. In the case of a 32 bit processor for example the word size is 32 bits (even though we generically define words as 16 bits - this is just a historical throwback) In a sense it is it the 'memory' equivalent of a disk sector.

By packing data, you could potentially have values such as integer crossing over word boundaries therefore requiring more memory manipulation (much like needing to read two disk sectors if a record crosses a sector boundary).

So basically what the change means, is the delphi is now fully using the packed keyword and forcing byte alignment (over word alignment) all of the time.

Hope this is enough to explain it. It is actually a bigger topic that I can reasonably answer in a few paragraphs.


UPDATE, continuation of comment thread below...

DavidH> I cannot imagine that you mean that the compiler is capable of producing different output given identical input

Yes David, that is exactly what I am saying. Not necessarily from one compile to the next on the same compiler, but certainly between versions of compiler, brands of compiler and different platforms.

TonyH > I think most of us always assumed packed = byte aligned, so now we are scratching our heads for a scenario where it doesn't. @David Heffernan is asking if anyone knows one

Tony, you have hit the nail on the head here. You have assumed that packed works like a compiler directive that specifies how the data memory is organised. This is not what it means. A better analogy is to consider the optimise directive for code generation. You know the code is being optimised, but you don't necessarily specifically what the underlying outputted code will be, nor do you need to.

I can give you plenty of examples. Lets say you have the following (I am using an array, it could apply just as easily to a record)

myarray = packed array[1..8] of boolean

If you then print a SizeOf(myarray) what would you expect? The answer is that delphi does not guarantee that packed is implemented in any particular way. So therefore the answer could be 8 bytes (if you are byte aligning each element). It is just as valid for delphi to pack these as bits, and so the entire array could fit into 1 byte. It could be 16 bytes if the compiler decides not to optimise on byte boundaries and is running on a 16 bit architecture. Yes it would be less efficient in terms of speed, but the whole point of packing is to optimise for size over speed. Whatever it does is invisible to the developer as long as they simply access the array elements and don't make any assumptions and try their own pointer mathematics to access the underlying data.

Similarly you cannot even guarantee the internal organisation of a primitive - it is platform dependent, not compiler dependent. E.g. If you have worked across multiple architectures you will be aware of issues surrounding how many bits make up an integer; whether or not the bytes within that integer are stored in byte order, or reverse byte order.

These types of architectural issues are exactly why various techniques such as comma delimited file, xml, etc have been developed. In order to provide a reliable common platform.

To summarise, David the issue is that you are actually asking the wrong question. If you consider the Embarcadero quote in the context of what I have said, you will see that it is consistent with what I am saying.

Peter
  • 1,065
  • 6
  • 18
  • 2
    I've **never** seen `packed array[1..8] of boolean` be packed as one `byte` by any version of Delphi compiler. FreePascal can do this - and this is especially useful for embedded targets like ARM with some KB of RAM. But I've never seen the Delphi compiler doing such bit-packing, which is very common with C compilers. – Arnaud Bouchez Dec 12 '11 at 16:31
  • @Arnaud What would `@myarray[1]` return in that case for such FPC output? I'm not sure what you mean about C either. In the C standard it is explicit that every element of an array must be addressable so bit packing can't be done with C arrays. – David Heffernan Dec 12 '11 at 16:46
  • @DavidHeffernan fpc gives "Error: The address cannot be taken of bit packed array elements and record fields" (except in the modes where each bit takes up a whole byte), and yes, packing can't be done in C arrays, but it can with C bit-fields: struct { unsigned bit1 : 1; unsigned bit2 : 1; } In such a struct, you're not allowed to take a bit field's address either. –  Dec 12 '11 at 20:03
  • @hvd I didn't know about packed arrays in FPC and it's interesting to know thanks. But since I use Delphi rather than FPC I'll just have to store that knowledge away in the locker for some future date. – David Heffernan Dec 12 '11 at 20:08
  • 1
    About FPC bit packed arrays, RTFM - see http://www.freepascal.org/docs-html/ref/refse14.html#QQ2-39-51 and http://www.freepascal.org/docs-html/ref/refsu15.html#x39-460003.3.1 For FPC @myArray[1] will probably return the same as @myArray[0] or won't ever compile (my guess) - but you will never use @myArray[1] but directly get or set the boolean value using myArray[1]. Delphi lacks of the pack/unpack functions. – Arnaud Bouchez Dec 12 '11 at 20:10
  • @DavidHeffernan I was just quoting FPC because I did not understand what Peter meant about Delphi bitpacking feature. Whereas this feature exists in FPC, and is very common in its embedded use (far away from Delphi, I confess). – Arnaud Bouchez Dec 12 '11 at 20:16
  • @Arnaud So far as I can tell, the syntax is `bitpacked array ...`. The compiler error when you try to get the address is exactly as hvd says. Good old ideone.com! – David Heffernan Dec 12 '11 at 20:29
  • @DavidHeffernan I actually used packed array[1..8] of Boolean, with {$MODE MacPas}. –  Dec 12 '11 at 21:08
  • @DavidHeffernan The manual says that in MACPAS mode, or with `$BITPACKING` directive set to ON, `packed` is equivalent to the `Bitpacked` keyword. – Arnaud Bouchez Dec 13 '11 at 08:43