How to tell gcc to not align function parameters on the stack?

Question

I am trying to decompile an executable for the 68000 processor into C code, replacing the original subroutines with C functions one by one.

The problem I faced is that I don't know how to make gcc use the calling convention that matches the one used in the original program. I need the parameters on the stack to be packed, not aligned.

Let's say we have the following function

int fun(char arg1, short arg2, int arg3) {
    return arg1 + arg2 + arg3;
}

If we compile it with

gcc -m68000 -Os -fomit-frame-pointer -S source.c

we get the following output

fun:
    move.b 7(%sp),%d0
    ext.w %d0
    move.w 10(%sp),%a0
    lea (%a0,%d0.w),%a0
    move.l %a0,%d0
    add.l 12(%sp),%d0
    rts

As we can see, the compiler assumed that parameters have addresses 7(%sp), 10(%sp) and 12(%sp):

but to work with the original program they need to have addresses 4(%sp), 5(%sp) and 7(%sp):

One possible solution is to write the function in the following way (the processor is big-endian):

int fun(int bytes4to7, int bytes8to11) {
    char arg1 = bytes4to7>>24;
    short arg2 = (bytes4to7>>8)&0xffff;
    int arg3 = ((bytes4to7&0xff)<<24) | (bytes8to11>>8);
    return arg1 + arg2 + arg3;
}

However, the code looks messy, and I was wondering: is there a way to both keep the code clean and achieve the desired result?

UPD: I made a mistake. The offsets I'm looking for are actually 5(%sp), 6(%sp) and 8(%sp) (the char-s should be aligned with the short-s, but the short-s and the int-s are still packed):

Hopefully, this doesn't change the essence of the question.

UPD 2: It turns out that the 68000 C Compiler by Sierra Systems gives the described offsets (as in UPD, with 2-byte alignment).

However, the question is about tweaking calling conventions in gcc (or perhaps another modern compiler).

I take it that the rest of the application is not being rewritten in C too? If it were then you’d have C code calling your C code, which would be just fine. — bazza, Jun 26 '20 at 18:15
[According to this](https://stackoverflow.com/questions/1855896/memory-alignment-on-modern-processors/1855994#1855994) the 68000 and 68010 don't support unaligned access, but the 68020 does. So I would try `-m68020`. Another possibility is `fun(int base)` with `struct packedArgs *args = &base;` where `struct packedArgs` is a packed structure containing the three arguments. — user3386109, Jun 26 '20 at 18:16
Also, it might be possible to cheat by passing a packed structure as an argument, but gcc may still align that to byte 7. To get really dirty you could take the address of that argument, subtract 4 bytes, assign it to a pointer to this packed structure — bazza, Jun 26 '20 at 18:19
@user3386109: `-m68020` isn't going to change the ABI / calling convention; it might be necessary (so good point) but not sufficient. — Peter Cordes, Jun 26 '20 at 20:28
With the update your question makes much more sense. Passing the parameters via `4(%sp)`, `5(%sp)` and `7(%sp)` would be very expensive on 68000/68010. — chtz, Jun 29 '20 at 10:10
Your update is actually a huge change. The first version was super-weird, with a misaligned `short` and misaligned `long`, a calling convention no sane compiler would ever use, and literally not usable on 68000 unless the callee uses multiple instructions to load the bytes separately and shift them together. (If comments are correct that 68000 won't do unaligned loads at all; I forget). A calling convention with 2-byte stack slots is totally plausible `alignof(int32_t) = 2`, if m68k supports 4-byte loads with 2-byte alignment. — Peter Cordes, Jun 29 '20 at 11:02

score 4 · Answer 1 · answered Jun 26 '20 at 19:26

Here's a way with a packed struct. I compiled it on an x86 with -m32 and got the desired offsets in the disassembly, so I think it should still work for an mc68000:

typedef struct {
    char arg1;
    short arg2;
    int arg3;
} __attribute__((__packed__)) fun_t;

int
fun(fun_t fun)
{

    return fun.arg1 + fun.arg2 + fun.arg3;
}

But, I think there's probably a still cleaner way. It would require knowing more about the other code that generates such a calling sequence. Do you have the source code for it?

Does the other code have to remain in asm? With the source, you could adjust the offsets in the asm code to be compatible with modern C ABI calling conventions.

I've been programming in C since 1981 and spent years doing mc68000 C and assembler code (for apps, kernel, device drivers), so I'm somewhat familiar with the problem space.

For ARM processors there's a qualifier `__packed` that can be used for function parameters. I stumbled upon it in this [question](https://stackoverflow.com/questions/28186492/what-is-the-use-of-packed-attribute-in-function-arguments). Maybe there's something similar for 68000? — Glinka, Jun 26 '20 at 20:38
@Glinka: That's an `armcc` feature, not GCC. Also, it corresponds to GCC `__attribute__((aligned(1)))` for the pointed-to data, not for the arg itself on the stack. From the linked Q&A, I don't see any hint that it changes the ABI for stack args, or any reason why it would. So despite the name and syntax, I'm pretty sure that does something else and isn't what we're looking for. — Peter Cordes, Jun 26 '20 at 21:00

score 3 · Answer 2 · answered Jun 29 '20 at 08:38

It's not a gcc 'fault', it is 68k architecture that requires stack to be always aligned on 2 bytes. So there is simply no way to break 2-byte alignment on the hardware stack.

but to work with the original program they need to have addresses 4(%sp), 5(%sp) and 7(%sp):

Accessing word or long values off the ODD memory address will immediately trigger alignment exception on 68000.

chtz · Accepted Answer · 2020-06-30T16:12:48.003

2

To get integral parameters passed using 2 byte alignment instead of 4 byte alignment, you can change the default int size to be 16 bit by -mshort. You need to replace all int in your code by long (if you want them to be 32 bit wide). The crude way to do that is to also pass -Dint=long to your compiler. Obviously, you will break ABI compatibility to object files compiled with -mno-short (which appears to be the default for gcc).

edited Jun 30 '20 at 16:12

answered Jun 29 '20 at 10:04

chtz

17,329
4
26
56

2

I can't find `-mlong` on the [Option Summary](https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html) page of GCC online docs, but the [M680x0 Options](https://gcc.gnu.org/onlinedocs/gcc/M680x0-Options.html) page says that `-mno-short` is the default. – Maxim Jun 30 '20 at 15:33
3

You are right, I picked that information from a page about the [tigcc fork](http://tigcc.ticalc.org/doc/comopts.html), which apparently is not in mainline gcc. – chtz Jun 30 '20 at 16:15

How to tell gcc to not align function parameters on the stack?

3 Answers3