RISCV Inline Assembly in GCC: Atomic Load/Store

Question

Edit:

I believe that the clue here was "Illegal *operand". The problem was with operands. I changed some operand ATTRIBUTES from "m" (or whatever) to "A", and I made progress. For RISC-V, if you are passing an address/pointer, you must use "A" attribute!

Colleagues!

Please consider this block of inline gcc assembly (excerpted from foo.c):

  asm volatile(
    "als_%=:\n\t"
    "  lr.w.aq  t1, %[src]\n\t"
    "  sc.w.rl  t0, t1, %[dst]\n\t"
    "  bnez     t0, als_%="
    : [dst] "=&m" (*((uint32_t*)(0x27402448ULL)))
    : [src] "m"   (*((uint32_t*)(0x000a0018ULL))));

It intends to, first, load the contents of the source address to t1.
Then to, second, store the contents of t1 to the destination address.
Finally, if the store fails (looking at status in t0), then the process will be attempted again.

Note that there is no "symbol" at these addresses. We're running bare-metal code, creating pathological coherency scenarios among "interesting" random memory locations.

I get this error using SiFive's pre-built gcc-8.3.0:

foo.c: Assembler messages:
foo.c:1106: Error: illegal operands `lr.w t1,8(a4)`
foo.c:1107: Error: illegal operands `sc.w t0,t1,1048(s0)`

I've tried to la from those locations into temporary registers and then to use the registers in the ld/st instructions, but I get similar errors on the la.

My command line is:

/path/to/riscv64-unknown-elf-gcc-8.3.0-2019.08.0-x86_64-linux-centos6/bin/riscv64-unknown-elf-gcc \
-ggdb3 \
-std=gnu99 \
-Wall -Wextra -Werror -Wshadow -Wstack-usage=1024 -Wno-implicit-fallthrough -Wno-unused-parameter \
-ffunction-sections \
-fdata-sections \
-I/path/to/target/include \
--specs=nano.specs \
-Os \
-nostartfiles \
-nostdlib \
-Xlinker -Map=foo.map \
-march=rv64imafdc \
-mabi=lp64d \
-mcmodel=medany \
-malign-data=natural \
-mriscv-attribute \
-I/path/to/freedom-e-sdk/bsp/qemu-sifive-u54mc/install/include \
-L/path/to/freedom-e-sdk/bsp/qemu-sifive-u54mc/install/lib/debug \
foo.c \
-T../riscv64/link.ld \
-Wl,--start-group \
  -lc -lgcc -lm -lmetal -lmetal-gloss \
-Wl,--end-group \
-Wl,--gc-sections \
-o foo.elf;

Have you tried manually assembling those instructions into machine code, then using `objdump -d` to see what syntax it wants you to use? Or maybe it's a matter of telling the assembler about CPU features if those instructions aren't baseline. (Sorry I don't know RISC-V well enough to know if they are or not.) Perhaps try using `stdatomic.h`'s `atomic_fetch_add_explicit()` with `memory_order_acq_rel` and see the syntax of LL/SC instructions GCC itself emits for its `__atomic` builtins? — Peter Cordes, Sep 02 '21 at 18:34
@PeterCordes, Thanks for your help! I can't use even for testing because no one has ever implemented the methods that underlie them for RISCV32/RISCV64, like __atomic_load_16(). Anything in bare-metal will fail with link errors that use those (even using gcc-11.2.0, which I've also tried). — Lance E.T. Compte, Sep 02 '21 at 19:42
https://godbolt.org/z/Pe6e4j7bf shows `atomic_fetch_add_explicit` on an `_Atomic int*` inlines just fine with rv64 GCC 10.2.0, to `fence iorw,ow;` and `amoadd.w.aq zero,a5,0(a0)` instructions. I'm not surprised `__atomic_load_16` couldn't inline, 16-byte atomics may not be lock-free even on RV64 and thus need library support. — Peter Cordes, Sep 02 '21 at 20:12
Or more relevantly, for an atomic_load (mo_acquire), with that set of ISA extensions, GCC uses plain `lw` + barrier, not an `lr.w.aq`. https://godbolt.org/z/6n8q3PjTh. Your GAS does apparently recognize them, though, otherwise I think it would complain about the mnemonic, not the operands. — Peter Cordes, Sep 04 '21 at 06:39
@old_timer: I don't think removing `[assembly]` was appropriate here. Clearly there's a problem with actually assembling those asm instructions, not with the GNU C inline asm syntax and constraints that got the compiler to emit that string and feed it to GAS. Some [inline-assembly] questions aren't really [assembly] questions, but that's not the case here. It would be the same question if using those instructions is a `.S` file. — Peter Cordes, Sep 04 '21 at 07:20
fair enough I didnt see that the OP had tried real assembly nor had tried godbolt or other sources of different versions of gas. — old_timer, Sep 04 '21 at 14:14
@old_timer: The question doesn't mention having actually tried a separate `.S` file, but we know that GCC works by creating a `.s` and feeding it to GAS. So those assembler error messages are coming from the real assembler that would be used if you did try that in stand-alone asm. So unlike clang or MSVC, there's no argument that the compiler's built-in assembler might be different from real assembly language; GCC doesn't have a built-in assembler. — Peter Cordes, Sep 04 '21 at 20:49
Okay, I *think* the clue here was "illegal **operand*". The problem was with operands. I changed operand ATTRIBUTES from "m" (or whatever) to "A", and I made progress. For RISC-V, if you are passing an address/pointer, you must use "A" attribute. — Lance E.T. Compte, Sep 07 '21 at 19:15

RISCV Inline Assembly in GCC: Atomic Load/Store

0 Answers0