5

Here's some sample test code I'm trying to run on an embedded Linux system:

#include <iostream>

int main(int argc, char *argv[])
{
    char c = 'A';
    int i = 7;

    std::cout << "Hello World from C++" << std::endl;
    std::cout << "c=" << c << std::endl;
    std::cout << "i=" << i << std::endl;
}

The embedded system is a Microblaze, which is a 32-bit RISC softcore processor running on a Xilinx FPGA. Please don't be put off by that, as a lot of your standard Linux knowledge will still apply. The processor is configured as LSB with an MMU, and the Linux build I'm using (PetaLinux, supplied by Xilinx) is expecting the same. I'm using the supplied GNU compiler; Microblaze appears to be officially supported in GCC.

The problem I'm having is that when the stdlib needs to interact with the integer, it segfaults. Here's the output:

Hello World from C++
c=A
Segmentation fault

Note that the char is handled fine. The C equivalent of this code also works fine:

#include <stdio.h>

int main(int argc, char *argv[])
{
    char c = 'A';
    int i = 7;

    printf("Hello World from C\n");
    printf("c=%c\n", c);
    printf("i=%i\n", i);

    return 0;
}

...

Hello World from C
c=A
i=7

This leads me to suspect an issue with the shared library libstdc++.so.6.0.20. That library is supplied by Xilinx though, so it should be correct. The file output of that library is:

libstdc++.so.6.0.20: ELF 32-bit LSB shared object, Xilinx MicroBlaze 32-bit RISC, version 1 (SYSV), dynamically linked, not stripped

The file output of my binary is:

cpptest: ELF 32-bit LSB executable, Xilinx MicroBlaze 32-bit RISC, version 1 (SYSV), dynamically linked, interpreter /lib/ld.so.1, for GNU/Linux 2.6.32, stripped

I've also tried statically linking my binary using the -static flag, but the result was the same.

Here are the most relevant compiler and linker settings I'm using, but I've tried changing these with no avail.

CC=microblazeel-xilinx-linux-gnu-gcc
CXX=microblazeel-xilinx-linux-gnu-g++

CFLAGS= -O2 -fmessage-length=0 -fno-common -fno-builtin -Wall -feliminate-unused-debug-types
CPPFLAGS?=
CXXFLAGS= -O2 -fmessage-length=0 -fno-common -fno-builtin -Wall -feliminate-unused-debug-types
LDFLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed

To compile:
@$(CCACHE) $(CXX) $(CPPFLAGS) $(CXXFLAGS) $(CFLAGS) $< -o "$@"

To link:
@$(CXX) $(RELOBJECTS) $(LDFLAGS) $(EXT_LIBS) -o $(RELBINARY)

Note that microblazeel refers to the little endian version of the microblaze compiler.

I would very much like to debug this or at least look at a coredump, but no coredump seems to be produced when the segfault happens, and there's no gdb executable in the Microblaze Linux build. Maybe I'm missing something?

Thanks for taking your time to read this. Any thoughts?

Sonic Atom
  • 436
  • 5
  • 15
  • what about `ulimit -c unlimited`, copy core file after crash to host, and open it with `gdb` from cross toolchain? – fghj Nov 16 '15 at 18:37
  • 1
    And what happened if you replace `char` and `int` declaration order? – fghj Nov 16 '15 at 18:38
  • It's not clear to me, but is the compiler and standard library both supplied by Xilinx? That is, are they compatible? – Anon Mail Nov 16 '15 at 18:39
  • What's the ldd output of your binary? – Mark B Nov 16 '15 at 18:40
  • Can you try stripping out all the non-essential compiler/linker switches from the makefile and see if that makes any difference (i.e. get rid of all the `-fxxx` and other switches that aren't strictly needed) ? – Paul R Nov 16 '15 at 18:41
  • Some rules of thumb I've picked up recently, which may help in narrowing down on the issue - declaring your variables in the static, global heap memory instead of in a function's stack memory can sometimes help. ALSO, going off @user1034749's recommendation of swapping declarations - some devices don't handle memory allocations with an odd number of bytes very well. A Motorola-based Coldfire (Netburner), for example, was giving me a TON of seemingly random faults when I added a char-type variable to the flash memory table, but cleared up as soon as I switched to a 16-bit or 32-bit int type. – depwl9992 Nov 16 '15 at 18:44
  • FYI, I've written C for a Microblaze before. Your std version has no `return 0`?. Microblaze can be configured with/without things like HW floating pt, HW debug support, etc. Are you sure you're compiling for and linking in the correct compatible library versions for your HW/firmware load? Adding -v during build may show. You may need another option and/or -L. gdb _should_ be available--Xilinx supports it. See http://www.xilinx.com/itp/xilinx10/help/platform_studio/ps_c_dbg_debugging_mb_sw_overview.htm – Craig Estey Nov 16 '15 at 20:35
  • @CraigEstey Yeah a possible difference between the firmware configuration and the configuration that libstdc++ was compiled for is one of the things that's worrying me. Right now I'm thinking that's the most likely cause. It seems very suspect that the one-byte char passes fine, but the 4-byte int doesn't. With the `file` command I can see endian-ness of the lib is ok, but that's about all I can see. The kernel and most of the filesystem come from the reference bsp for the dev board, but that bsp didn't contain libstdc++ so I had to extract that from the petalinux-sdk, also supplied by Xilinx. – Sonic Atom Nov 17 '15 at 10:25
  • @CraigEstey Something tells me I may end up having the re-compile libstdc++ from source. – Sonic Atom Nov 17 '15 at 10:25
  • @CraigEstey Thanks for that link to the debugging tutorial, that looks useful. Trouble is it's written for the old ISE IDE, whereas I'm using Vivado, the Xilinx SDK (modified Eclipse) and the petalinux-sdk. Do you know of an updated version of that tutorial for those tools? Thanks. – Sonic Atom Nov 17 '15 at 10:30
  • Bare metal, 2019.2 std::printf with %s also causes a stack issue. Perhaps cout and printf call similar code that causes an issue? – Flip Nov 10 '21 at 12:28

1 Answers1

1

From what I could see after a bit of research, vivado is the HW development IDE [because they offer a trial period--so it's the HW devel, which they always want to charge for].

If you're using the standard SDK board from Xilinx, everything should be preconfigured. Otherwise, a HW designer produces a HW design that has Microblaze in it.

From that, you may have to use petalinux to generate a new boot, kernel, etc. image that is compatible.

You may need to rebuild libstdc++ from source, but I'd do that as a last resort. For example, don't bother with it, until you've got gdb working and have test results.


Here are some petalinux PDF files:
http://www.xilinx.com/support/documentation/sw_manuals/petalinux2013_10/ug977-petalinux-getting-started.pdf
http://www.xilinx.com/support/documentation/sw_manuals/petalinux2013_10/ug981-petalinux-application-development-debug.pdf


The development guide shows how to invoke gdb (e.g.):
On the target system:
gdbserver host:1534 /bin/myapp
On the development system:
petalinux-utils --gdb myapp followed by target remote 192.168.0.10:1534


I've done some editing on your Makefile with annotations. I've commented out some of the non-essential options. Note that I'm using the += operator to build up CFLAGS/CXXFLAGS gradually

The basic idea here is to do a build with minimum deviations from "standard". Add only proven essential options. Build and test. Add back the options one-by-one [rebuild and test each time] until you find the option that is causing the problem.

I do, however, have a strong suspicion about -fno-common being a source of problems. Also, to a lesser extent, I'm a bit suspicious of -Wl,--as-needed

Should these options work? Sure, but xilinx/microblaze ain't no x86 ...

I've added two command line make variables:
DEBUG -- generate for debug with gdb
VERBOSE -- show everything about the build process

For example, try make <whatever> DEBUG=1 VERBOSE=1

CC = microblazeel-xilinx-linux-gnu-gcc
CXX = microblazeel-xilinx-linux-gnu-g++

CPPFLAGS ?=

CMFLAGS += -Wall -Werror
CMFLAGS += -fmessage-length=0

# compile for gdb session
# NOTES:
# (1) -gdwarf-2 may or may not be the the right option for microblaze
# (2) based on doc for -feliminate-unused-debug* petalinux/microblaze may want
#     stabs format
ifdef DEBUG
  CMFLAGS += -gdwarf-2
  CMFLAGS += -O0

# compile for normal build
#else
  CMFLAGS += -O2
  CMFLAGS += -feliminate-unused-debug-types
endif

# NOTE: I used to use "@" on commands, but now I leave it off -- debug or not
# sure it's "ugly" but you can get used to it pretty quickly--YMMV
ifndef VERBOSE
  Q :=
else
  ###Q := @
  Q :=
endif

# let compiler/linker tell you _everything_:
# (1) configure options when tool was built
# (2) library search paths
# (3) linker scripts being used
ifdef VERBOSE
  CMFLAGS += -v
  LDFLAGS += -Wl,--verbose=2
endif

CMFLAGS += -fno-builtin

# NOTE: I'd _really_ leave this off as it may confuse c++ std as "<<" calls
# _M_insert (which is in the library, which is almost certainly _not_ using
# -fno-common)
###CMFLAGS += -fno-common

# NOTE: I'm also suspicious of this a little bit because the c++ lib may have
# some "weak" symbols that the c library doesn't
###LDFLAGS += -Wl,--as-needed

# NOTE: this seems harmless enough, but you can comment it out to see if it
# helps
LDFLAGS += -Wl,--hash-style=gnu

# NOTE: an optimization only
ifndef DEBUG
  LDFLAGS += -Wl,-O1
endif

CFLAGS += $(CMFLAGS)
CXXFLAGS += $(CMFLAGS)

# NOTES:
# (1) leave this off for now -- doesn't save _that_ much and adds complexity
#     to the build
# (2) IMO, I _never_ use it and I erase/uninstall it on any system I
#     administrate (or just ensure the build doesn't use it by removing it
#     from $PATH)--YMMV
###XCCACHE = $(CCACHE)

# to compile
$(Q)$(XCCACHE) $(CXX) $(CPPFLAGS) $(CXXFLAGS) $(CFLAGS) $< -o "$@"

# to link
$(Q)$(CXX) $(RELOBJECTS) $(LDFLAGS) $(EXT_LIBS) -o $(RELBINARY)
Craig Estey
  • 30,627
  • 4
  • 24
  • 48