Why is Java/Linux appending 0xa to end of file?

Question

I'm writing a file using FileOutputStream, and I'm noticing the hex seems to include a terminating 0x0a. Here is the output from the file:

0000000: d100 0b00 0000 4865 6c6c 6f20 776f 726c  ......Hello worl
0000010: 6429 0043 0500 0000 7072 696e 740a       d).C....print.

(Note hex code comes from vim on Osx Yosimite x64)

Notice that the file ends in 0x0a. Is this a linux thing (like some sort of EOF character) or is this java/FileOutputStream appending the character?

Edit: Code

public class FileImageOpcodeRenderer implements OpCodeRenderer, AutoCloseable {
  private final FileOutputStream stream;
  private final Iterable<OpCode> opcodes;
  private final List<Byte> data = new ArrayList<>();

  public FileImageOpcodeRenderer(Iterable<OpCode> opcodes, String path) {
    this.opcodes = opcodes;
    try {
      this.stream = new FileOutputStream(path);
    } catch (IOException ex) {
      ex.printStackTrace();

      throw new CompilerException("Unable to open output file");
    }
  }

  public void save() throws Exception {
    List<Byte> renderedContent = renderOpCodes();
    byte[] content = toArray(renderedContent);
    stream.write(content, 0, content.length);
  }

  private List<Byte> renderOpCodes() {
    for (OpCode opCode : opcodes) {
      writeInt8(null, opCode.opNumber());
      opCode.render(this);
    }

    return this.data;
  }

  private byte[] toArray(List<Byte> data) {
    byte[] content = new byte[data.size()];

    for (int index = 0; index < data.size(); index++) {
      content[index] = data.get(index);
    }

    return content;
  }

  @Override
  public void writeInt8(String label, int value) {
    data.add((byte)value);
  }

  @Override
  public void writeInt16(String label, int value) {
    writeInt8(null, (byte)(value & 0xFF));
    writeInt8(null, (byte)((value >> 8) & 0xFF));
  }

  @Override
  public void writeInt32(String label, int value) {
    writeInt8(null, (byte)(value & 0xFF));
    writeInt8(null, (byte)((value >> 8) & 0xFF));
    writeInt8(null, (byte)((value >> 16) & 0xFF));
    writeInt8(null, (byte)((value >> 24) & 0xFF));
  }

  @Override
  public void writeInt64(String label, long value) {
    writeInt8(null, (byte)(value & 0xFF));
    writeInt8(null, (byte)((value >> 8) & 0xFF));
    writeInt8(null, (byte)((value >> 16) & 0xFF));
    writeInt8(null, (byte)((value >> 24) & 0xFF));
    writeInt8(null, (byte)((value >> 32) & 0xFF));
    writeInt8(null, (byte)((value >> 40) & 0xFF));
    writeInt8(null, (byte)((value >> 48) & 0xFF));
    writeInt8(null, (byte)((value >> 56) & 0xFF));
  }

  @Override
  public void writeString(String label, String value) {
    writeInt32(null, value.length());
    for (byte letter : value.getBytes()) {
      System.out.println((int)letter);
      writeInt8(null, letter);
    }
  }

  @Override
  public void close() throws Exception {
    stream.close();
  }
 }

I'm even looking at the raw bytes in save and it ends with 0x74, not 0x0a.

Update: Moar Code:

So I just wrote this code to load the file in c++:

int main() {
    int fd = open("/Users/sircodesalot/Desktop/image.vbaj", O_RDONLY);

    char buffer[256] { };
    int amout = read(fd, buffer, 256);

    cout << amout << endl;

    for (int index = 0; index != 256; ++index) {
        if (index > 0 && (index % 10 == 0)) cout << endl;
        cout << hex << (int)buffer[index] << " ";
    }

    close(fd);
}

which outputs this:

29
ffffffd1 0 b 0 0 0 48 65 6c 6c 
6f 20 77 6f 72 6c 64 29 0 43 
5 0 0 0 70 72 69 6e 74 0 
0 0 0 0 0 0 0 0 0 0
... (rest of the 256 byte buffer)

Look ma, no 0xa? What's the deal with that? Maybe some linux convention?

0x0a is a linefeed, the line separator on Linux. What did you write exactly? — Henry, Sep 23 '14 at 14:27
Are linefeeds always appended to the end of a file? The routine used to write 'Hello world' is the same as the one to write 'print', so why does one have a linefeed, but not the other? — sircodesalot, Sep 23 '14 at 14:28
linefeeds are not automatically appended to files. It would only happen if you used `println` for example. — Henry, Sep 23 '14 at 14:31
Updated with code. Both "Hello World" and "print" use `writeString`. — sircodesalot, Sep 23 '14 at 14:37
I guess your hex output shows the bytes swapped and the file actually ends with 0x0a 0x74. How did you produce the hexdump? — Henry, Sep 23 '14 at 15:11
Because vim does not like a last line that does not end with a line feed. To check, use some other program to see the contents of the file, for example `hexdump -C filename` — Henry, Sep 23 '14 at 20:02

score -3 · Answer 1 · answered Sep 23 '14 at 14:45

-3

The problem probably lies here:

@Override
public void writeString(String label, String value) {
  writeInt32(null, value.length());
  for (byte letter : value.getBytes()) {
      System.out.println((int)letter);
      writeInt8(null, letter);
  }
}

You are writing the length in characters, but then proceed to write the bytes from getBytes(). getBytes() uses the platform encoding for the String, and depending on what the encoding is getBytes() can return you about anything.

Try explicitly specifying the encoding e.g. value.getBytes(Charset.forName("ISO-8859-1"));

Edit: Also make sure the Strings you output do contain what you think they contain.

answered Sep 23 '14 at 14:45

Durandal

19,919
4
36
70

Not the downvoter, but that didn't work. – sircodesalot Sep 23 '14 at 14:47
Stopped caring about anonymous downvotes on the second day on SO :) – Durandal Sep 23 '14 at 14:51
I wrote some c code to load the file, and it seems to be ignoring the `0xa`. I wonder if terminating a file with a linefeed some sort of posix convention. – sircodesalot Sep 23 '14 at 14:57
@sircodesalot Where do all the zeros at the end come from in your edit? Looks like whatever you used to generate the hex dump modified the input (assuming they are from the same file). Did you by chance load it as *text* and then switched to hex display? – Durandal Sep 23 '14 at 15:00
I'm using a 256 byte array buffer, so the uninitialized stuff is written too. – sircodesalot Sep 23 '14 at 15:18

Why is Java/Linux appending 0xa to end of file?

1 Answers1