8

Instead of the overhead with saving binary as Base64, I was wondering if you could directly store double-byte binary streams into XML files, using CDATA, or commenting it out, or something?

Community
  • 1
  • 1
Robin Rodricks
  • 110,798
  • 141
  • 398
  • 607

4 Answers4

14

No you can't use CDATA alone to inject binary data in an XML file.

In XML1.0 (because XML 1.1 is more permissive, but not about control chars), the following restrictions apply to CDATA characters:

CData      ::=      (Char* - (Char* ']]>' Char*)) 
Char       ::=      #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]

That means there are several characters illegal, among them are:

  • illegal XML control characters 0x00 to 0x20 except new lines, carriage returns and tabs
  • illegal UTF-8 sequences like 0xFF or the non canonical 0b1100000x 0b10xxxxxx

In addition to that, in a standard entity content without CDATA :

  • "<" and ">" use are illegal
  • "&" use is restricted (&eacute; is OK, &zajdalkdza; is not)

So CDATA is just a way to allow "<", ">" and "&", by restricting "]]>" instead. It doesn't solve the illegal XML, Unicode and UTF-8 characters issue which is the main problem.

Solutions:

  1. Use Base64 with 33% overhead but a large support in all programming languages and the fact that it's a standard
  2. Use BaseXML with still limited implementations but 20% overhead only
  3. Don't encode binary data within XML if possible, transfer it separately
Community
  • 1
  • 1
KrisWebDev
  • 9,342
  • 4
  • 39
  • 59
13

The Nul character ( '\0' in C ) is not valid anywhere in XML, even as an escape ( & #0; ).

Pete Kirkham
  • 48,893
  • 5
  • 92
  • 171
  • I suppose because XML files are null-terminated. – Robin Rodricks Feb 02 '09 at 11:12
  • 2
    @Jeremy: They aren't. Null is just not a valid XML character, likely because of null-terminated strings in a popular programming language... – Christoph Feb 02 '09 at 11:34
  • Note that the standard is not 100% clear about this. The character range definitions exclude the 0 byte but some other texts say that any character below 127 is valid. – Aaron Digulla Feb 02 '09 at 12:29
  • Also, Carriage Return are substituted in XML CDATA. http://stackoverflow.com/questions/1437874/parsing-xml-cdata-with-xmllite – rwong May 29 '13 at 17:04
  • 1
    @rwong they are preserved if escaped, but you cannot do that with nul. – Pete Kirkham May 30 '13 at 15:15
5

XML is a plain-text format - don't use it to store binary data. Put the binary blobs in separate files and add an element to your XML which references these files. If you want to store all binary blobs in a single file, add an offset attribute or something like that...

Christoph
  • 164,997
  • 36
  • 182
  • 240
-1

You can store it as CDATA, but there's the risk that some byte sequences will evaluate to valid XML that closes the CDATA section. After a quick look at http://www.w3.org/TR/2006/REC-xml-20060816/#sec-cdata-sect, it seems you can have any sequence of chars except "]]>". Have a look at what is a valid XML char too.

Joao da Silva
  • 7,353
  • 2
  • 28
  • 24
  • 2
    Doesn't that mean no you can't, since 0-8,B,C,E,F FFFE, and FFFF are invalid characters? – David Sykes Apr 28 '10 at 10:32
  • 1
    Downvoting because there are many byte sequences that won't be preserved. By "not preserving", I mean that it's not possible to recover the original binary data from CDATA-encoded data. See Pete's answer and comments. – rwong May 29 '13 at 17:06