0

Platform: OS X Mountain Lion
Python 2.7.4

I have a piece of script that needs to convert all eol characters in a string to unix style \n. I get a string supplied by Qsci.Scintilla.text() referenced to below as txt. 

print 'original text: %s' % repr(unicode(txt))
print 'linesep: %s' % repr(os.linesep)
print 'fixed text: %s' % repr(unicode(txt).replace(os.linesep, '\n'))

This shows output like:

original text: u'exp.cnvs.show()\rself.sleep(1000)'
linesep: '\n'
fixed text: u'exp.cnvs.show()\rself.sleep(1000)'

All eol characters in there are reported as \r (which is correct for OS X af far as I know). Still os.linesep reports \n to be the eol character, causing the bottom statement to do nothing. The output of that statement of course should have been:

fixed text: u'exp.cnvs.show()\nself.sleep(1000)'

Does anyone know why os.linesep appears to report the wrong eol character on OSX?

Daniel Schreij
  • 773
  • 1
  • 10
  • 26

1 Answers1

2

No, \r (carriage return, or CR) is only correct for old Mac OS, so up to release 9.

OS X is a complete rewrite, and now a UNIX-based OS. It uses \n (line feed, or LF).

From Wikipedia on the subject of Newline:

  • LF: Multics, Unix and Unix-like systems (GNU/Linux, Mac OS X, FreeBSD, AIX, Xenix, etc.), BeOS, Amiga, RISC OS and others.

[...]

  • CR: Commodore 8-bit machines, Acorn BBC, ZX Spectrum, TRS-80, Apple II family, Mac OS up to version 9 and OS-9

See Historical reason behind different line ending at different platforms for some history behind this.

Instead, it's Scintilla that is in the wrong here. The Line endings documenation suggests it defaults to \r when on Mac:

Scintilla can interpret any of the Macintosh (\r), Unix (\n) and Windows (\r\n) line ends.

You can use the SCI_SETEOLMODE() function to alter that behaviour, whatever the Python binding equivalent is should be passed the os.linesep value to make Scintilla use the correct line separator for your platform.

I see that there is a QsciScintilla.setEolMode() method and a EOLMode enum to accomplish that; I guess you'll need to map python line ending variants to the enum and pass the right enum value to the .setEolMode() method.

Community
  • 1
  • 1
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Alright, that makes sense. Why would Qscintilla then still return the text with a \r as the newline character? – Daniel Schreij May 15 '13 at 14:31
  • @DanielS: Perhaps that's a bug in Qscintilla? – Martijn Pieters May 15 '13 at 14:32
  • @DanielS: Ah, Scintilla is rather dated; the [documentation I found](http://www.scintilla.org/ScintillaDoc.html#LineEndings) indeed suggests `\r` is used regardless of Mac OS version. Setting `SCI_SETEOLMODE` to `os.linesep` seems to be a good work-around. – Martijn Pieters May 15 '13 at 14:40
  • Thanks! I found the documentation on the SCI EOL methods and it appears to work fine! Strange that QScintilla upholds these old standards, but it least it can be solved. – Daniel Schreij May 16 '13 at 11:21