2

At the moment, I have a process-buffer which is utf-8-auto (emacs modeline reports the buffer as utf-8-auto-dos) with CRLF style newlines. When I write multi-line text into the buffer via a process-send-region or process-send-string each line is suffixed with ^M.

What makes this problem odd is that text written to the process-buffer directly from the process, does not contain ^M's.

It doesn't seem to make any difference where the source text comes from, in fact, even a multi-line region marked and sent that already appears in the process buffer (that doesn't contain ^M) will have them when sent.

(Note the source text for the process-send-region will always come from a Emacs buffer, process-send-string, when multi-line will be from the Windows clipboard interface to the killring, or again from an Emacs buffer to killring.)

I should also add that the incoming text to the buffer is parsed by a after-change-functions hook (to do some colorisation based on input) so a last resort I'd do an additional regexp-replace-in-string on this incoming text as part of that hook function, I'd like to avoid that because it seems wrong, but I'll add it as a hacky solution if nothing else works.

Addendum

I updated the encoding settings for the buffer and the process to use utf-8-dos instead of utf-8-auto and the ^M's vanished.

So in the buffer setup part of my app, I did...

(switch-to-buffer "sock-buffer")
(set-process-coding-system (get-process sock-process) 'utf-8-dos 'utf-8-dos)
(set-buffer-file-coding-system 'utf-8-dos nil)
(set-buffer-process-coding-system 'utf-8-dos 'utf-8-dos)

Then reduced this to just...

(switch-to-buffer "sock-buffer")
(set-buffer-process-coding-system 'utf-8-dos 'utf-8-dos)

And everything worked fine.

ocodo
  • 29,401
  • 18
  • 105
  • 117
  • Please note the update, I've refined the question to the specific use case, with a process buffer, in this case the `dos2unix` approach is not relevant, however, I am able to process this text as it's inserted using emacs-lisp. – ocodo Nov 07 '10 at 22:40

3 Answers3

3

This is because those files are in DOS/Windows line endings. You can use C-x [Enter] f unix [Enter] to convert them to the Unix encoding.

^L is a page break. I've seen them some times to separate different parts of source code (for old-fashioned listings in a text printer), or in text documentation to insert an actual "new page" command.

As of the update, here you can see that you have to select set-process-coding-system to the correct coding system.

Diego Sevilla
  • 28,636
  • 4
  • 59
  • 87
1

Alternately to the dos2unix approach, you could use one of the MULE commands in Emacs, or (my favorite), since these characters are mistakenly treated as part of the text, you can replace them using the command to replace a string in the text: M-% C-q C-M RETURN

M-% is the query-replace command.

C-q means "let me type the next character without interpreting it as the RETURN key".

Pascal Cuoq
  • 79,187
  • 7
  • 161
  • 281
  • @slomojo If you do not want to do this interactively use `replace-string` instead of `query-replace`. Did you read the answer before copy-pasting your comment? – Pascal Cuoq Nov 07 '10 at 22:50
  • I'll have a look at the `replace-string` method of doing this after I've tried `set-process-coding-system`. – ocodo Nov 07 '10 at 22:55
  • Because the process-buffer has an `'after-change-functions` hook, I needed to do the replace as a `regexp-replace-in-string` - this is what I suspected I'd need to do, I was just hoping there was an encoding setting I could've used to avoid this overhead. - I'll give Diego's encoding methods a try before marking the 'correct' answer. – ocodo Nov 07 '10 at 23:37
0

I believe you see those because of the inconsistencies in your newlines (e.g. windows newlines vs *nux ones), you should probably try dos2unix

Soufiane Hassou
  • 17,257
  • 2
  • 39
  • 75