12

Recently I have downloaded emacs on my system to edit large files as it claims. When trying opening large files, the emacs gives me a problem that buffer size is exceeded.

Is there a way to increase it?! I have a system with memory of 24 GB!

Mohamad Ibrahim
  • 5,085
  • 9
  • 31
  • 45
  • 1
    will [this](http://www.emacswiki.org/emacs/EmacsFileSizeLimit) help? also, see this http://stackoverflow.com/questions/159521/text-editor-to-open-big-giant-huge-large-text-files – Fredrik Pihl Dec 02 '11 at 08:25
  • @Fredrik I passed over it but I do not know how to install it. help?! – Mohamad Ibrahim Dec 02 '11 at 08:27
  • If you like me needed to change (not add or remove) text in a huge file, `hexedit` may be the answer (just changed a 100 GB file). – Ole Tange Mar 10 '20 at 16:50

4 Answers4

9

I've never used it but vlf.el is an elisp hack to view overly large files. I believe it uses insert-file-contents, but reads from an offset so as to do something akin to pagination.

There are some notes on the wiki but no solutions apart from vlf.el which I've linked to above.

Aaron Miller
  • 3,692
  • 1
  • 19
  • 26
Noufal Ibrahim
  • 71,383
  • 13
  • 135
  • 169
5

Type M-:(log most-positive-fixnum 2) to see whether the return value is closer to 64 or 32. If the latter, compile or get yourself a 64-bit Emacs. Also look at M-x find-file-literally if large files cause sluggishness; it should disable some time consuming operations such as font-lock-mode.

huaiyuan
  • 26,129
  • 5
  • 57
  • 63
3

Are you running the 64-bit version of emacs? Check with file, you should get something like this:

$ file /usr/bin/emacs
/usr/bin/emacs: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), for GNU/Linux 2.6.9, stripped
Mario
  • 1,801
  • 3
  • 20
  • 32
  • ON the start up page of the program I am reading this: This is GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.16.6) of 2010-04-01 on x86-06.phx2.fedoraproject.org – Mohamad Ibrahim Dec 02 '11 at 08:31
2

Why do you want to edit huge 14Gb files? I believe it is wrong to want to do that.

You don't have the human time to read all (or even most) of such a huge file. What kind of file is it? (some log file, some MySQL dump file???)

So you are searching some specific pattern inside that file. Do yourself a favor, write a script (or even a program) to do that, and to edit (if relevant) your big file where needed...

Very probably, you don't care of most of the content of that huge file. Only a few lines matter to you.

Or split the file into smaller pieces (e.g. with csplit), and edit the relevant smaller chunks, then rebuild the huge file with cat

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • 15
    That may be exactly the use case - search a small pattern, edit that, and save the whole stuff back. I don't see what is wrong in thinking a editor could be the appropriate tool for this. Isn't it a flaw in the editor if you have to resort back to splitting, editing and rebuilding? After the all the editor could do exactly that under the hood. – Gunther Piez Dec 02 '11 at 09:16
  • An editor is designed to be able to edit any line or character in the file, so it has to maintain internal data structures bigger than the edited file; I don't feel it is a flaw in editors, but a required feature in them; and of course, it does have some cost. – Basile Starynkevitch Dec 02 '11 at 09:18
  • 4
    I don't see the ability to edit every line as a flaw - but your assumption that the internal structures needed to be able to edit every line must be bigger than the whole file may be wrong. The editor could just hold a kind of 'diff file' with additional pointers (which may be very small), and still be able to edit every line. And switch back to 'standard mode' if the file is small or the changes are frequent. But the inability to handle a large file is certainly a flaw – Gunther Piez Dec 02 '11 at 09:25
  • 7
    BTW, I had my share of fighting with large files too: Some very large (about 100M lines) files with PGN (chess) data, with some (few) not correctly closed parentheses in them. This needed to be fixed, otherwise the data wouldn't be correctly read by the statistical analyzer. Short story: Emacs just couldn't handle it, vi could, but was unexpected slow, kate (the kde thingy) worked fast after switching of syntax highlighting – Gunther Piez Dec 02 '11 at 09:36
  • @BasileStarynkevitch: Not too sure why you questioned the requirement to open a big file in editor. If I want to open a big file, it might not be because I want to read it all. I recently used Xemacs and it took quite a while to open a 150MB file. Yeah, you could say I should use grep or whatever to do search for text, but I want to open the file in an editor and do the search there because I want to look at the context of each occurrence as well. The other day I needed to look at a 700MB log file. I ended up using UltraEdit which can open a file of any size. – boggy Nov 23 '15 at 23:06
  • emacs should be capable to do everything. – Daniil Iaitskov May 04 '23 at 20:36