16

There is a command pdflatex, which I want to use in my bash script. It takes as input a filename on which content it will work.

Now I have an algorithms that looks as follows:

for stuff in list of stuff; do
  echo "${stuff}" > /tmp/tmpStuff
  pdflatex /tmp/tmpStuff
done

Now this works as expected. But I was thinking I could speed that up by doing less disk I/O(as > redirection writes to a file). I wish I could write something like echo "$stuff" | pdflatex /tmp/tmpStuff but pdflatex uses a file and not stdin as its input. Is there any way of keeping "$stuff" in memory and passing it to pdflatex as a sort of file?

TLDR: I would be happy if I could create a temporary file which could be named and be in memory.

Simonlbc
  • 591
  • 1
  • 4
  • 16

4 Answers4

24

You can use process substitution for this:

pdflatex <(echo "$stuff")

From the Bash Reference Manual:

3.5.6 Process Substitution

Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files. It takes the form of

<(list)

or

>(list)

The process list is run with its input or output connected to a FIFO or some file in /dev/fd. The name of this file is passed as an argument to the current command as the result of the expansion. If the >(list) form is used, writing to the file will provide input for list. If the <(list) form is used, the file passed as an argument should be read to obtain the output of list. Note that no space may appear between the < or > and the left parenthesis, otherwise the construct would be interpreted as a redirection.

And I also wonder if a here-string would make it as well:

pdflatex <<< "$stuff"
fedorqui
  • 275,237
  • 103
  • 548
  • 598
  • Thank you for your answer, is there any way of naming the file resulting file "temporary" file coming from `<(echo "$stuff")` ? I'm wondering that because it seems to cause errors in pdflatex. – Simonlbc Aug 30 '16 at 14:52
  • 1
    @Simonlbc no, not really. If you want this, you may want to take a look to [mktemp](http://stackoverflow.com/a/10983009/1983854). – fedorqui Aug 30 '16 at 14:53
  • if I create a file with mktemp, will the file be in memory? – Simonlbc Aug 30 '16 at 15:25
  • 1
    @Simonlbc You can check creating a named pipe as explained in [Pseudo files for temporary data](http://unix.stackexchange.com/a/63933/40596). However, it seems better to use process substitution. – fedorqui Aug 30 '16 at 15:30
  • 1
    Thank you for your help. It seems echo `"$stuff" | pdflatex -jobname outputPdfName`seems to work after all. I tried `pdflatex <<< "$stuff" -jobname outputPdfName` too and it seems to works without errors too, which is nice. So I guess `<<< "$stuff"` is like redirecting a string to stdin, am I right(I'm just genuinely curious)? – Simonlbc Aug 30 '16 at 15:50
  • @Simonlbc you are absolutely right :) See [here strings](https://www.gnu.org/software/bash/manual/bash.html#Here-Strings) in the Bash manual. – fedorqui Aug 30 '16 at 19:34
6

Many shells, and Linux as a whole, accept:

echo "${stuff}"  | pdflatex /dev/stdin
Gilbert
  • 3,740
  • 17
  • 19
  • Thank you for your answer. I'm just wondering why you would at the same time use "|" symbol and at the same time precise /dev/stdin is there a reason for that? – Simonlbc Aug 30 '16 at 15:52
  • 2
    http://stackoverflow.com/users/5432899/simonlbc: /dev/stdin was added just for programs that don't have a specific way to access stdin... your post implies pdflatext is one of them. In the simple case it is also easier than process substitution. Use of the longer name can also be useful, and self documenting, at times. FYI: The other two of the /dev/std* family also exist. – Gilbert Aug 30 '16 at 21:18
6

None of the presented solutions is guaranteed to work on all possible commands.

Consider myCommand below (instead of pdflatex):

#!/bin/bash
# myCommand
test -f "$1" || { echo "error: Please provide a file"; exit 1; }
echo "The file content is:"$(cat $1)

Trying the existing solutions:

./myCommand <(echo "abcdef")
# error: Please provide a file

echo "abcdef" | ./myCommand /dev/stdin
# error: Please provide a file

In my Ubuntu I use the following approach (utilizing /dev/shm in-memory file storage):

ramtmp="$(mktemp -p /dev/shm/)"
echo "abcdef" > $ramtmp
./myCommand $ramtmp
# The file content is:abcdef

So in your case it would be:

ramtmp="$(mktemp -p /dev/shm/)"
for stuff in list of stuff; do
  echo "${stuff}" > $ramtmp
  pdflatex $ramtmp
done

In case pdflatex execution is non-blocking add the first line inside the loop.

Note that, I'm not sure how many distributions currently support it. See Wikipedia article:

Linux distributions based on the 2.6 kernel and later offer /dev/shm as shared memory in the form of a RAM disk, more specifically as a world-writable directory (a directory in which every user of the system can create files) that is stored in memory. Both the RedHat and Debian based distributions include it by default. Support for this type of RAM disk is completely optional within the kernel configuration file.[5]

Marinos An
  • 9,481
  • 6
  • 63
  • 96
  • I've asked a question specifically about something that works in *all* cases: https://stackoverflow.com/questions/67050503/standard-linux-utility-similar-to-bash-process-substitution-to-provide-output-of – Peter V. Mørch Apr 11 '21 at 21:50
1

Another solution is to use a ramdisk, a filesystem that lives in ram. I always compile my LaTeX on a ramdisk to save disk writes, and also use one for Portage software builds on Gentoo for the same reason.

Here's how you create one:

Linux: sudo mkdir /mnt/ramdisk sudo mount -t tmpfs -o size=2G ramdisk /mnt/ramdisk

feel free to change the mount point to anything. a popular option is /tmp/tmpfs or /tmp/ram

macOS: diskutil erasevolume HFS+ "ramdisk" `hdiutil attach -nomount ram://$((sizeInGB*1024*2048))`

this example will mount on /Volumes/ramdisk

if you wish to use MB replace the end with ram://$((sizeInMB*2048))

Ryan Knutson
  • 108
  • 2
  • 14