Concatenate multiple Markdown files using Pandoc on Windows

Question

I have several Markdown (.md) files in a folder and I want to concatenate them and get a final Markdown file using Pandoc. I wrote a bash file like this:

#!/bin/bash 
pandoc *.md > final.md

But I am getting the following error when I double-click on it:

pandoc: *.md: openBinaryFile: invalid argument (Invalid argument)

and the final.md file is empty.

If I try this:

pandoc file1.md file2.md .... final.md

I am getting the results I expect: a final.md file with the contents of all the other Markdown files.

On macOS it works fine. Why doesn't this work on Windows?

`ls *.md` gives back the 3 md files that i have for the testing. file1.md, file2.md and file3.md — ziselos, Apr 24 '18 at 08:19
And how are you running your script? It looks like your shell is failing to expand the file glob for some reason… — ChrisGPT was on strike, Apr 24 '18 at 11:01
Ah. I guess you're on Windows? Are you using cmd.exe or PowerShell? — ChrisGPT was on strike, Apr 24 '18 at 11:43
@Chris I just found the solution. Based on your comment, i tried and run it via Cygwin like `bash run.sh` and the final file now has content. I do not know why, but if i try double click the file in current directory, i am getting file with no content — ziselos, Apr 24 '18 at 12:07

score 2 · Accepted Answer · answered Apr 24 '18 at 13:33

On Unix-like shells (like bash, for which your script is written) glob expansion (e.g. turning *.md into file1.md file2.md file3.md) is performed by the shell, not the application you're running. Your application sees the final list of files, not the wildcard.

However, glob expansion in cmd.exe is performed by the application:

The Windows command interpreter cmd.exe relies on a runtime function in applications to perform globbing.

As a result, Pandoc is being passed a literal *.md when it expects to see a list of files like file1.md file2.md file3.md. It doesn't know how to expand the glob itself and tries to open a file whose name is *.md.

You should be able to run your bash script in a unix-like shell like Cygwin or bash on Windows. It may also work on PowerShell, though I don't have a machine handy to test. As a last resort you could jump through some hoops to write a batch file that expands the glob and passes file names to Pandoc.

This works with PowerShell: `pandoc (get-item *.md).FullName -o final.md` — gijswijs, Oct 01 '18 at 07:02
I wrote a plain old batch file for Windows that will operate recursively on a directory and its subdirectories looking for files of a certain type and will then send them to pandoc. I call it `pancompile.bat`. The code is in this answer: https://stackoverflow.com/questions/41471253/how-can-i-use-pandoc-for-all-files-in-the-folder-in-windows/52883778#52883778 . No doubt it could be rewritten much better in PowerShell, but it works for me. — Jaifroid, Oct 18 '18 at 23:18

Concatenate multiple Markdown files using Pandoc on Windows

1 Answers1

Linked