How to convert a shell script (bash) to Scala code?

Question

I have a shell script with series of defined steps. I am trying to convert the script to Scala code by maintaining the order of steps. Basically I want my Scala code to be a mirror of the shell script.

I've used sys.process._ library. The shell script has commands like:

mkdir <pathToDir>
hadoop fs -copyToLocal <source> <dest>
rm -r <pathToDir>
java -jar <someScript>

I need to maintain the order of execution of these steps.

I tried something like:

import sys.process._
Class A {
  def processMethod(): Unit = { 
    Process("mkdir -p <dir1>") #| Process("hadoop fs -copyToLocal 
    <hdfsDir1> <localDir1>") #| Process("mkdir -p <dir2>") #| 
    Process("hadoop fs -copyToLocal <hdfsdir2>/*.parquet <localDir2>") 
    #| Process("java -jar <pathToJarFile> -script <sampleScripts> 1>&2") 
    #| Process("rm -r<localDir1>") #| Process("rm -r <localDir2>") ! 
  }
}

I'm expecting the operations to execute in the order they have been defined. I'm confused about how ProcessBuilder/Process works or if there is an alternative to convert this whole thing to Scala code?

Ammonite bills itself as bash replacement: http://ammonite.io/#Ammonite-Shell — som-snytt, Jan 31 '19 at 08:21

ymonad · Accepted Answer · 2019-01-31T02:36:13.943

According to the document, #| constructs a command that will run command and pipes the output to other. Thats being said, the following code in Scala:

(Process("echo hello") #| Process("wc -c")).!

is equivalent to following Bash code:

echo hello | wc -c

which is not what you want.

What you are looking for is ### operator which constructs a command that will run one command and then other.

Using this operator, you can do write following Scala code:

(Process("ls") ### Process("echo hello")).!

Which is equivalent to following Bash code:

ls
echo hello

However, note that using Process in the way above is not completely bash equivalent because it cannot change current directory with cd nor use bash syntax such as if, for, case.

If you really want bash equivalent code, the only way is running the script with bash.

Ammonite has a notion of working dir. http://ammonite.io/#Ammonite-Shell — som-snytt, Jan 31 '19 at 08:27

score 2 · Answer 2 · answered Jan 31 '19 at 10:05

In addition to the ### method, as @ymonad has pointed out, you should really be using the ProcessBuilder methods more directly. It makes the code easier to read and understand.

import sys.process._

Seq("mkdir", "SO")        ###
Seq("touch", "SO/file")   ###
Seq("mv", "SO", "SoWhat") !

Using a Seq[String] in place of a simple String offers some advantages when parsing multiple arguments passed to the process.

score 2 · Answer 3 · answered Jan 31 '19 at 10:53

I haven't digged into this project so I can't help you with a first-hand example, but have a look at the Ammonite Scala project.

Quoting from the home page:

Ammonite lets you use the Scala language for scripting purposes: in the REPL, as scripts, as a library to use in existing projects, or as a standalone systems shell.

...

The goal of Ammonite is to liberate your Scala code from heavyweight "projects", using the lightweight Ammonite runtime: if you want to run some Scala, open the Ammonite-REPL and run it, interactively! If you want to run it later, save it into some Scala Scripts and run those later.

At this link are some examples of what you can do:

import ammonite.ops._

// Let's pick our working directory
val wd: Path = pwd/'ops/'target/"scala-2.11"/"test-classes"/'example3

// And make sure it's empty
rm! wd
mkdir! wd

// You can create folders through `mkdir!`. This behaves the same as
// `mkdir -p` in Bash, and creates and parents necessary
val deep = wd/'this/'is/'very/'deep
mkdir! deep

// You can also use backticks to execute commands which aren't valid Scala identifiers, e.g.
`ssh-add`
Enter passphrase for /Users/haoyi/.ssh/id_rsa:

How to convert a shell script (bash) to Scala code?

3 Answers3