How to get the placeholder's value which is stored in a different file (same directory) using JSch exec

Question

With the conditions:

I cannot use any XML parser tool as I don't have permission , read only
My xmllint version does not support xpath, and I cannot update it , read only
I dont have xmlstarlet and cannot install it
I run my script using Java JSch exec channel ( I have to run it here )

So we have 3 files in a directory.

sample.xml
values1.properties
values2.properties

The contents of the files are as follows:

Sample.xml

<block>
 <name>Bob</name>
 <address>USA</address>
 <email>$BOB_EMAIL</email>
 <phone>1234567</phone>
</block>

<block>
 <name>Peter</name>
 <address>France</address>
 <cell>123123123</cell>
 <drinks>Coke</drinks>
 <car>$PETER_CAR</car>
 <bike>Mountain bike</bike>
</block>

<block>
 <name>George</name>
 <hobby>$GEORGE_HOBBY</hobby>
 <phone>$GEORGE_PHONE</phone>
</block>

values1.properties

JOE_EMAIL=joe@google.com
BOB_EMAIL=bob@hotshot.com
JACK_EMAIL=jack@jill.com
MARY_EMAIL=mary@rose.com
PETER_EMAIL=qwert1@abc.com
GEORGE_PHONE=Samsung

values2.properties

JOE_CAR=Honda
DAISY_CAR=Toyota
PETER_CAR=Mazda
TOM_CAR=Audi
BOB_CAR=Ferrari
GEORGE_HOBBY=Tennis

I use this script to get the xml block to be converted to a properties file format

NAME="Bob"
sed -n '/name>'${NAME}'/,/<\/block>/s/.*<\(.*\)>\(.*\)<.*/\1=\2/p' sample.xml

OUTPUT:

name=Bob
address=USA
email=$BOB_EMAIL
phone=1234567

How do I get the value of $BOB_EMAIL in values1.properties and values2.properties. Assuming that I do not know where it is located between the two (or probably more) properties file. Bacause it should work differently if I entered

Name=Peter

in the script, it should get

name=Peter
address=France
cell=123123123
drinks=Coke
car=$PETER_CAR
bike=Mountain bike

and the think that will be searched will be PETER_CAR

EXPECTED OUTPUT (The user only needs to input 1 Name at a time and the output expected is one set of data in properties format with the $PLACEHOLDER replaced with the value from the properties file):

User Input: Name=Bob

name=Bob
address=USA
email=bob@hotshot.com
phone=1234567

User Input: Name=Peter

name=Peter
address=France
cell=123123123
drinks=Coke
car=Mazda
bike=Mountain bike

Ultimately, the script that I need has this logic:

for every word with $
in the result of sed -n '/name>'${name}'/,/<\/block>/s/.*<(.*)>(.*)<.*/\1=\2/p' sample.xml ,
it will search for the value of that word in all of the properties file in that directory(or specified properties files),
then replace the word with $ with the value found in the properties file

PARTIALLY WORKING ANSWER:

Walter A's answer is working in cmd line (putty) but not in Jsch exec. I keep getting an error of No value found for token 'var' .

How about `x=$(sed -n '/name>'${NAME}'/,/<\/block>/s/.*<$.*$>$.*$<.*/\1=\2/p' sample.xml); echo "$x"` — Walter A, Jun 24 '15 at 21:20
@Walter it still displayed the value with placeholder, the values that I need there are in the properties file — Philip Morris, Jun 24 '15 at 22:07
Your conditions are making it very difficult to perform your job. — glenn jackman, Jun 25 '15 at 14:01
@glennjackman what do I need to parse through properties files and look for the placeholders value? — Philip Morris, Jun 25 '15 at 14:42
@glenjackman I had a workaround with that. My problem now is how to populate the values of the placeholders — Philip Morris, Jun 25 '15 at 22:00

Walter A · Accepted Answer · 2015-06-28T09:12:50.747

1

The solution beneath will look in the properties files a lot of times, so I think there is a faster solution for the problem.
The solution beneath will get you started and with small files you might be happy with it.

# Question has a bash en ksh tag, choose the shebang line you want
# Make sure it is the first line without space or ^M after it.
#!/bin/ksh
#!/bin/bash
# Remove next line (debugging) when all is working
# set -x
for name in Bob Peter; do
   sed -n '/name>'${name}'/,/<\/block>/s/.*<\(.*\)>\(.*\)<.*/\1=\2/p' sample.xml |
      while IFS="\$" read line var; do
         if [ -n "${var}" ]; then
            echo "${line}$(grep "^${var}=" values[12].properties | cut -d= -f2-)"
         else
            echo "${line}"
         fi
      done
   echo
done

EDIT: Commented two possible shebang lines, set -x and added output.

Result:

name=Bob
address=USA
email=bob@hotshot.com
phone=1234567

name=Peter
address=France
cell=123123123
drinks=Coke
car=Mazda
bike=Mountain bike

edited Jun 28 '15 at 09:12

answered Jun 25 '15 at 08:44

Walter A

19,067
2
23
43

Im getting No value found for token 'var' – Philip Morris Jun 25 '15 at 09:49
can you explain to me where var is populated here? – Philip Morris Jun 25 '15 at 14:18
I can not reproduce your problem. I tested it with bash but tried to use portable commands only. In the line `while IFS="\$" read line var` the variable var is filled when the line has a $ character or stays empty when there is no $. The next line should check the token. I will reverse the check (not -z but -n), maybe that helps. Some info about checking empty variables: [checking vars](http://stackoverflow.com/questions/307503/whats-the-best-way-to-check-that-environment-variables-are-set-in-unix-shellscr). – Walter A Jun 25 '15 at 15:07
-n did not work also. hmm . what is wrong . . I execute the script using Jsch's exec channel. – Philip Morris Jun 26 '15 at 01:26
Ahh. Jsch might do or omit other things. 1. Please test my script from the command-line. 2. You can add `#!/bin/ksh` or `#!/bin/bash` as the first line, that might overrule Jsch specific behaviour. – Walter A Jun 26 '15 at 07:46
You might update your tags, I didn't know you wanted to call this using Java. When you already have a Java application, consider solving this in Java. A Java developer who needs to change your code will get really confused. – Walter A Jun 26 '15 at 07:48
The #!/bin/ksh and #!/bin/bash did not do the trick. What is the difference when I execute this in cmd and in JSch exec? There were no issues since this one – Philip Morris Jun 27 '15 at 02:01
The [shebang](https://en.wikipedia.org/wiki/Shebang_%28Unix%29) tries to force a kind of shell to execute this. It is ignored when the code is [sourced](http://stackoverflow.com/questions/9326695/what-does-it-mean-to-source-a-file-in-unix-linux-context) – Walter A Jun 27 '15 at 06:33
Try executing it from cmdline and see if it works there. I also added a `set -x` debugging option you can try when the cmdline also fails. The only reference to your errormessage I find with Google is [in some Java code](https://github.com/sqh4/debug/blob/master/NumberToken.java) so I think the script itself is working and your problem is how you look at the results in Java. – Walter A Jun 27 '15 at 06:41
I tried running it in cmd. It returned everything and not just the values I need – Philip Morris Jun 28 '15 at 05:37
No error with cmd, thats fine. It returned everything? Is that the `set -x` debug info? I show above my output, can you update the question if this is incorrect? – Walter A Jun 28 '15 at 09:14
It is returning everything with a placeholder ($) . I am running it in a bigger directory with bigger files (I just simplified it with the question). But the problem here is that I only need the placeholder that is present in the output of the XML to properties format. (not all) – Philip Morris Jun 28 '15 at 12:56
you have the best answer so far , only need a few more twicks – Philip Morris Jun 28 '15 at 13:08
Then remove the two lines `else` and `echo "${line}"` – Walter A Jun 28 '15 at 13:49
It is still returning more data than I need. Lets just say there are 1000 more blocks with different names and placeholders inside. So I just need the specific ones in which my `sed` script returns. You can check the "EXPECTED OUTPUT" section in my question. – Philip Morris Jun 28 '15 at 14:11
With `for name in Bob Peter; ` I show data for Bob and Peter. I get the expected output for Bob, add an empty line and show the expected output for Peter. You can change it into `while read name` or `name=$1` when you want to enter the name differently. – Walter A Jun 28 '15 at 19:31
It did not work, still displaying many data. Can you try making your xml file larger by placing new entries and placeholders? So you can replicate the one I am facing with the real files – Philip Morris Jun 28 '15 at 23:57
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/81846/discussion-between-walter-a-and-philip-morris). – Walter A Jun 29 '15 at 08:53

score 1 · Answer 2 · edited Jun 28 '15 at 17:17

1

. values1.properties
. values2.properties
sed -n '/name>'${NAME}'/,/<\/block>/s/.*<\(.*\)>\(.*\)<.*/echo \1="\2"/p' sample.xml >output
. output

Dangerous, and not the way I would prefer to do it.

edited Jun 28 '15 at 17:17

anishsane

20,270
5
40
73

answered Jun 25 '15 at 22:56

Mel

111
1

Im getting Exit Code: 1 – Philip Morris Jun 26 '15 at 01:13
The . sources the file. Basically executes it in your current shell, which assigns all the variables.. ie PETER_EMAIL, etc. Your variables are then expanded when the output is sourced. – Mel Jun 26 '15 at 03:48
Im using JSch's exec channel to execute my script. Is this the problem? – Philip Morris Jun 27 '15 at 01:34
I tried something like this first, but had problems with `bike=Mountain bike`. Additionally vars containing quotes will also be evaluated. – Walter A Jun 27 '15 at 06:28
Sourcing was a cool way! But dangerous too... +1 since you have included the warning too.. – anishsane Jun 28 '15 at 05:43
$ -ksh: output: cannot create [Permission denied] – Philip Morris Jun 28 '15 at 13:05

score 0 · Answer 3 · answered Jun 28 '15 at 05:35

0

A sed based version:

$ temp_properties=`mktemp`
$ NAME=Bob
$ sed '/./{s/^/s|$/;s/=/|/;s/$/|g/}' values*.properties > $temp_properties
$ sed -n '/name>'${NAME}'/,/<\/block>/s/.*<\(.*\)>\(.*\)<.*/\1=\2/p' sample.xml | sed -f $temp_properties

Gives:

name=Bob
address=USA
email=bob@hotshot.com
phone=1234567

It does have issues of script injection. However, if you trust the values*.properties files & contents of NAME variable, you are good to go.

answered Jun 28 '15 at 05:35

anishsane

20,270
5
40
73

Did you by any chance miss out any character in above snippet? These command run as-are on my system. – anishsane Jun 28 '15 at 09:57
sed: file /tmp/tmp.YUHfVm4670 line 229: unterminated `s' command - Still getting this – Philip Morris Jun 28 '15 at 13:06
Does your file have `crlf` line endings? check `file values*.properties`... If yes, you can change using `dos2unix filename`... – anishsane Jun 28 '15 at 17:15
^^ Does your file have `crlf` line endings? I could still have got alternate ways, if you had also answered that... :D – anishsane Jun 29 '15 at 00:57
`sed '/./{s/\r//;s/^/s|$/;s/=/|/;s/$/|g/}' values*.properties > $temp_properties` – anishsane Jun 29 '15 at 00:58
I see a normal properties file with key value pairs inside. no crlf line endings – Philip Morris Jun 29 '15 at 08:19

How to get the placeholder's value which is stored in a different file (same directory) using JSch exec

3 Answers3