0

I'm not much of a Python dev, so I'm sorry if there is not much sense in this question..

I'm currently working in a project that contains a py script that communicates with a server written in Java. A considerable part of the script is dedicated to find where the server jar is located based on the current work directory (cwd) from where the script is being executed. I though that I could pack both, the script and the server jar, in some sort of zip that I could distribute and execute in a standalone fashion, removing the whole logic to fetch the jar from the script. Is it possible to have that?

I've been trying to use BUCK python_binary for this purpose, however, it doesn't seem I can add jar as a dependency for the pex.

EDIT 1

Following @sdwilsh suggestion I was able to use BUCK to create a pex containing the jar, however, whenever I try to access it from the main python script, it fails because the jar cannot be found:

BUCK file

java_library(
  name = 'src_main',
  srcs = glob(['src/main/java/**/*.java']),
  source = '8',
  target = '8',
  visibility = [
    'PUBLIC',
  ],
)

java_binary(
  name = 'sample_jar',
  main_class = 'br.com.samples.jar.Main',
  deps = [':src_main'],
  visibility = [
    'PUBLIC',
  ],
)

python_library (
  name = 'jarlib',
  resources = [':sample_jar'],
)

python_binary(
  name = 'wrapper',
  main = 'wrapper.py',
  deps = [':jarlib'],
)

PEX content

-rw-r--r--  1 staff    1319 Mar 28 13:07 sample_jar
-rw-r--r--  1 staff     259 Mar 28 13:15 wrapper.py
-rw-r--r--  1 staff     308 Mar 28 13:15 wrapper.pyc
-rw-r--r--  1 staff     917 Mar 28 13:15 __main__.pyc
-rw-r--r--  1 staff     737 Mar 28 13:15 __main__.py
-rw-r--r--  1 staff     242 Mar 28 13:15 PEX-INFO

wrapper.py

import sys, subprocess

process = subprocess.Popen(['java', '-jar', 'sample_jar'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print process.communicate()

Output for wrapper.py

('', 'Error: Unable to access jarfile sample_jar\n')

EDIT 2

So it seems we cannot reference the jar file from the PEX env. A workaround is to unpack the pex resources in a tmp folder and execute everything from there.

Humble Student
  • 3,755
  • 4
  • 20
  • 35

1 Answers1

2

You can, but you'd need to use a python_library and add the rule that produces the java_binary in the resources argument.

sdwilsh
  • 4,674
  • 1
  • 22
  • 20
  • I'll try it and post a complete solution and accept your answer if it works. Thank you very much – Humble Student Mar 27 '18 at 15:27
  • Hi @sdwish, could you please take a look on the edit I've made in the question? – Humble Student Mar 28 '18 at 11:40
  • 1
    Per the docs for `python_library`, you need to use the `pkg_resources` API to access the jar: http://setuptools.readthedocs.io/en/latest/pkg_resources.html – sdwilsh Mar 28 '18 at 15:50
  • So in case I need to ship jars along with pyspark pex, I cannot access from sparkconfig within pex or pkg_resources can do this? – kensai May 08 '19 at 15:54