0

Forgive me the possibly trivial question, but: How do I run the script published by pybuilder?


I'm trying to follow the official Pybuilder Tutorial.

I've walked through the steps and successfully generated a project that

  • runs unit tests
  • computes coverage
  • generates setup.py
  • generates a .tar.gz that can be used by pip install.

That's all very nice, but I still don't see what the actual runnable artifact is?

All that is contained in the target directory seems to look more or less exactly the same as the content in src directory + additional reports and installable archives.

The tutorial itself at the end of the "Adding a runnable Script"-section concludes that "the script was picked up". Ok, it was picked up, now how do I run it? At no point does the tutorial demonstrate that we can actually print the string "Hello, World!" on the screen, despite the fact that the whole toy-project is about doing exactly that.


MCVE

Below is a Bash Script that generates the following directory tree with Python source files and build script:

projectRoot
├── build.py
└── src
    └── main
        ├── python
        │   └── pkgRoot
        │       ├── __init__.py
        │       ├── pkgA
        │       │   ├── __init__.py
        │       │   └── modA.py
        │       └── pkgB
        │           ├── __init__.py
        │           └── modB.py
        └── scripts
            └── entryPointScript.py

7 directories, 7 files
================================================================================
projectRoot/build.py
--------------------------------------------------------------------------------
from pybuilder.core import use_plugin

use_plugin("python.core")
use_plugin("python.distutils")

default_task = "publish"

================================================================================
projectRoot/src/main/scripts/entryPointScript.py
--------------------------------------------------------------------------------
#!/usr/bin/env python

from pkgRoot.pkgB.modB import b

if __name__ == "__main__":
  print(f"Hello, world! 42 * 42 - 42 = {b(42)}")

================================================================================
projectRoot/src/main/python/pkgRoot/pkgA/modA.py
--------------------------------------------------------------------------------
def a(n):
  """Computes square of a number."""
  return n * n

================================================================================
projectRoot/src/main/python/pkgRoot/pkgB/modB.py
--------------------------------------------------------------------------------
from pkgRoot.pkgA.modA import a

def b(n):
  """Evaluates a boring quadratic polynomial."""
  return a(n) - n

The full script that generates example project (Disclaimer: provided as-is, modifies files and directories, execute at your own risk):

#!/bin/bash

# Creates a very simple hello-world like project
# that can be build with PyBuilder, and describes
# the result.

# Uses BASH heredocs and `cut -d'|' -f2-` to strip
# margin from indented code.

# strict mode
set -eu

# set up directory tree for packages and scripts
ROOTPKG_PATH="projectRoot/src/main/python/pkgRoot"
SCRIPTS_PATH="projectRoot/src/main/scripts"
mkdir -p "$ROOTPKG_PATH/pkgA"
mkdir -p "$ROOTPKG_PATH/pkgB"
mkdir -p "$SCRIPTS_PATH"

# Touch bunch of `__init__.py` files
touch "$ROOTPKG_PATH/__init__.py"
touch "$ROOTPKG_PATH/pkgA/__init__.py"
touch "$ROOTPKG_PATH/pkgB/__init__.py"

# Create module `modA` in package `pkgA`
cut -d'|' -f2- <<__HEREDOC > "$ROOTPKG_PATH/pkgA/modA.py"
  |def a(n):
  |  """Computes square of a number."""
  |  return n * n
  |
__HEREDOC

# Create module `modB` in package `pkgB`
cut -d'|' -f2- <<__HEREDOC > "$ROOTPKG_PATH/pkgB/modB.py"
  |from pkgRoot.pkgA.modA import a
  |
  |def b(n):
  |  """Evaluates a boring quadratic polynomial."""
  |  return a(n) - n
  |
__HEREDOC

# Create a hello-world script in `scripts`:
cut -d'|' -f2- <<__HEREDOC > "$SCRIPTS_PATH/entryPointScript.py"
  |#!/usr/bin/env python
  |
  |from pkgRoot.pkgB.modB import b
  |
  |if __name__ == "__main__":
  |  print(f"Hello, world! 42 * 42 - 42 = {b(42)}")
  |
__HEREDOC

# Create a simple `build.py` build script for PyBuilder
cut -d'|' -f2- <<__HEREDOC > "projectRoot/build.py"
  |from pybuilder.core import use_plugin
  |
  |use_plugin("python.core")
  |use_plugin("python.distutils")
  |
  |default_task = "publish"
  |
__HEREDOC

#################################################
#   Directory tree construction finished, only  #
#   debug output below this box.                #
#################################################

# show the layout of the generater result
tree "projectRoot"

# walk through each python file, show path and content
find "projectRoot" -name "*.py" -print0 | \
while IFS= read -r -d $'\0' pathToFile
do
  if [ -s "$pathToFile" ]
  then
    printf "=%.0s" {1..80} # thick horizontal line
    echo ""
    echo "$pathToFile"
    printf -- "-%.0s" {1..80}
    echo ""
    cat "$pathToFile"
  fi
done

The simplest way that I've found to run the freshly built project was as follows (used from the directory that contains projectRoot):

virtualenv env
source env/bin/activate
cd projectRoot
pyb
cd target/dist/projectRoot-1.0.dev0/dist/
pip install projectRoot-1.0.dev0.tar.gz
entryPointScript.py

This indeed successfully runs the script with all its dependencies on user-defined packages, and prints:

Hello, world! 42 * 42 - 42 = 1722

but the whole procedure seems quite complicated. For comparison, in an analogous situation in SBT, I would just issue the single

run

command from the SBT-shell -- that's why the seven-step recipe above seems a bit suspicios to me.

Is there something like a pyb run or pyb exec plugin that does the same, but does not require from me that I set up all these environments and install anything? What I'm looking for is the analogon of sbt run in SBT or mvn exec:java in Maven that will build everything, set up all the classpaths, and then run the class with the main method, without leaving any traces outside the project directory.

Since there is essentially no difference between the source code and the output of the target, I'm probably missing some obvious way how to run the script. If the PyBuilder itself is not needed at all for that, it's fine too: all I want is to somehow get Hello, world! 42 * 42 - 42 = 1722-string printed in the terminal.

Andrey Tyukin
  • 43,673
  • 4
  • 57
  • 93

2 Answers2

3

Create task in build.py

@task
def run(project):
    path.append("src/main/python")
    from test_pack import test_app
    test_app.main()

Try:

pyb run

abhimanyu
  • 730
  • 1
  • 10
  • 23
2

Apparently the following workflow:

  • pyb publish
  • pip install .tar.gz
  • runMyScript.py
  • uninstall

is exactly what is proposed by the creator of PyBuilder in this talk.

Note that the linked video is from 2014. If someone can propose a more streamlined recently provided solution, I'll of course accept that.

Andrey Tyukin
  • 43,673
  • 4
  • 57
  • 93
  • I'm just curious, did you want an exe or something like it? What you've described as a way to run a script seems to me to be a very standard Python workflow – HFBrowning Apr 09 '18 at 21:03
  • @HFBrowning Do I need an `exe`? I actually don't want an `exe`. I thought that Python code can be executed immediately as-is, but I don't see how to do it for an interconnected set of mutually interdependent packages, modules, and scripts. In SBT, I simply issue the command `run`, and it `run`s, and once it's done, it leaves no traces anywhere. Here, I seemingly have to leave the project directory, and create some completely unrelated environments elsewhere, and then install stuff, and *then* run it, and then additionally don't forget to clean up. This seems astonishingly complicated. – Andrey Tyukin Apr 09 '18 at 21:10
  • @HFBrowning Maybe I'm overlooking something obvious, maybe there is some `python --include-all-modules-in-this-package-directory-tree` option that helps me run complex projects. But right now, I just don't see how I can easily run projects that consist of more than one directory with a bunch of modules. – Andrey Tyukin Apr 09 '18 at 21:12
  • I am *so* accustomed to having to set up a virtualenv and installing packages into it that perhaps I am not interpreting your issue correctly. My usual workflow is create a virtualenv > use pip requirements.txt to install packages into it > create a new project > start work. Which is why the Anaconda distribution has been so popular, because this method frankly sucks when you're trying to manage a bunch of different (scientific computing) packages. – HFBrowning Apr 09 '18 at 21:16
  • However, the virtualenv stuff is fairly close to what you want. The point of activating it is to have it point at the correct interpreter and packages installed into that environment. You shouldn't have to uninstall anything after you are done either; just `deactivate`. Again, maybe I am talking about something different than you :) – HFBrowning Apr 09 '18 at 21:18
  • @HFBrowning Forgive me, I don't quite understand it: "use pip [...] to install packages" comes *before* "create a new project" in this workflow. This, by definition, works only for packages that have been created *before* I have written tons of code in my project. But now suppose I have created *my own* set of *new* packages that I want to use in *my own* project. Do I have to `install`-`uninstall` them with `pip` into `virtualenv` every single time I make an update and want to take a look at the intermediate result? – Andrey Tyukin Apr 09 '18 at 21:21
  • Ah, I think it's coming clearer to me. No, you don't. Creating your own package is stupid easy in Python - you just stick an `__init__.py` file into the directory. I'm a hack so I just use `import sys; sys.path.append("local path to package") import my_package` all the time. If I was getting fancy I would put it on my bitbucket and import that. If I wanted to be extra fancy I'd put it on PyPI. For what it's worth I've never seen PyBuilder before but my sense is it's there to ease the development process (testing) more than it is for publishing. There's already lots of easy ways to publish. – HFBrowning Apr 09 '18 at 21:25
  • @AndreyTyukin: - were you able to find something, I opened a similar ticket - https://stackoverflow.com/questions/56190047/pybuilder-add-source-current-directory-into-virtualenv – user1050619 May 20 '19 at 16:36