0

I have master/worker EC2 instances that I'm using for Grinder tests. I need to try out a load test that directly gets files from an S3 bucket, but I'm not sure how that would look in Jython for the Grinder test script.

Any ideas or tips? I've looked into it a little and saw that Python has the boto package for working with AWS - would that work in Jython as well?

(Edit - adding code and import errors for clarification.)

Python approach:
Did "pip install boto3"
Test script:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
import boto3

# boto3 for Python
test1 = Test(1, "S3 request")
resource = boto3.resource('s3')

def accessS3():
    obj = resource.Object(<bucket>,<key>)

test1.record(accessS3)

class TestRunner:
    def __call__(self):
        accessS3()

The error for this is:

net.grinder.scriptengine.jython.JythonScriptExecutionException: : No module named boto3

Java approach:
Added aws-java-sdk-1.11.221 jar from .m2\repository\com\amazonaws\aws-java-sdk\1.11.221\ to CLASSPATH

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
import com.amazonaws.services.s3 as s3

# aws s3 for Java
test1 = Test(1, "S3 request")
s3Client = s3.AmazonS3ClientBuilder.defaultClient()
test1.record(s3Client)

class TestRunner:
    def __call__(self):
        result = s3Client.getObject(s3.model.getObjectRequest(<bucket>,<key>))

The error for this is:

net.grinder.scriptengine.jython.JythonScriptExecutionException: : No module named amazonaws

I'm also running things on a Windows computer, but I'm using Git Bash.

syim
  • 501
  • 1
  • 4
  • 16

1 Answers1

0

Given that you are using Jython, I'm not sure whether you want to execute the S3 request in java or python syntax.

However, I would suggest following along with the python guide at the link below.

http://docs.ceph.com/docs/jewel/radosgw/s3/python/

Ted
  • 126
  • 1
  • 10
  • Thanks for your reply! I'm having some issues getting the import working. I've tried doing pip install boto for the Python approach, and I've tried adding an aws-java-sdk jar to my classpath for the Java approach, but either way I am receiving the "No module named amazonaws" or "No module named boto3" error. Do you have any advice on how to import AWS? – syim Nov 21 '17 at 13:50
  • Can you post the stack trace as well as your code in the body of your request please? – Ted Nov 21 '17 at 14:16
  • I know you mentioned that you attempted to install boto3 with pip. Can you reference this page and post the error result if it fails to install. https://boto3.readthedocs.io/en/latest/guide/quickstart.html Since youre on Windows, do you have pip installed? – Ted Nov 21 '17 at 15:13
  • Yes, I have pip and can run it through Git Bash. When I run pip install boto3, I get several statements that say "Requirement already satisfied". Should I post the full statements in the body of the question? – syim Nov 21 '17 at 15:23
  • So on further reasearch, I found that any package that has a C backend (rather than a java backend) does not work in Jython, you would instead need to use CPython, which unfortunately makes your use of Jython a bit redundant. If I were in your position, I would contain the aws request in a python script and call the script from Java when needed. [Installing python libraries in jython](https://stackoverflow.com/questions/6787015/how-can-i-install-various-python-libraries-in-jython) – Ted Nov 21 '17 at 15:33
  • Ok, thank you for letting me know. So I should use boto3 in a python script and call it using Java? Or can I call the python script using the Jython test script for Grinder? – syim Nov 21 '17 at 15:52
  • That is a choice for you to make based on your preference. Personally, I don't know why you are opting to use both languages. I would try to consolidate it all into one language to minimize weird edge case errors. – Ted Nov 21 '17 at 16:43
  • I guess I'm just confused on how all of this integrates together. Doesn't the Grinder test script need to be in Jython? If so, I'm not sure how to call the python script that uses boto3 from the Grinder test. – syim Nov 21 '17 at 17:00