EDIT: Solution found in this new thread
I am using modified sample code to pull PubSub messages generated from Object Change Notifications. It works fine so far, in the Deb9 VM. In a python dictionary named message.data, there is this pair
"name":"folder2/1.testing"
Next, I need to pass strings along to issue unix commands, which I use subprocess for.
In a separate test script, but executed in native Windows 10, I have found that
testdictionary = {"name":"folder2/1.testing"}
dirpath = "~/subfolder1/"
namepath = testdictionary["name"]
fullpath = dirpath + namepath
subprocess.call(["echo abcd '/%{0}'".format(fullpath)])
to be valid code, echoing
abcd ~/subfolder1/folder2/1.testing
in the Win10 terminal.
However, when attempting to combine the code, and run in a Deb9 VM, I get this error returned:
No handlers could be found for logger "google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager"
Here is my full combined code (tested working perfectly before adding the dirpath to subprocess lines):
import time
import subprocess
from google.cloud import pubsub_v1
project_id = "redacted"
subscription_name = "redacted"
def receive_messages_with_custom_attributes(project_id, subscription_name):
"""Receives messages from a pull subscription."""
# [START pubsub_subscriber_sync_pull_custom_attributes]
subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(
project_id, subscription_name)
def callback(message):
print('Received message: {}'.format(message.data))
if message.attributes:
#print('Attributes:')
for key in message.attributes:
value = message.attributes.get(key);
#commented out to not print to terminal
#which should not be necessary
#print('{}: {}'.format(key, value))
message.ack()
dirpath = "~/subfolder1/"
namepath = message.data["name"]
print(namepath)
fullpath = dirpath + namepath
print(fullpath)
print("this is before subprocess")
subprocess.call(["echo", "abcd", '/%{0}'.format(fullpath)], shell=True)
#subprocess.call(["bash job-submit.sh -r hg19.fa.mmi -f '/%{0}' -o file_out.bam".format(fullpath)], shell=True)
subscriber.subscribe(subscription_path, callback=callback)
# The subscriber is non-blocking, so we must keep the main thread from
# exiting to allow it to process messages in the background.
print('Listening for messages on {}'.format(subscription_path))
while True:
time.sleep(60)
# [END pubsub_subscriber_sync_pull_custom_attributes]
receive_messages_with_custom_attributes(project_id, subscription_name)
I am not familiar with coding in general, and will greatly appreciate if someone can tell me what to change in the code to get desired behavior instead of logger errors.
I've seen something about import logging, but other questions had contexts too different from mine, and solutions aren't really transferable.
Full returns on the console with a test file:
Listening for messages on projects/[redacted]
Received message: {
"kind": "storage#object",
"id": "[redacted]/0.testing/1548033442364022",
"selfLink": "https://www.googleapis.com/storage/v1/b/[redacted]/o/BSD%2F0.testing",
"name": "BSD/0.testing",
"bucket": "[redacted]",
"generation": "1548033442364022",
"metageneration": "1",
"contentType": "application/octet-stream",
"timeCreated": "2019-01-21T01:17:22.363Z",
"updated": "2019-01-21T01:17:22.363Z",
"storageClass": "MULTI_REGIONAL",
"timeStorageClassUpdated": "2019-01-21T01:17:22.363Z",
"size": "0",
"md5Hash": "1B2M2Y8AsgTpgAmY7PhCfg==",
"mediaLink": "https://www.googleapis.com/download/storage/v1/b/[redacted]/o/BSD%2F0.testing?generation=1548033442364022&alt=media",
"crc32c": "AAAAAA==",
"etag": "CPb0uvvZ/d8CEAE="
}
No handlers could be found for logger "google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager"
Further traceback after I Ctrl + C:
^CTraceback (most recent call last):
File "ps_parse_pull.py", line 47, in <module>
receive_messages_with_custom_attributes(project_id, subscription_name)
File "ps_parse_pull.py", line 44, in receive_messages_with_custom_attributes
time.sleep(60)
KeyboardInterrupt
Exception in thread Thread-Heartbeater (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 754, in run
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/heartbeater.py", line 40, in heartbeat
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/streaming_pull_manager.py", line 283, in heartbeat
File "/usr/local/lib/python2.7/dist-packages/google/api_core/bidi.py", line 453, in is_active
File "/usr/lib/python2.7/threading.py", line 168, in acquire
<type 'exceptions.TypeError'>: 'NoneType' object is not callable
Exception in thread Thread-2 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 754, in run
File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 207, in consume_request_iterator
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'return_from_user_request_generator'
Exception in thread Thread-LeaseMaintainer (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 754, in run
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/leaser.py", line 103, in maintain_leases
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/histogram.py", line 142, in percentile
<type 'exceptions.TypeError'>: 'NoneType' object is not callable
