Kubernetes Python client returns pod's JSON HTTP response from proxy verb as string with single quotes instead of double quotes

Question

I'm requesting some JSON data from a pod's web server via the Kubernetes API proxy verb. That is:

corev1 = kubernetes.client.CoreV1Api()
res = corev1.connect_get_namespaced_pod_proxy_with_path(
    'mypod:5000', 'default', path='somepath', path2='somepath')
print(type(res))
print(res)

The call succeeds and returns a str containing the serialized JSON data from my pod's web service. Unfortunately, res now looks like this ... which isn't valid JSON at all, so json.loads(res) denies to parse it:

{'x': [{'xx': 'xxx', ...

As you can see, the stringified response looks like a Python dictionary, instead of valid JSON. Any suggestions as to how this convert safely back into either correct JSON or a correct Python dict?

Not a good idea, as this breaks when there are escaped "'" inside the names or strings. Also, `eval()` isn't a good idea either, because then someone might attack my Kubernetes client. — TheDiveO, Apr 25 '19 at 13:54

score 2 · Answer 1 · answered Jul 24 '20 at 09:55

I ran into a similar problem with executing on a pod and found a solution. I guess it could work for you as well.

add the parameter _preload_content=False, to the stream call -> you receive a WSClient object
call run_forever(timeout=10) on it
Then you get the correct and unformatted string with .read_stdout()

For example:

wsclient_obj = stream(v1.connect_get_namespaced_pod_exec, m
                    y_name,
                    'default',
                    command=['/bin/sh', '-c', 'echo $MY_VAR'],
                    stderr=True,
                    stdin=False,
                    stdout=True,
                    tty=False,
                    _preload_content=False
                )
wsclient_obj.run_forever(timeout=10)
response_str = wsclient_obj.read_stdout()

After that json.loads(response_str) will work :)

Kennedn · Accepted Answer · 2022-05-16T14:22:47.923

Looking at the source code for core_v1_api.py. The method calls generally accept a kwarg named _preload_content.

Setting this argument to False instructs the method to return the urllib3.HTTPResponse object instead of a processed str. You can then work directly with the data, which cooperates with json.loads().

Example:

corev1 = client.CoreV1Api()
res = corev1.connect_get_namespaced_pod_proxy_with_path(
    'mypod:5000', 'default', path='somepath', 
    path2='somepath', _preload_content=False)
json.loads(res.data)

score 1 · Answer 3 · answered Apr 25 '19 at 17:32

After reading through some code of the Kubernetes Python client, it is now clear that connect_get_namespaced_pod_proxy() and connect_get_namespaced_pod_proxy_with_path() force the response body from the remote API call to be converted to str by calling self.api_client.call_api(..., response_type='str', ...) (core_v1_api.py). So, we're stuck with the Kubernetes API client giving us only the string representation of the dict() representing the original JSON response body.

To convert the string back to a dict(), the anwer to Convert a String representation of a Dictionary to a dictionary? suggests to use ast.literal_eval(). Wondering whether this is a sensible route to take, I've found the answer to Is it a best practice to use python ast library for operations like converting string to dict says that it's a sensible thing to do.

import ast
corev1 = kubernetes.client.CoreV1Api()
res = corev1.connect_get_namespaced_pod_proxy_with_path(
    'mypod:5000', 'default', path='somepath', path2='somepath')
json_res = ast.literal_eval(res)

Kubernetes Python client returns pod's JSON HTTP response from proxy verb as string with single quotes instead of double quotes

3 Answers3