1

I cannot use Go nor Python etcd APIs to perform interaction with etcd.

I need to query:

env ETCDCTL_API=3 etcdctl --endpoints=server:2379 --cert ca.pem --key ca.key --cacert cacert.pem --command-timeout=60s get $(echo L3ZvdGUvADHd0kmW58d9JqTD8whMYS9vY3Nw | base64 -d)

This will return:

-bash: warning: command substitution: ignored null byte in input

However running alone:

echo L3ZvdGUvADHd0kmW58d9JqTD8whMYS9vY3Nw | base64 -d

Returns:

/vote/1��I���}&��La/ocsp

What is the problem and how could I maybe escape, but still remain valid key for etcd and GET query it?

armaka
  • 191
  • 5
  • 2
    A NUL byte cannot be passed on the command line -- not in _any_ language, because UNIX command lines are comprised of C strings, and C strings are terminated by NULs. This is a design oversight on the part of etcdctl's authors -- if key names can contain NULs, they should be providing an option to pass them in an escaped form; but that escaping is something that needs to be reversed _by etcdctl itself_; there's nothing the shell can do to fix this. – Charles Duffy Mar 02 '23 at 17:17
  • @CharlesDuffy is it possible to somehow escape NUL byte, so I could still query for the etcd key? Because etcd key contains NUL byte in its name – armaka Mar 02 '23 at 17:19
  • 2
    Only if etcdctl itself supports reversing that escaping. The shell is just generating argument vectors composed of C strings, and you can't have a NUL byte in the middle of a string in C. – Charles Duffy Mar 02 '23 at 17:20
  • @CharlesDuffy seems like this is impossible to accomplish via bash :( – armaka Mar 02 '23 at 17:36
  • 1
    As I said, not just bash, but _any other language too_. You can't use etcdctl for this purpose, even with no shell involved anywhere, unless it supports escaped key names. – Charles Duffy Mar 02 '23 at 17:37
  • 1
    For example, you'd have the same problem with Python's `subprocess.Popen(['etcdctl', 'get', keyname], shell=False)`, despite that not using a shell at all, because it's passing the key name via an `execve` syscall, and to do that the name needs to be coerced to a C string. For the same reason, you'd have the same problem from a C program using `execve()` directly. So I hesitate to endorse your description of the thing as being impossible "via bash"; instead, I'd say it's impossible _via etcdctl_ (unless, again, etcdctl supports some form of escaping). – Charles Duffy Mar 02 '23 at 17:39
  • 1
    ...you could write your own etcd client that accepted escaped key names (or accepted a key name on stdin instead of via the command line), and then it'd be _perfectly_ possible via bash. – Charles Duffy Mar 02 '23 at 17:42
  • @CharlesDuffy yeah, you just clarified this concept as clear as possible!! Thank you very much, if you wish you could put any of your answers into "Answer" on this question and I will accept it – armaka Mar 02 '23 at 17:45

1 Answers1

2

The Short Story

Unfortunately, unless etcdctl is extended to explicitly support accepting escaped contents, and unescaping that content to get the key name to use (or to accept key names passed via a means other than the command line), passing a key name containing a literal NUL is not possible.

Consider writing your own command-line client for etcd that accepts encoded strings if this is something you need to do.

Background: How Programs Start Programs On UNIX

When a program starts another program on UNIX, the process involves a syscall called execve() (or an equivalent: there are higher-level tools like the POSIX spawn() family, but they still act similarly for our purposes here). This might look like:

pid = execve("/usr/bin/etcd", char[][]{
  "etcd",
  "--endpoints=server:2379",
  /* other content here elided */
  "get",
  keyname
}, environment)

Because this API is defined in C, all strings are NUL-terminated C strings; they start at the character that an individual char* points to, and end at the next NUL after that character (so when you type "get" in C, it creates an array like {"g", "e", "t", NULL}).

Thus, if you have something that actually looks like {"g", "e", "t", NULL, "f", "o", "o", NULL}, only get is part of the string starting to the g; there needs to be another pointer to the f in foo for foo to be seen as a separate argument, and there's no possible way for get\x00foo to be seen as a single string with \x00 representing a single-byte NUL literal by anything written to interact with C-style strings.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441