How can I use IPython interactive shell in gdb? || How can I get tab-completion to work in gdb's Python-interactive (pi) shell?

Question

Normally, in Python shells I can press Tab twice to get a list of prompts.

On the other hand, in gdb's Python shell (pi or python-interactive command), there's only gdb-style completion.

Example session:

$ gdb -q
(gdb) pi
>>> gdb
<module 'gdb' from '/usr/share/gdb/python/gdb/__init__.py'>
>>> gdb.TabTab
... nothing ...
>>> show TabTab
Display all 148 possibilities? (y or n)
ada                              exec-direction                   record
agent                            exec-done-display                remote
annotate                         exec-file-mismatch               remoteaddresssize
[...]

Python auto complete should be at least like this.

$ python
Python 3.X.Y ...
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.TabTab
sys.abiflags                              sys.hash_info
sys.addaudithook(                         sys.hexversion
sys.api_version                           sys.implementation
[...]

How can I get the same/similar thing in gdb? in particular IPython shell with tab completion is fine.

---

Failed attempts:

• This solution

  import readline
  import rlcompleter
  readline.parse_and_bind("tab: complete")
  

  makes the shell output a literal tab when Tab is pressed after sys. or similar.

  At least it does work for identifier tab completion (aTabTab) does list some entries)

  Looks like this is because of some interaction with gdb -- get_completer_delims get reset to some value every time, and if the code above is run then the tab completion outside gdb switches to "Python mode" as well.

• Use background_zmq_ipython causes segmentation fault, because some gdb API (such as gdb.Value) cannot be read from outside the main thread.

• Use IPython.embed() also make Tab output a literal tab character.

• The official gdb documentation https://sourceware.org/gdb/current/onlinedocs/gdb/Completion.html doesn't mention anything about Python.

Answer 1

There are a few ways I've figured out.

I can't figure out any way to use the built-in readline library. See the failed attempts in the question for more details.

---

1. Reset stdout and stderr before calling IPython.embed.

   import sys
   sys.stdout=sys.__stdout__
   sys.stderr=sys.__stderr__
   
   import IPython
   IPython.embed(colors="neutral")
   

   Remember to reset stdout and stderr afterwards to avoid possible issues.

   Reference:

   • How to recognize whether a script is running on a tty?
   • IPython.embed() does not use terminal colors

   IPython will only use tab-completion and color when all of stdin, stdout and stderr are tty devices. By default gdb sys.stdout and sys.stderr are gdb wrappers (so that gdb can do "press enter to continue" when pagination limit is exceeded)

2. Start a kernel, and start a console separately.

   import IPython
   IPython.embed_kernel()
   

   Read the console output on how to connect, and how to exit the terminal from the remote console.

   Using my other answer it's also possible to exit the terminal remotely programmatically.

3. Start a kernel (the complex way)

   Read the source code of IPython to figure out how to start a kernel manually, and get the connection file path in the process.

   import threading
   import subprocess
   import IPython
   from ipykernel.kernelapp import IPKernelApp
   import sys
   
   app = IPKernelApp.instance()
   app.initialize([])
   app.kernel.user_module = sys.modules[__name__]
   app.kernel.user_ns = locals()
   app.shell.set_completer_frame()
   
   def console_thread_run():
       subprocess.run(["jupyter-console", "--no-confirm-exit", "--existing", 
           app.abs_connection_file
           ])
       app.kernel.do_shutdown(restart=False)
   console_thread=threading.Thread(target=console_thread_run)
   console_thread.start()
   
   app.start()
   

4. Start a kernel using background_zmq_ipython (accesses internal property, might break at any time).

   The main difference is that sys.stdin, sys.stdout etc. are not affected. See background_zmq_ipython documentation and ipython - Provide remote shell for Python script - Stack Overflow for more details.

   import subprocess
   import logging
   import threading
   
   from background_zmq_ipython import IPythonBackgroundKernelWrapper
   
   kernel_wrapper = IPythonBackgroundKernelWrapper(
           banner="", # default value is "Hello from background-zmq-ipython."
           user_ns=globals(),
           logger=logging.Logger("IPython", level=logging.INFO)
           # no handler
           # otherwise it will print "To connect another client to this IPython kernel" ...
           )
   kernel_wrapper.thread=threading.main_thread()  # workaround for assertions
   
   subprocess.Popen(["python", "-c",
       (
       "from jupyter_console.app import ZMQTerminalIPythonApp;"
       "app = ZMQTerminalIPythonApp();"
       "app.initialize();"
       "app.shell.own_kernel=True;"
       "app.start();"
       ),
       "--no-confirm-exit",
       "--existing", kernel_wrapper.connection_filename
       ])
   
   kernel_wrapper._thread_loop()
   

   Also show how to change the message "keeping kernel alive" to "Shutting down kernel".