Multithreading in Google Colab with communication between Javascript and IPython Kernel

Question

my problem is the following:

There's along running function that sends data to the HTML widget interface
The interface needs to stay interactive and trigger other functions immediately

I created an interface with the IPython HTML Widget. The widgets has buttons, etc. Inside this widget I'm displaying dynamically changing messages that are send from a python function using a comm channel from client to kernel. This function takes quite some time and while it is running, it's blocking the interface from reacting to button clicks. Those button clicks are only executed after the function has stopped running. As I want these things to happen simultaneously I tried to put in a different thread so it doesn't block the kernel. But on Google Colab after I start the thread, the cell stops and nothing happens. I've taken a look at this: Python Semaphore does not seem to work in Google Colab But the solution of thread.join() does not work for me, because this blocks the kernel yet again.

Now I'm very new to threading and all of this but I can't get both the training function and the interface running. And I'm probably having a terrible approach to all of this. I also tried to look into Asyncio, but I guess I'm lacking the right understanding for my problem.

Here's the content of my HTML widget:

html = """
<div>
<div id="message">Message display</div>
<button id="button">Click</button>
</div>
  
"""
javascript = """<script>
var kernel = google.colab.kernel;
(async () => {
  const channel = await kernel.comms.open('comm_target');

  (async function() {
    for await (const message of channel.messages) {
      console.log(message.data)
      document.getElementById("message").innerHTML = message.data
    }
  })();
})()

/*  Button click */
const addBtn = document.querySelector('#button');
addBtn.addEventListener('click', function(e) {
  e.preventDefault();
  kernel.invokeFunction("button_click");
});
/* start long function */
kernel.invokeFunction("start_function");
</script>
"""

And here are my functions in Python:

import ipywidgets as widgets
from IPython.display import display, HTML, Image
import google.colab.output
import json
import threading
import time

#this part is from the stackoverflow answer
comm_ = None

def target_func(comm, msg):
  global comm_
  comm_ = comm 

  ## Register handler for later messages
  @comm.on_msg
  def _recv(msg):
    # Use msg['content']['data'] for the data in the message
    comm.send({'echo': msg['content']['data']})
        
  @comm.on_close
  def _close(msg):
    global comm_
    comm_ = None

get_ipython().kernel.comm_manager.register_target('comm_target', target_func)

#simulation for the long running function
def long_function():
   for i in range(100):
     time.sleep(2)
     comm_.send(data=i)

#button click for testing
def button_click():
  print("Button clicked") 

google.colab.output.register_callback('button_click', button_click)

def start():
  mainthread = threading.Thread(target = long_function)
  mainthread.start()
  #mainthread.join()

google.colab.output.register_callback('start_function', start)  

whtml = widgets.HTML(value = (html + javascript))
display(whtml)

Multithreading in Google Colab with communication between Javascript and IPython Kernel

0 Answers0