1

I want to spawn several processeses each hosting a HTTPServer listening to a different port.

I'm doing:

def launch_webserver(host, port):
    server = HTTPServer((host, port), handler_class)
    try:
        server.serve_forever()
    finally:
        server.socket.close()


for num in range(arguments.number):
    port = arguments.port + num
    process = Process(target=launch_webserver, args=(arguments.host, port))
    process.start()

It works and I can confirm that each webserver is listening properly. These just need to collect some data and are not heavily demanding on CPU or IO.

How do I profile this kind of application? How can I determine the maximum number of processess. Is there any Python or system limit I should be aware of? I'm running this on Python 3 in an Ubuntu machine.

Josep Valls
  • 5,483
  • 2
  • 33
  • 67
potato
  • 36
  • 5
  • 1
    Well since you're using a new port number for each process, you can spawn around 64k processes (65k total ports - 1k privileged ports). That's a hard limit. Check this answer for the maximum number of processes in Linux: https://stackoverflow.com/questions/9361816/maximum-number-of-processes-in-linux One other limit to be aware of the is the `max_open_file_descriptors` – rdas Apr 12 '19 at 21:14
  • 1
    For limits you can pull in the [resource module](https://docs.python.org/3/library/resource.html) and use the resource.getrlimit(...) to check all kinds of limits, cpu time, max procs, max open files, ... – estabroo Apr 12 '19 at 21:18

0 Answers0