0

Hey I am learning Python at the moment. I wrote a few programs. Now I have a question: Is it possible to run more "operations" at once? According to my knowledge the scripts runs from the top to the bottom (except from thing like called def and if statements and so on).

For example: I want to do something and wait 5 seconds an then continue but while my program "waits" it should do something other? (This one is very simple) Or: While checking for input do something other output things.

The examples are very poor but I do not finde something better at the moment. (If something comes to my mind, I will add it later) I hope you understand what my question is.

Cheers

1 Answers1

0

TL;DR: Use an async approach. Raymond Hettinger is a god, and this talk explains this concept more accurately and thoroughly than I can. ;)

The behavior you are describing is called "concurrency" or "asynchronicity", where you have more than one "piece" of code executing "at the same time". This is one of the hardest problems in practical computer science, because adding the dimension of time causes scheduling problems in addition to logic problems. However, it is very much in demand these days because of multi-core processors and the inherently parallel environment of the internet

"At the same time" is in quotes, because there are two basic ways to make this happen:

  1. actually run the code at the same time
  2. make it look like it is running at the same time.

The first option is called Concurrent programing, and the second is called Asynchronous programming (commonly "async").

Generally, "modern" programming seems to favor async, because it's easier to reason about and comes with fewer, less severe pitfalls. If you do it right, async programs can look a lot like the synchronous, procedural code you're already familiar with. Golang is basically built on the concept. Javascript has embraced "futures" in the form of Promises and async/await. I know it's not Python, but this talk by the creator of Go gives a good overview of the philosophy.

Python gives you three main ways to approach this, separated into three major modules: threading, multiprocessing, and asyncio

multiprocessing and threading are concurrent solutions. They do very similar things, but accomplish them in slightly different ways by delegating to the OS in different ways. This answer has a concise explanation of the difference. Concurrency is notoriously difficult to debug, because it is not deterministic: small differences in timing can result in completely different sequences of execution. You also have to deal with "race conditions" in threads, where two bits of code want to read/change the same piece of shared state at the same time.

asyncio, or "asynchronous input-output" is a more recent, async solution. You'll need at least Python 3.4. It uses event loops to allow long-running tasks to execute without "blocking" the rest of the program. Processes and threads do a similar thing, running two or more operations on even the same processor core by interrupting the running process periodically, forcing them to take turns. But with async, you decide where the turn-taking happens. It's like designing mature adults that interact cooperatively rather than designing kindergarteners that have to be baby-sat by the OS and forced to share the processor.

There are also third-party packages like gevent and eventlet that predate asyncio and work in earlier versions of Python. If you can afford to target Python >=3.4, I would recommend just using asyncio, because it's part of the Python core.

Phil
  • 141
  • 1
  • 7
  • A beginner-focused question is not really the place to introduce one side of an ongoing holy war, even if it's a well-argued side. (BTW, I personally split my dev work between Go and Clojure, so I have a foot in both worlds -- continuation-passing async, and race-free lockless threading by virtue of immutable data with explicit state-management primitives; they both have their value, and their place). – Charles Duffy Jan 31 '18 at 22:28
  • (If you aren't familiar with Clojure's concurrency model, I'd suggest looking it -- basically, it puts the same transactional semantics around threads modifying in-memory variables that a modern RDBMS puts on updates, ensuring that each read is based on a consistent view of an individual point in time, and that each write is based on content that wasn't modified while the transaction was in-flight, by retrying transactions -- which can be done by virtue of using pure functions inside your transactional code) – Charles Duffy Jan 31 '18 at 22:33
  • You've left out that all those pitfalls of concurrent parallelism mostly (if not entirely) disappear once you start coding in a functional style. – Eli Korvigo Jan 31 '18 at 22:34
  • @Phil, ...point being, the "all threading is racy, hard to reason about and deadlock-prone" strawman you're building might represent threading as-practiced in a whole lot of the world (particularly those parts of the world whose building blocks for developing software center around mutable objects), but it's a strawman nonetheless. – Charles Duffy Jan 31 '18 at 22:34
  • @CharlesDuffy, fair points. I did open a can of worms by bringing up other languages. But, I'm fairly certain that the general consensus (that I've seen, at least, in my research) is that concurrency-type problems *in Python* are best handled with `asyncio`, especially for devs just starting out in that realm – Phil Feb 01 '18 at 02:20
  • It is true that you *can* do threading stuff in a way that does not deadlock. It seems like it's easier for beginners to shift to a parallel mindset with async. Dunno. Certainly didn't mean it in a flamey sort of way. – Phil Feb 01 '18 at 02:22