2

I'm a Java Spring boot developer and I develop 3-tier crud applications. I talked to a guy who seemed knowledgeable on the subject, but I didn't get his contact details. He was advocating for Python's FastApi, because horizontally it scales better than Spring boot. One of the reasons he mentioned is that FastApi is single-threaded. When the thread encounters a database lookup (or other work the can be done asyncly), it picks up other work to later return to the current work when the database results have come in. In Java, when you have many requests pending, the thread pool may get exhausted.

I don't understand this reasoning a 100%. Let me play the devil's advocate. When the Python program encounters an async call, it must somehow store the program pointer somewhere, to remember where it needs to continue later. I know that that place where the program pointer is stored is not at all a thread, but I have to give it some name, so let's call it a "logical thread". In Python , you can have many logical threads that are waiting. In Java, you can have a thread pool with many real threads that are waiting. To me, the only difference seems to be that Java's threads are managed at the operating system level, whereas Python's "logical threads" are managed by Python or FastApi. Why are real threads that are waiting in a thread pool so much more expensive than logical threads that are waiting? If most of my threads are waiting, why can't I just increase the thread pool size to avoid exhaustion?

Albert Hendriks
  • 1,979
  • 3
  • 25
  • 45

3 Answers3

6

The issues with Java threads in the question are addressed by the project Loom, which is planned for inclusion in a future Jdk. It is very well explained here https://www.baeldung.com/openjdk-project-loom :

Presently, Java relies on OS implementations for both the continuation [of threads] and the scheduler [for threads].

Now, in order to suspend a continuation, it's required to store the entire call-stack. And similarly, retrieve the call-stack on resumption. Since the OS implementation of continuations includes the native call stack along with Java's call stack, it results in a heavy footprint.

A bigger problem, though, is the use of OS scheduler. Since the scheduler runs in kernel mode, there's no differentiation between threads. And it treats every CPU request in the same manner. (...) For example, consider an application thread which performs some action on the requests and then passes on the data to another thread for further processing. Here, it would be better to schedule both these threads on the same CPU. But since the [OS] scheduler is agnostic to the thread requesting the CPU, this is impossible to guarantee.

The question really boils down to Why are OS threads considered expensive?

Albert Hendriks
  • 1,979
  • 3
  • 25
  • 45
4

It very much depends on what your threads are doing. Imagine that your request is calling some external endpoint. Most of the time your java thread (which is heavy) is just doing nothing. Is it a problem? Depends. In java ecosystem threads are not created on the fly, that problem is solved in any decent framework with threadpools.

But still thread stack size occupies memory. If you would have 1000 threads (or 1000 concurent requests) in a thread pool waiting for IO operation your memory might go beyond 1GB easily.

For such particular scenario some async / await abstraction could utilise memory much better. Whether all task would be executed faster is a different story and depends on many factors.

So truly you need to go deeper, what scaling you are talking about? memory? or speed of execution of all operations (which usually also involve some cpu).

Not neglectable pros of java is that requests handled by threads can run in parallel and utilize many processors while in python this is simply not possible.

FastAPI and their async await model is easy to use, cuts developer off from threading model and from thinking about concurrency. Unfortunately it also means smaller control in case this is needed.

Just calling async method in python is not making any code execution faster. In the end under the hood there is one event loop that runs on one Cpu only.

Depending on the usecase it's good enough and efficient or just the opposite.

Usually people from Python ecosystem are hyped against heavy java and just use argument that it's faster / lighter without seeing other cons.

Here also a link to tease you a bit with typical webapp performance comparison across few frameworks (in web / rest scenarios)

https://www.travisluong.com/fastapi-vs-fastify-vs-spring-boot-vs-gin-benchmark/

Piotr
  • 317
  • 1
  • 13
  • Beware that Python can run in parallel simply by running multiple processes (containers), so it's false that it's not possible with Python. This way Python processes that utilize x processors can handle more requests than a Java process that utilizes x processors. That's what's meant by horizontal scaling. Interesting benchmark though, +1. – Albert Hendriks Aug 18 '22 at 17:11
  • 1
    True, but certain technics known from jvm (not java specific) world cannot be applied then. For instance - for jvm many threads can share some cache. This is huge advantage if you can cache some things. The need for external cache can be applied much later (when you hit the scale) In python if you want to share some state between multiple python processes you need external state (database or cache) from the very beginning. Many python advocates advertise python is easy - but it's harder when you go beyond "hello world" stateless microservice. – Piotr Oct 05 '22 at 22:13
3

FastAPI is a fast framework, and you can quickly (and easily) create API backends in it. To be honest, if you are a Java developer, I would recommend Quarkus or something for building a REST API, not FastAPI. FastAPI is a fantastic tool, absolutely great if you are already in the Python ecosystem.

When it goes about multithreading; Java is 'real' multithreading where as Python is very much not. Java threads will run concurrently; two tasks can and will be executed at the same time. In Python, within one Python process, this is (nearly) impossible. The reason for this is GIL (google it, there is ton's of stuff out there on how it works). The result is; even if you use 'real' threads in Python, code is still not executed concurrently but rather serially, where the interpreter (big difference to Java) is jumping from one call stack to another constantly.

As to what you refer to as 'logical threads', I think you mean the asynchronous capability of Python. This is basically the same as using threads (not really, but on an abstract level they are very similar); tasks are not run concurrently. There is just one thread that constantly switches between tasks. Tasks will yield back control to the event loop (the object that coordinates tasks and decides what is executed in which order), and another task is further executed until that task yields control, etc. It is basically the same kind of execution pattern as with threads within Python.

Comparing a Python framework to a Java framework is just weird in my opinion. They are both useful and cool, but not really competitors.

JarroVGIT
  • 4,291
  • 1
  • 17
  • 29
  • Thanks for your reply, but in imho it doesn't answer my question why FastApi scales better horizontally than Spring Boot for crud applications. – Albert Hendriks Jul 19 '22 at 19:22
  • 1
    It doesn’t, Fastapi doesn’t scale better horizontally per se than Spring Boot. The reasoning of your buddy does not make any sense at all, especially on the “because it’s single threaded” part, that is what my answer tries (but fails) to explain. – JarroVGIT Jul 19 '22 at 19:39