Process vs Thread ​

We have introduced multiprocessing and multithreading, which are two common ways to implement multitasking. Now, let's discuss the advantages and disadvantages of these two approaches.

First, to achieve multitasking, we usually design a Master-Worker model where the Master is responsible for task allocation, and the Workers are responsible for executing tasks. In a multitasking environment, there is typically one Master and multiple Workers.

• In a multiprocessing setup, the main process acts as the Master, while other processes serve as Workers.
• In a multithreading setup, the main thread acts as the Master, and other threads serve as Workers.

Advantages and Disadvantages ​

Advantages of Multiprocessing: The biggest advantage of multiprocessing is its high stability. If a child process crashes, it does not affect the main process or other child processes. (Of course, if the main process crashes, all processes will terminate, but the Master process, which only handles task allocation, is less likely to crash.) The famous Apache server initially adopted a multiprocessing model.

Disadvantages of Multiprocessing: However, multiprocessing has some drawbacks. The cost of creating processes is high; in Unix/Linux systems, using fork is manageable, but in Windows, creating a process incurs significant overhead. Additionally, the operating system can only run a limited number of processes simultaneously. With memory and CPU limitations, having thousands of processes running concurrently can overwhelm the operating system's scheduling capabilities.

Advantages of Multithreading: Multithreading is usually a bit faster than multiprocessing, but not by much. A critical drawback of multithreading is that if any one thread crashes, it can potentially bring down the entire process since all threads share the process's memory. In Windows, if a thread encounters an error, you often see messages like, "The program has performed an illegal operation and will close," which usually indicates that a problem in one thread has caused the operating system to terminate the entire process.

On Windows, multithreading is generally more efficient than multiprocessing, which is why Microsoft's IIS server defaults to a multithreading model. However, because of stability issues with multithreading, IIS is less stable than Apache. To address this problem, both IIS and Apache have adopted a hybrid model that combines multiprocessing and multithreading, complicating matters further.

Thread Switching ​

Regardless of whether you use multiprocessing or multithreading, if the number of tasks becomes too large, efficiency will inevitably suffer. Why is that?

Let’s illustrate this with an analogy: suppose you are unfortunately preparing for an important exam and need to complete assignments for five subjects—Chinese, Math, English, Physics, and Chemistry—each taking 1 hour.

If you spend 1 hour on Chinese, then 1 hour on Math, and so on until all five subjects are completed, this single-task model takes a total of 5 hours.

Now, if you switch to a multitasking model where you work for 1 minute on Chinese, then switch to Math for 1 minute, then English, and so on, as long as the switching speed is fast enough, it might appear, from a kindergartener's perspective, that you are simultaneously completing assignments for all five subjects.

However, switching tasks incurs a cost. For example, to switch from Chinese to Math, you must first tidy up your Chinese materials (this is called "saving the state"), then open your Math book and gather the necessary tools (this is called "preparing the new environment") before you can begin working on Math. The operating system performs a similar task when switching between processes or threads. It needs to save the current execution state (CPU register status, memory pages, etc.) and prepare the new task's execution environment (restoring the previous register state, switching memory pages, etc.) before starting execution. Although this switching process is fast, it still consumes time. If there are thousands of tasks running concurrently, the operating system may spend most of its time switching tasks instead of executing them. This often leads to scenarios where the hard drive is excessively active, windows become unresponsive, and the system appears to freeze.

Therefore, when multitasking reaches a certain limit, it can exhaust all system resources, resulting in a sharp decline in efficiency, making it impossible to complete any tasks effectively.

CPU-Bound vs IO-Bound ​

A second consideration for whether to adopt multitasking is the type of task. We can categorize tasks into CPU-bound and IO-bound.

• CPU-bound tasks are characterized by requiring significant computation, consuming CPU resources (e.g., calculating π, decoding high-definition videos). Although CPU-bound tasks can be executed with multitasking, the more tasks there are, the more time is spent on context switching, resulting in lower CPU execution efficiency. To optimize CPU utilization, the number of concurrent CPU-bound tasks should equal the number of CPU cores.

  Given that CPU-bound tasks primarily consume CPU resources, the efficiency of the code is crucial. Scripting languages like Python have low execution efficiency, making them unsuitable for CPU-bound tasks. For CPU-bound tasks, it is best to use a language like C.

• IO-bound tasks involve network or disk I/O operations, and their characteristic is that they consume little CPU but spend most of their time waiting for I/O operations to complete (since I/O speeds are much lower than those of the CPU and memory). For IO-bound tasks, the more concurrent tasks there are, the higher the CPU efficiency, but there is also a limit. Most common tasks are IO-bound, such as web applications.

  During IO-bound task execution, 99% of the time is spent waiting for I/O operations, with very little time spent on CPU operations. Therefore, switching from a high-speed language like C to a low-speed scripting language like Python yields minimal performance improvement. For IO-bound tasks, the most suitable language is one with the highest development efficiency (least code required), making scripting languages the preferred choice, while C has the lowest development efficiency.

Asynchronous I/O ​

Considering the significant speed difference between CPU and I/O, a task waiting for I/O operations most of the time can prevent other tasks from executing in parallel with a single-process, single-thread model. This limitation is why we need multiprocessing or multithreading models to support concurrent task execution.

Modern operating systems have made substantial improvements to I/O operations, with the most significant enhancement being support for asynchronous I/O. By fully utilizing the operating system's asynchronous I/O capabilities, we can execute multiple tasks using a single-process, single-thread model. This new model is called the event-driven model. Nginx is a web server that supports asynchronous I/O and can efficiently handle multiple tasks using a single-process model on a single-core CPU. On multi-core CPUs, multiple processes can run (with the number equal to the number of CPU cores), fully utilizing the multi-core CPU's capabilities. Given that the total number of processes in the system is limited, the operating system's scheduling is highly efficient. Implementing an asynchronous I/O programming model to achieve multitasking is a major trend.

In Python, the single-thread asynchronous programming model is referred to as a coroutine. With coroutine support, it becomes possible to write efficient multitasking programs based on the event-driven model. We will discuss how to implement coroutines in the following sections.