Linux - Linux Process Scheduling and CPU Affinity

Linux process scheduling is the mechanism used by the operating system to decide which process or thread should run on a CPU, when it should run, and for how long. Since a computer may have many processes competing for a limited number of CPU cores, the Linux kernel uses scheduling algorithms to distribute CPU time efficiently. CPU affinity complements scheduling by allowing administrators or applications to control which CPU cores a process or thread is permitted to execute on.

1. What is a Process?

A process is a running instance of a program. When you open a text editor, web browser, terminal, or any other application, Linux creates processes to execute the required instructions.

A process contains information such as:

  • Process ID (PID)

  • Program instructions

  • Memory allocation

  • Open files

  • CPU registers and execution state

  • Scheduling information

  • User and group ownership

  • Resource usage information

For example, you can use the following command to view running processes:

ps aux

Each process receives a unique process ID. You can also inspect a particular process with:

ps -p PID -o pid,ppid,ni,pri,stat,comm

Here, PID identifies the process, PPID identifies its parent process, NI represents its nice value, PRI represents scheduling priority, and STAT shows the process state.

2. Why Does Linux Need a Scheduler?

A modern Linux system may have hundreds or thousands of processes and threads, while the system may have only a few CPU cores.

For example, consider a computer with four CPU cores and hundreds of runnable threads. It is impossible for every thread to execute simultaneously on the same four cores.

The Linux scheduler therefore continuously makes decisions such as:

  • Which runnable task should execute next?

  • Which CPU should execute it?

  • Should the currently running task continue?

  • Should another task receive CPU time?

  • Should a task with higher priority be scheduled?

  • Should a task be moved from one CPU to another?

This process is known as CPU scheduling.

3. Runnable and Non-Runnable Processes

Not every process requires the CPU at every moment.

A process can be waiting for an event, such as:

  • Keyboard input

  • Network data

  • Disk I/O

  • A timer

  • Another process

  • A synchronization event

When a process is waiting for such an event, the scheduler can allow another runnable task to use the CPU.

This improves CPU utilization.

For example, suppose a process is reading a large file from disk. While the process waits for the disk operation to complete, another process can use the CPU. Once the disk operation finishes, the waiting process becomes runnable again.

4. Preemptive Scheduling

Linux uses preemptive scheduling. This means the kernel can interrupt a currently running task and give CPU time to another task.

Consider two processes:

Process A
Process B

If Process A is currently running and the scheduler determines that Process B should run, the kernel can temporarily stop Process A and allow Process B to execute.

When Process A is scheduled again, Linux restores its execution state and continues it.

This switching process is called a context switch.

5. Context Switching

A context switch occurs when the CPU stops executing one task and starts executing another.

The kernel must preserve the state of the task being interrupted, including relevant processor and scheduling information. It then loads the state required to resume the next task.

Conceptually:

Task A running
     |
     v
Scheduler decision
     |
     v
Save Task A state
     |
     v
Load Task B state
     |
     v
Task B running

Context switching makes multitasking possible, but it also has a cost. Excessive context switching can reduce performance because the CPU spends more time switching between tasks instead of doing useful application work.

6. Linux Scheduling Classes

Linux does not use one single scheduling policy for every type of workload. It provides different scheduling classes and policies.

Common scheduling policies include:

  • SCHED_OTHER

  • SCHED_FIFO

  • SCHED_RR

  • SCHED_BATCH

  • SCHED_IDLE

  • SCHED_DEADLINE

The normal policy for ordinary applications is generally SCHED_OTHER, while specialized workloads can use other policies.

SCHED_OTHER

This is the normal scheduling policy for typical applications and background processes.

It is designed to provide fair CPU access among ordinary runnable tasks.

SCHED_FIFO

SCHED_FIFO is a real-time scheduling policy.

A runnable FIFO task with an appropriate real-time priority can run until it:

  • Blocks

  • Voluntarily yields the CPU

  • Is preempted by a higher-priority real-time task

Because real-time policies can significantly affect system responsiveness, they should be used carefully.

SCHED_RR

SCHED_RR, or round-robin scheduling, is another real-time policy.

Multiple real-time tasks at the same priority can share CPU time using a time quantum.

For example:

Task A -> CPU
Task B -> CPU
Task C -> CPU
Task A -> CPU
...

This allows multiple same-priority real-time tasks to receive CPU time.

SCHED_BATCH

This policy is intended for CPU-intensive batch workloads where interactive responsiveness is less important.

Examples include:

  • Large data processing

  • Offline calculations

  • Batch conversion jobs

SCHED_IDLE

This policy is intended for tasks that should receive CPU time only when there is little other runnable work.

It can be useful for extremely low-priority background workloads.

SCHED_DEADLINE

SCHED_DEADLINE is designed for workloads with explicit timing requirements.

It allows scheduling based on concepts such as:

  • Runtime

  • Deadline

  • Period

This is an advanced real-time scheduling mechanism and is primarily relevant to specialized applications.

7. Process Priority and Nice Values

Linux provides the concept of niceness for ordinary processes.

A nice value influences how the scheduler treats a process relative to other normal tasks.

The typical range is:

-20  to  +19

A lower nice value means a higher scheduling priority among normal tasks, while a higher nice value means the process is more willing to give CPU time to other tasks.

You can start a command with a specific nice value:

nice -n 10 command

For example:

nice -n 10 ./backup.sh

This starts the backup process with a relatively lower scheduling priority.

You can inspect nice values using:

ps -eo pid,ni,comm

8. Changing the Priority of a Running Process

The renice command can change the nice value of an existing process.

For example:

renice 10 -p 1234

Here, 1234 represents the PID.

You can verify the result with:

ps -p 1234 -o pid,ni,comm

Increasing the nice value generally makes an ordinary process less favored relative to other normal processes.

Changing a process to a negative nice value generally requires appropriate privileges because it gives the process greater scheduling preference.

9. CPU Cores and Multicore Systems

Modern computers usually have multiple CPU cores.

For example:

CPU 0
CPU 1
CPU 2
CPU 3

Linux can schedule different runnable tasks on different cores.

For example:

CPU 0 -> Process A
CPU 1 -> Process B
CPU 2 -> Process C
CPU 3 -> Process D

This allows multiple tasks to execute concurrently.

However, scheduling tasks across CPUs is not always as simple as assigning each task to any available core. The scheduler also considers factors such as workload distribution, processor utilization, cache locality, and task migration.

10. What is CPU Affinity?

CPU affinity determines the CPU cores on which a process or thread is allowed to run.

By default, a process may be permitted to run on multiple CPUs.

For example:

Process A
Allowed CPUs: 0, 1, 2, 3

With CPU affinity, you could restrict it to:

Process A
Allowed CPUs: 2, 3

The scheduler can then run the process only on CPU 2 or CPU 3.

CPU affinity does not force a process to continuously run on one particular CPU. Instead, it establishes the set of CPUs that are permitted for that task.

11. Using taskset

The taskset command is commonly used to view or modify CPU affinity.

To start a program on a particular CPU:

taskset -c 2 ./application

This allows the application to run on CPU 2.

To allow it to run on CPUs 2 and 3:

taskset -c 2,3 ./application

You can also use a CPU mask.

For example:

taskset 0x03 ./application

A CPU mask represents the CPUs available to the process. Understanding hexadecimal CPU masks becomes particularly useful when working with systems containing many CPU cores.

12. Checking the CPU Affinity of a Process

You can check a running process using:

taskset -cp 1234

Example output might look like:

pid 1234's current affinity list: 0-3

This means the process can run on CPUs 0 through 3.

If the output is:

pid 1234's current affinity list: 2

the process is restricted to CPU 2.

13. CPU Affinity and Cache Locality

One important reason for using CPU affinity is CPU cache locality.

Modern processors have several levels of cache. Frequently accessed data and instructions can remain in cache, allowing the CPU to access them much faster than retrieving them from main memory.

If a process frequently moves between CPU cores, some of the useful cached information may not remain available in the same way on the new CPU.

Restricting a workload to a smaller group of CPUs can sometimes improve cache locality and reduce unnecessary migrations.

However, CPU affinity does not automatically improve performance. Restricting a process too aggressively can actually hurt performance if the selected CPUs become overloaded.

14. CPU Affinity and CPU Load

Consider a system with eight CPUs:

CPU 0
CPU 1
CPU 2
CPU 3
CPU 4
CPU 5
CPU 6
CPU 7

Suppose an intensive process is restricted to CPU 0:

Process A -> CPU 0 only

If Process A requires substantial CPU time, CPU 0 may become heavily loaded while CPUs 1 through 7 remain relatively underutilized.

Therefore, affinity should be applied carefully.

A better configuration might be:

Process A -> CPU 0-3
Process B -> CPU 4-7

This gives both workloads a reasonable amount of CPU capacity while still controlling where they can execute.

15. CPU Affinity for Threads

CPU affinity can also be applied at the thread level.

This is particularly important for multithreaded applications.

Suppose an application has four worker threads:

Application
 |
 +-- Thread 1
 +-- Thread 2
 +-- Thread 3
 +-- Thread 4

You could potentially configure different threads to run on different CPUs.

This can be useful in specialized applications such as:

  • High-performance computing

  • Network processing

  • Database workloads

  • Real-time systems

  • Media processing

  • Scientific computation

Thread-level CPU placement requires careful analysis because forcing threads onto specific CPUs can sometimes interfere with the scheduler's ability to balance workloads.

16. CPU Affinity and NUMA Systems

CPU affinity becomes especially important on large servers with NUMA, or Non-Uniform Memory Access, architectures.

In NUMA systems, CPUs are organized into nodes, and memory access time can depend on where the memory is physically located relative to the CPU.

For example:

NUMA Node 0
CPU 0
CPU 1
Memory 0

NUMA Node 1
CPU 2
CPU 3
Memory 1

If a workload executes primarily on Node 0 but frequently accesses memory located on Node 1, memory access can be less efficient than accessing local memory.

Therefore, CPU placement and memory placement can be considered together when optimizing large systems.

Linux provides tools such as numactl for controlling CPU and memory placement.

For example:

numactl --cpunodebind=0 --membind=0 ./application

This requests CPU execution and memory allocation from NUMA node 0.

17. CPU Affinity Does Not Equal CPU Reservation

A common misunderstanding is that CPU affinity reserves a CPU for a process.

It does not.

Suppose:

Process A -> CPU 2

This does not necessarily mean CPU 2 is exclusively reserved for Process A.

Other tasks may also be permitted to run on CPU 2.

Affinity simply restricts Process A's allowed CPU set.

If you need stronger CPU resource isolation or allocation guarantees, Linux provides other mechanisms, including cgroups and CPU control facilities.

18. Viewing CPU Information

Before tuning scheduling or affinity, it is useful to understand the system's CPU topology.

You can use:

lscpu

This provides information about:

  • CPU architecture

  • Number of CPUs

  • CPU cores

  • Threads

  • Sockets

  • NUMA nodes

  • CPU model

  • CPU frequency information

You can also use:

nproc

to display the number of processing units available to the system according to the command's configuration.

19. Monitoring Scheduling Activity

Several Linux tools can help analyze CPU usage and scheduling behavior.

top provides a real-time overview:

top

htop, where installed, provides a more interactive view:

htop

ps provides detailed process information:

ps -eo pid,ppid,psr,ni,pri,stat,comm

The PSR field can indicate the processor on which a task was last observed running.

For deeper performance analysis, Linux also provides tools such as:

perf

perf can be used for advanced performance profiling and analysis of CPU behavior, scheduling, context switches, and other kernel/application performance characteristics.

20. Process Scheduling Versus CPU Affinity

These two concepts are related but different.

Process Scheduling CPU Affinity
Decides which runnable task gets CPU time Restricts which CPUs a task can use
Managed primarily by the kernel scheduler Configurable by administrators/applications
Balances CPU workloads Controls CPU placement
Applies scheduling policies and priorities Defines an allowed CPU set
Operates continuously Can be configured and changed when required

In simple terms:

Scheduling answers:
"Which task should run now?"

CPU affinity answers:
"Which CPUs is this task allowed to run on?"

21. Practical Example

Suppose a server has eight CPUs and runs three major workloads:

Web application
Database
Backup process

The backup process may consume significant CPU resources.

If it competes freely with the web application and database, it could affect system responsiveness.

One possible approach is to restrict the backup process:

taskset -c 6,7 ./backup.sh

Now the backup workload is permitted to run on CPUs 6 and 7, while the other CPUs remain available to other workloads.

However, this should be evaluated using monitoring tools. If CPUs 6 and 7 become overloaded, the restriction could make the backup slower than allowing Linux to distribute it across more CPUs.

22. Important Considerations

CPU scheduling and affinity should be used based on measured performance rather than assumptions.

Important considerations include:

  1. Do not unnecessarily restrict CPU affinity.
    The Linux scheduler is generally very capable of distributing workloads.

  2. Monitor CPU utilization before and after changes.
    Tools such as top, htop, vmstat, and perf can help identify bottlenecks.

  3. Consider cache locality for performance-sensitive workloads.
    Keeping workloads on suitable CPUs can sometimes improve performance.

  4. Consider NUMA topology on large servers.
    CPU placement and memory placement can interact significantly.

  5. Be careful with real-time scheduling policies.
    Incorrect real-time configuration can interfere with normal system operation.

  6. Do not confuse affinity with resource reservation.
    Affinity limits CPU eligibility; it does not necessarily dedicate a CPU exclusively to a process.

23. Summary

Linux process scheduling is responsible for efficiently sharing CPU resources among processes and threads. The scheduler considers scheduling policies, priorities, task states, CPU availability, and system workload when determining what should execute.

CPU affinity provides an additional mechanism for controlling where processes or threads can run. Tools such as taskset, ps, lscpu, numactl, and perf help administrators inspect and manage CPU placement and performance.

Understanding these concepts is particularly valuable when working with multicore servers, high-performance applications, real-time workloads, NUMA systems, databases, and CPU-intensive applications. The most important principle is that scheduling determines when and which task runs, while CPU affinity determines where that task is permitted to run.