Unix - UNIX File Locking and Concurrent File Access
Introduction
In UNIX systems, multiple processes can access the same file at the same time. This is useful in many applications, but it can also create problems when two or more processes try to modify the same data simultaneously. For example, suppose two processes read the same bank account file, make different changes, and then write their results back. If there is no mechanism to coordinate their access, one process may overwrite the changes made by another process. This problem is known as a race condition.
UNIX file locking provides a way for processes to coordinate access to shared files. A process can place a lock on a file or a specific portion of a file to indicate that other processes should not modify that data at the same time. File locking is particularly important in databases, log-processing applications, configuration management, multi-user applications, and programs that share data through files.
What Is File Locking?
File locking is a mechanism that controls how multiple processes access the same file. A lock can be placed on an entire file or on a particular region of a file.
Consider a file called accounts.dat that contains customer account information. If Process A is updating a customer's balance, Process B should not modify that same record simultaneously. Process A can acquire a lock before performing the update. After completing the operation, it releases the lock so another process can access the data.
The important point is that UNIX file locking is generally used for coordination between cooperating processes. A lock does not automatically prevent every possible program from opening or modifying a file. Programs need to follow the locking protocol for the mechanism to be effective.
Why Concurrent File Access Is a Problem
Concurrent access occurs when multiple processes access the same resource at approximately the same time. Reading a file concurrently is usually harmless, but simultaneous modifications can cause inconsistent results.
For example, assume a file contains:
Balance = 1000
Process A wants to add 500, while Process B wants to subtract 200.
Process A reads:
1000
Process B also reads:
1000
Process A calculates:
1000 + 500 = 1500
Process B calculates:
1000 - 200 = 800
If Process A writes 1500 and Process B subsequently writes 800, the final value becomes 800. Process A's update has effectively been lost.
The expected result should have been:
1000 + 500 - 200 = 1300
File locking can prevent this type of race condition by ensuring that only one process modifies the protected data at a time.
Types of UNIX File Locks
UNIX provides several mechanisms for file locking. Two important approaches are advisory locking and mandatory locking.
Advisory Locking
Advisory locking is the commonly used approach on UNIX and UNIX-like systems.
With advisory locking, the operating system records that a file or file region has been locked, but processes are expected to cooperate by checking or respecting the lock.
For example:
-
Process A requests a lock.
-
The lock is granted.
-
Process A modifies the file.
-
Process B attempts to acquire the same lock.
-
Process B waits or receives an indication that the resource is locked.
-
Process A releases the lock.
-
Process B can then acquire the lock.
If a program completely ignores the locking protocol and directly modifies the file, advisory locking may not stop it.
This approach is flexible and is widely used because applications can decide how they want to coordinate access.
Mandatory Locking
Mandatory locking attempts to make the kernel enforce locking rules so that conflicting access can be blocked even when a process does not explicitly cooperate.
However, mandatory locking has important limitations and is not the preferred general-purpose approach on modern UNIX-like systems. Its availability and behavior depend on the operating system and filesystem configuration.
For most applications, advisory locking is the safer and more portable design choice.
flock() File Locking
The flock() interface provides a relatively simple way to apply locks to files.
A program can request:
-
A shared lock
-
An exclusive lock
-
Removal of a lock
A shared lock is generally used when a process only needs to read data. Multiple processes can normally hold shared locks simultaneously.
An exclusive lock is generally used when a process needs to modify data. It prevents other cooperating processes from obtaining conflicting locks.
Conceptually:
Shared Lock
|
+---- Process A reads
+---- Process B reads
+---- Process C reads
Exclusive Lock
|
+---- Process A modifies
|
+---- Other processes wait
A simplified command-line example is:
flock /tmp/mylock.lock -c "echo 'Processing data'"
Here, flock obtains a lock associated with the specified lock file before executing the command.
A common application is preventing multiple copies of a script from running simultaneously.
For example:
flock -n /tmp/backup.lock -c "/usr/local/bin/backup.sh"
The -n option requests a non-blocking lock. If another process already holds the lock, the command can fail immediately instead of waiting.
fcntl() File Locking
The fcntl() system call provides more flexible file-locking capabilities.
Unlike simple whole-file locking, fcntl() can lock specific portions of a file. This is called record locking or byte-range locking.
For example, imagine a file containing 1,000 customer records:
Record 1
Record 2
Record 3
...
Record 1000
Process A may need to update Record 100 while Process B updates Record 500.
Instead of locking the entire file, the application can lock only the relevant regions.
Conceptually:
File
------------------------------------------------
| Record 1 | ... | Record 100 | ... | Record 500 |
------------------------------------------------
LOCK LOCK
This allows unrelated parts of the file to be processed concurrently.
This capability makes fcntl() useful for applications that need fine-grained control over shared files.
Shared and Exclusive Locks
Locks can generally be categorized according to the type of access they permit.
Shared Lock
A shared lock is used when a process wants to read data without another cooperating process modifying the protected region.
Multiple processes can normally hold shared locks simultaneously.
For example:
Process A ---- Shared Lock
Process B ---- Shared Lock
Process C ---- Shared Lock
All three processes can read the protected data.
Exclusive Lock
An exclusive lock is used when a process needs to modify protected data.
Only one cooperating process can normally hold an exclusive lock for a particular region.
For example:
Process A ---- Exclusive Lock
Process B ---- Waiting
Process C ---- Waiting
After Process A finishes and releases the lock, another process can acquire it.
Blocking and Non-Blocking Locks
A process attempting to acquire a lock has two common choices.
Blocking Lock
With a blocking lock, the process waits until the requested lock becomes available.
For example:
Process A: Lock acquired
Process B: Requests lock
Process B: Waits
Process A: Releases lock
Process B: Lock acquired
This approach is useful when the operation must eventually be completed.
Non-Blocking Lock
With a non-blocking lock, the process does not wait.
If the lock is unavailable, the application receives an indication that another process already owns the conflicting lock.
This can be useful when an application should perform another task rather than remain blocked.
For example:
if lock_available:
process_file()
else:
report_already_running()
This technique is commonly used to prevent duplicate execution of administrative scripts.
File Locking and Race Conditions
A race condition occurs when the result of an operation depends on the unpredictable timing of multiple processes.
Consider two processes updating a shared counter:
Initial value = 10
Both processes read the value before either writes it.
Process A reads 10
Process B reads 10
Process A calculates 11
Process B calculates 11
Process A writes 11
Process B writes 11
The final value is:
11
But two increments should have produced:
12
Using a suitable lock changes the sequence:
Process A obtains lock
Process A reads 10
Process A writes 11
Process A releases lock
Process B obtains lock
Process B reads 11
Process B writes 12
Process B releases lock
The final result is now correct.
File Locking and Lock Granularity
Lock granularity refers to how much data is protected by a lock.
A coarse-grained lock may protect the entire file.
A fine-grained lock may protect only a small region.
For example:
Entire file:
[---------------- LOCKED ----------------]
versus:
[ Data ][ LOCK ][ Data ][ Data ][ LOCK ][ Data ]
A coarse-grained lock is easier to implement but can reduce concurrency because processes must wait even when they are working with unrelated data.
Fine-grained locking can improve performance, but it requires more careful programming and can introduce complicated synchronization problems.
Lock Files
A common UNIX technique is to create a dedicated lock file to indicate that an operation is already running.
For example:
/tmp/application.lock
An application can use this lock to prevent multiple instances from performing the same operation simultaneously.
This is especially useful for:
-
Backup scripts
-
Data synchronization jobs
-
Scheduled tasks
-
Log-processing programs
-
Database maintenance scripts
-
System administration scripts
A safer design uses an actual locking mechanism such as flock() rather than simply checking whether a lock file exists.
A simple existence check can suffer from a race condition:
Process A checks: file does not exist
Process B checks: file does not exist
Process A creates file
Process B creates file
Both processes may believe that they successfully obtained the lock.
Atomic locking mechanisms avoid this problem.
File Locking and Deadlocks
Incorrect locking can introduce another serious problem called a deadlock.
A deadlock occurs when two or more processes wait indefinitely for resources held by each other.
For example:
Process A:
Locks File 1
Waits for File 2
Process B:
Locks File 2
Waits for File 1
Neither process can continue.
A common strategy for reducing deadlock risk is to establish a consistent locking order.
For example, an application can always acquire:
File 1 → File 2 → File 3
rather than allowing different parts of the program to acquire locks in arbitrary orders.
Applications should also avoid holding locks longer than necessary.
Best Practices for UNIX File Locking
When implementing file locking, several practices should be followed.
First, keep the locked section as small as practical. A process should acquire a lock immediately before accessing shared data and release it as soon as the operation is complete.
Second, use the appropriate locking mechanism for the application. Simple whole-file coordination may be handled with flock(), while applications requiring byte-range control may benefit from fcntl().
Third, ensure that every cooperating process follows the same locking rules. Advisory locking only works reliably when applications respect the established protocol.
Fourth, carefully handle errors. A program should check whether a lock was successfully obtained instead of assuming that the operation always succeeds.
Finally, design the locking strategy to avoid deadlocks. Consistent lock ordering and short lock durations can significantly reduce synchronization problems.
Practical Example
Suppose a UNIX server runs a backup script every hour. If the previous backup is still running when the next scheduled execution begins, two backup processes could operate on the same files simultaneously.
A lock can prevent this:
Scheduled Backup
|
v
Acquire Lock
|
+---+---+
| |
Success Failed
| |
v v
Backup Exit
|
v
Release Lock
The first process obtains the lock and performs the backup. A second process attempting to start during the same period cannot obtain the lock and can exit or wait, depending on the application's requirements.
Conclusion
UNIX file locking is an important mechanism for safely managing concurrent access to shared files. It helps applications coordinate reading and writing operations, prevent race conditions, maintain data consistency, and avoid accidental overwriting of information.
The most important concepts include advisory locking, shared locks, exclusive locks, blocking and non-blocking operations, flock(), fcntl(), byte-range locking, lock files, and deadlock prevention. Understanding these mechanisms is particularly valuable when developing multi-process UNIX applications, administrative scripts, data-processing systems, and applications that rely on shared files.
Sign In
Sign Up
Forgot Password