Unix - System Call Tracing and Debugging Using strace and truss in UNIX
1. Introduction
System call tracing is an important debugging technique used in UNIX and UNIX-like operating systems to understand how an application communicates with the operating system kernel. Whenever a program performs operations such as opening a file, reading data, writing information, creating a process, or accessing system resources, it requests services from the kernel through system calls.
Sometimes, an application may fail to perform an operation even though its source code appears correct. In such situations, developers and system administrators need to understand what happens internally when the program executes. System call tracing helps identify these problems by monitoring the system calls made by an application.
UNIX provides specialized utilities such as strace and truss to observe system calls, examine their arguments, and identify errors. These tools are particularly useful for troubleshooting application failures, permission problems, missing files, and unexpected program behaviour.
2. Understanding System Calls in UNIX
A system call is a mechanism through which an application requests a service from the operating system kernel. Applications cannot directly access all hardware and protected system resources. Instead, they communicate with the kernel using predefined system calls.
For example, when a program needs to read a file, it uses system calls to request the file's contents from the operating system.
Some commonly used system calls include:
-
open()– Opens a file or other resource. -
read()– Reads data from a file or input source. -
write()– Writes data to a file or output destination. -
close()– Closes an opened file descriptor. -
fork()– Creates a new process. -
execve()– Executes a new program. -
wait()– Waits for a child process to complete. -
stat()– Retrieves information about a file. -
mmap()– Maps files or devices into memory.
System call tracing allows administrators to observe when these operations occur and whether they complete successfully.
3. What Is strace in UNIX?
strace is a command-line debugging utility commonly available on Linux systems. It monitors the system calls made by a running process or a newly executed program.
It displays detailed information about system call names, arguments, return values, and errors. By examining this information, developers can understand how an application interacts with the operating system.
For example, if an application cannot locate a configuration file, strace can reveal which file paths the application attempted to access and whether the operating system returned a file-not-found error.
The basic syntax of the strace command is:
strace command
Example:
strace ls
This command executes the ls utility while displaying the system calls generated during its execution.
The output may contain information about file access, directory operations, memory management, and process execution.
4. Important Features of strace
The strace utility provides several features that help developers and administrators investigate application behaviour.
System Call Monitoring: It records the system calls performed by an application, helping users understand how programs interact with the kernel.
Error Detection: It identifies failed system calls and displays error information such as ENOENT, which indicates that a requested file or directory does not exist.
File Access Tracking: It helps identify which files and directories an application attempts to open, read, or modify.
Process Monitoring: It can trace process creation and execution operations, making it useful for understanding how applications launch other programs.
Performance Investigation: It provides timing information for system calls, helping identify operations that take unusually long.
Output Redirection: It allows tracing information to be saved in a file for later examination.
5. Common strace Commands and Their Uses
The following commands demonstrate how strace can be used for system call tracing.
1. Trace a Command
strace ls
This command monitors all traceable system calls made while executing the ls command.
2. Save Trace Output to a File
strace -o trace.log ls
This command stores the tracing information in a file named trace.log. Administrators can examine the file later without running the command again.
3. Trace a Running Process
strace -p 1234
This command attaches strace to the running process with process ID 1234 and monitors its system calls. The user generally needs sufficient permissions to attach to another process.
4. Trace Only File-Related System Calls
strace -e trace=file ls
This command filters the output to show file-related system calls, making it easier to investigate file access problems.
5. Trace Network-Related System Calls
strace -e trace=network command
This command focuses on network-related system calls generated by the specified command.
6. Display System Call Execution Time
strace -T ls
This command displays the time spent inside each system call, helping identify operations that may be taking longer than expected.
6. What Is truss in UNIX?
truss is a system call tracing and debugging utility available on several UNIX and UNIX-like operating systems, including Solaris and some BSD-derived systems.
It performs a function similar to strace by monitoring the system calls and signals generated by an application. It helps administrators understand how a program communicates with the operating system and identify problems during execution.
The truss utility is particularly useful in traditional UNIX environments where administrators need to investigate application failures, system resource access, and process behaviour.
The basic syntax is:
truss command
Example:
truss ls
This command executes the ls utility and displays the system calls made during its execution.
The output includes information about system call names, arguments, return values, and execution results.
7. Important Features of truss
The truss utility offers several useful capabilities for debugging and system administration.
System Call Tracing: It monitors the system calls performed by an application and displays how the program interacts with the kernel.
Signal Monitoring: It helps identify signals received by a process, including signals related to process termination and interruptions.
Process Behaviour Analysis: It provides information about process execution and operations involving child processes.
Error Investigation: It helps detect unsuccessful system calls and identify possible reasons for application failures.
Application Debugging: Developers can use tracing information to understand unexpected program behaviour and investigate operating system interactions.
Execution Monitoring: It provides useful information about the sequence of operations performed by an application.
8. Common truss Commands and Their Uses
Some commonly used truss commands are explained below. Exact options may vary between UNIX implementations.
1. Trace a Command
truss ls
This command monitors the system calls generated by the ls utility.
2. Save Trace Information to a File
truss -o trace.log ls
This command saves the tracing output in a file named trace.log.
3. Trace an Existing Process
truss -p 1234
This command attaches the tracing utility to an existing process with process ID 1234, where supported.
4. Trace System Calls and Signals
truss -f command
On implementations supporting this option, it follows child processes created by the specified command.
5. Display Detailed Execution Information
truss -d ls
On Solaris-style implementations, this option adds relative timestamp information to the trace output.
9. Difference Between strace and truss
Both utilities are designed to monitor system calls, but they are associated with different operating system environments.
|
Feature |
strace |
truss |
|---|---|---|
|
Primary environment |
Linux |
Solaris and some BSD-derived systems |
|
Main purpose |
System call tracing |
System call and signal tracing |
|
File monitoring |
Supported |
Supported |
|
Process tracing |
Supported |
Supported |
|
Signal monitoring |
Supported |
Supported |
|
Output redirection |
Supported |
Supported |
|
Command syntax |
|
|
The availability and exact command options depend on the operating system and its version. These tools are not completely interchangeable because their supported features and syntax differ.
10. Practical Applications of System Call Tracing
System call tracing is useful in several real-world situations.
Troubleshooting Missing Files: When an application cannot locate a required file, tracing helps identify the file paths it attempts to access.
Identifying Permission Problems: If an application receives a permission-denied error, tracing can reveal which file or resource caused the failure.
Investigating Application Crashes: Developers can examine the final system calls and signals before a program terminates to understand possible causes.
Monitoring Resource Access: Administrators can identify how applications access files, directories, devices, and other operating system resources.
Debugging Network Problems: Tracing network-related system calls can help investigate connection failures and communication errors.
Performance Analysis: System call timing information can help identify slow operations and potential performance bottlenecks.
11. Advantages of System Call Tracing
System call tracing offers several benefits:
-
It provides visibility into the internal operating system interactions of applications.
-
It helps identify errors that may not be visible through normal application output.
-
It reduces debugging time by showing the exact system calls that fail.
-
It helps administrators investigate file access and permission-related issues.
-
It supports troubleshooting of complex applications and background processes.
-
It helps developers understand application behaviour without modifying the original source code.
12. Limitations and Precautions
Although system call tracing is a powerful debugging technique, it has certain limitations.
-
Tracing produces large amounts of output for applications that perform many system calls.
-
Monitoring can introduce additional execution overhead and affect application performance.
-
Some processes cannot be traced without appropriate administrative privileges.
-
Sensitive information, such as file paths and data passed to system calls, may appear in trace logs.
-
System call tracing does not automatically identify or fix programming errors; users must interpret the collected information.
-
The commands and available tracing options may differ between UNIX and Linux distributions.
Administrators should therefore use tracing carefully, protect log files, and avoid leaving unnecessary tracing sessions active on production systems.
13. Conclusion
System call tracing is an essential technique for understanding how applications interact with the UNIX operating system. Utilities such as strace and truss allow developers and system administrators to monitor system calls, examine their arguments, identify errors, and investigate unexpected application behaviour.
While strace is widely used in Linux environments, truss is commonly associated with traditional UNIX systems such as Solaris. Both tools help simplify application debugging by providing detailed information about operations performed at the operating system level. A clear understanding of these utilities enables administrators to troubleshoot software problems, investigate resource access, and improve the reliability of UNIX-based applications.