Unix - UNIX Shared Libraries and Dynamic Linking
UNIX shared libraries and dynamic linking are important concepts that allow multiple programs to use common library code without storing a separate copy of that code inside every executable. This approach reduces executable file size, saves memory, simplifies software maintenance, and makes it possible to update libraries independently from applications.
1. What Is a Shared Library?
A shared library is a collection of compiled functions and other reusable code that can be used by multiple programs. Instead of copying the library code into every application during compilation, a program can refer to the shared library and load the required code when the program runs.
On many UNIX-like systems, shared libraries commonly use extensions such as .so, which stands for "shared object." For example:
libmath.so
libnetwork.so
libexample.so
A program that needs functions from libmath.so does not necessarily contain the complete implementation of those functions. Instead, it contains information that allows the system to locate and use the library.
This is different from a static library, where the required library code is copied into the executable during the linking stage.
2. Static Libraries vs Shared Libraries
UNIX systems commonly support both static and shared libraries.
A static library is linked directly into an executable. Once the program has been built, the executable contains the required library code.
For example:
Application
|
+---- Library Code
|
+---- Application Code
With a shared library, the executable contains references to the library rather than incorporating all of its code.
Application
|
+---- Reference to Shared Library
|
+---- libexample.so
The major advantage of shared libraries is that several applications can use the same library code.
For example, if ten programs use the same shared library, the operating system can potentially keep one copy of the library's code in physical memory and allow the programs to share it.
3. What Is Dynamic Linking?
Dynamic linking is the process of connecting an executable with shared libraries during program loading or execution rather than placing all library code into the executable during compilation.
Consider a program that calls a function called:
printf()
The application may not contain the complete implementation of printf(). Instead, it can depend on a system library that provides the function.
When the program starts, the dynamic linker helps locate the required library and resolves the program's references to functions and variables contained in that library.
The general process looks like this:
Source Code
|
v
Compilation
|
v
Object File
|
v
Linking
|
v
Executable
|
v
Program Starts
|
v
Dynamic Linker
|
v
Shared Libraries Loaded
|
v
Symbols Resolved
|
v
Program Executes
4. Dynamic Linker
The dynamic linker is a system component responsible for preparing an executable that uses shared libraries.
Depending on the UNIX-like operating system, the dynamic linker may have different names. On Linux systems, it is commonly associated with the runtime loader such as:
ld.so
or
ld-linux.so
Its responsibilities include locating required shared libraries, loading them into memory, resolving symbols, and preparing the program for execution.
For example, if an executable requires:
libexample.so
the dynamic linker searches appropriate library locations and loads the required library.
5. What Are Symbols?
A symbol represents a function, variable, or other identifiable entity that can be referenced by another piece of compiled code.
For example:
int calculate(int a, int b)
{
return a + b;
}
The function calculate can become a symbol in the compiled object or shared library.
Another program can call:
calculate(10, 20);
The linker needs to determine where the implementation of calculate exists.
This process is known as symbol resolution.
If a required symbol cannot be found, the program may fail to start or may produce a dynamic linking error.
6. Compile-Time and Run-Time Linking
Dynamic linking involves multiple stages.
During compilation, the compiler converts source code into object code.
For example:
program.c
|
v
program.o
The linker then creates an executable and records information about required shared libraries and symbols.
When the executable starts, the dynamic linker loads the necessary shared libraries and resolves the required references.
Therefore, the complete process can be viewed as:
Source Code
|
v
Compiler
|
v
Object Code
|
v
Linker
|
v
Executable
|
v
Dynamic Loader
|
v
Shared Library
7. Library Search Paths
One important aspect of dynamic linking is finding the correct shared library.
UNIX-like systems can search several locations for shared libraries.
Common library directories include:
/lib
/usr/lib
/usr/local/lib
The exact locations depend on the operating system and configuration.
On Linux systems, the LD_LIBRARY_PATH environment variable can also be used to specify additional library search locations.
For example:
export LD_LIBRARY_PATH=/opt/mylibs:$LD_LIBRARY_PATH
This tells the runtime loader to consider /opt/mylibs when searching for shared libraries.
However, relying heavily on LD_LIBRARY_PATH for system-wide configuration can create maintenance and security problems. System administrators generally prefer appropriate system library configuration mechanisms for permanent installations.
8. Shared Library Naming
Shared libraries usually follow naming conventions.
A common pattern is:
libname.so
For example:
libssl.so
libpthread.so
libz.so
Libraries can also contain version information:
libexample.so.1
libexample.so.1.2.3
The version number helps maintain compatibility between different releases of a library.
A symbolic link may provide the generic library name:
libexample.so
|
v
libexample.so.1
|
v
libexample.so.1.2.3
This arrangement allows applications to request a compatible library version while the actual file can point to a specific release.
9. ABI Compatibility
One important concept in shared libraries is the Application Binary Interface, commonly called ABI.
The ABI defines low-level rules that compiled programs rely on when interacting with libraries. These rules can include calling conventions, data representations, symbol names, and binary compatibility requirements.
Suppose an application was compiled against a particular version of a library. If a newer version changes the ABI in an incompatible way, the existing application may fail even though the source-level function names appear similar.
For this reason, library developers generally try to maintain backward ABI compatibility when releasing compatible versions.
10. Benefits of Shared Libraries
Shared libraries provide several important advantages.
Reduced Executable Size
Applications do not need to contain complete copies of commonly used library code.
This can significantly reduce executable sizes.
Memory Sharing
Multiple processes can potentially share the same read-only library code in physical memory.
For example:
Program A ----\
Program B -----+---- Shared Library
Program C ----/
Instead of maintaining separate physical copies of identical library code, the operating system can share appropriate memory pages.
Easier Maintenance
If a library contains a bug or security vulnerability, updating the library can potentially fix the problem for multiple applications without recompiling every application.
However, applications may need to be restarted to use the newly loaded library version.
Code Reuse
Developers can create reusable libraries containing common functionality.
This avoids repeatedly implementing the same functions in different applications.
11. Disadvantages of Shared Libraries
Shared libraries also introduce some challenges.
Dependency Problems
An application may require a specific library that is missing from the system.
For example:
error while loading shared libraries:
libexample.so: cannot open shared object file
The program may fail to start until the required library is installed or made available.
Version Conflicts
Different applications may require different versions of the same library.
This can create compatibility problems if the system does not provide suitable versions.
Runtime Complexity
Dynamic linking requires the loader to locate libraries and resolve symbols during program startup or execution.
This adds complexity compared with a completely statically linked executable.
12. Useful UNIX Tools
UNIX-like systems provide several utilities for examining shared libraries.
ldd
The ldd command can show the shared-library dependencies of an executable.
For example:
ldd myprogram
A result may look conceptually like:
libc.so.6 => /lib/libc.so.6
libm.so.6 => /lib/libm.so.6
This is useful when diagnosing missing library dependencies.
ldconfig
On Linux systems, ldconfig helps maintain the shared-library cache and symbolic links used by the runtime loader.
An administrator may use:
sudo ldconfig
after installing libraries into configured system library directories.
readelf
readelf can display information contained in ELF executable and library files.
For example:
readelf -d myprogram
This can provide information about dynamic dependencies and other dynamic-linking information.
nm
The nm command can display symbols contained in object files and libraries.
For example:
nm libexample.a
It can help developers investigate which functions and variables are available as symbols.
13. Lazy and Immediate Symbol Resolution
Dynamic linking can resolve symbols in different ways.
With immediate resolution, required symbols are resolved when the program starts.
With lazy resolution, some symbols may be resolved only when the program actually uses them.
Lazy resolution can reduce unnecessary work during program startup because functions that are never called may not need to be resolved immediately.
However, it can also mean that a missing symbol becomes apparent only when the corresponding function is first used.
14. Position-Independent Code
Shared libraries commonly use Position-Independent Code, or PIC.
Position-independent code is designed so that the same compiled code can operate correctly regardless of where it is loaded into a process's address space.
This is especially useful for shared libraries because different processes may load the same library at different virtual addresses.
On systems using modern security mechanisms such as Address Space Layout Randomization, position independence also plays an important role in making predictable memory addresses harder to exploit.
15. Runtime Loading of Libraries
Libraries do not always have to be declared as ordinary dependencies at program startup.
Programs can also load libraries dynamically while they are running.
On UNIX-like systems, functions such as:
dlopen()
dlsym()
dlclose()
are commonly used for this purpose.
A simplified example is:
void *handle;
handle = dlopen("libexample.so", RTLD_LAZY);
The program can then use dlsym() to locate a particular function within the loaded library.
Finally, dlclose() can release the library handle.
This technique is useful for plugin architectures.
For example:
Main Application
|
+---- Plugin A
|
+---- Plugin B
|
+---- Plugin C
The main application can load different plugins when required without permanently incorporating all plugin code into the executable.
16. Shared Libraries and Security
Shared libraries are also important from a security perspective.
If a program loads an unexpected library, malicious code could execute with the privileges of that program.
For this reason, administrators should carefully control library search paths and permissions.
Environment variables such as:
LD_LIBRARY_PATH
can be useful during development but should be handled carefully, especially when executing privileged programs.
Secure software environments should ensure that libraries come from trusted locations and that unauthorized users cannot modify them.
17. Example of Dynamic Linking
Consider an application called:
calculator
Suppose it uses mathematical functions provided by a shared library.
The application might have this structure:
calculator
|
+---- Application Code
|
+---- Reference to libmath.so
When the application starts:
1. Operating system loads calculator
2. Dynamic loader examines its dependencies
3. libmath.so is located
4. libmath.so is loaded into memory
5. Required symbols are resolved
6. calculator begins execution
If libmath.so cannot be found, the application may terminate before normal execution begins.
18. Practical Troubleshooting
When a UNIX program reports a shared-library error, administrators can follow a systematic approach.
First, identify the missing library from the error message.
Next, check the application's dependencies:
ldd application
Then determine whether the library exists in an expected directory.
If the library exists in a custom directory, the administrator can investigate the configured library search paths.
For example:
echo $LD_LIBRARY_PATH
On systems using a library cache, the administrator can also inspect the configured library paths and update the cache when appropriate.
This process helps distinguish between a missing library, an incorrect search path, and an incompatible library version.
19. Summary
UNIX shared libraries provide reusable compiled code that can be accessed by multiple applications. Dynamic linking allows applications to connect to these libraries during program loading or execution rather than embedding all library code directly into the executable.
The dynamic linker locates required libraries, loads them into memory, and resolves symbols used by the application. Shared libraries reduce executable size, support memory sharing, encourage code reuse, and simplify software maintenance.
Important concepts associated with this topic include shared objects, dynamic loaders, symbols, library search paths, ABI compatibility, position-independent code, runtime loading, and library dependency troubleshooting. Understanding these concepts is particularly valuable for UNIX system administrators, software developers, and anyone working with compiled applications and system-level software.
Sign In
Sign Up
Forgot Password