Linking Unix utilities

Introduction
Linker diagnostics
Which library do I need?
Ordering the command-line
OS-specific issues
- AIX dynamic linking
- HP-UX dynamic linking
- Linux's dl library
a summary perspective: Larry's rules of linking

Introduction

Suppose you want a particular utility (much of my consultation has to do with Tcl, Perl, Java, Python, nsgmls, system and network administration, ...) on a particular Unix host. You have sources and a generation procedure. You apply the latter to the former--but it fails. Now what?

In Unix-speak, we categorize such problems as either compile-time, or link-time. This document describes the link-time solutions that I've seen over and over again. Larry Virden covers related territory in his post on linking advice. Simon Burr writes specifically about -R searching, with a slight Solaris bent.

... [Ref linking concepts; discovery; order; Unix]

[Explain that dynamic linking is not all it's cracked up to be.]

Linker diagnostics

Almost all faults detected by the Unix linker manifest as:

a missing library; or
a missing reference

The ldd command is invaluable for diagnosing references involving dynamically-linked libraries. [explain prefix, LD_..., ...]

Which library do I need?

First it's important to understand what a library is.

Your application appears to depend on an entry point, "SOME-SYSTEM-FUNCTION"; where will you find it? Here are the best ways:

`man SOME-SYSTEM-FUNCTION`: this answers the overwhelming majority of needs. "man" is a command universally available on (properly installed) Unix systems which delivers "help"-style documentation. There are a few mis-written "man" pages, and also mangled systems from which they've been stripped; if you're in one of those unhappy situations, you'll need to
find / -name "lib*" -exec nm -o {} \; | grep SOME-SYSTEM-FUNCTION | grep T
or

	LIST="/usr/lib/*.a /usr/lib/lib*.so*"
	for F in $LIST
	do
		echo "$F:"
		nm $F | grep SOME-SYSTEM-FUNCTION
	done

[explain]

Ordering the command-line

It surprises some developers that the order in which libraries are linked matters. Thus, for example,

cc -o myprogram myprogram.o -lX11 -lmylib

might fail while

cc -o myprogram myprogram.o -lmylib -lX11

gives happy results.

Operating-system-specific issues

HP-UX dynamic linking

HP-UX developers historically have many problems with dynamic linking. Exacerbating all of them are the diagnostics the compiler, linker, and loader generate, which are sometimes cruelly misleading and often uninformative. I've just started to collect below the pertinent information. Complaints that something is "out of memory", for example, often resolve to unavailable shared libraries. The good news, though, is that the HP-UX Linker and Libraries User's Guide is on-line. The manual page for ld under HP-UX also contains a great deal of information. Moreover, the README.hpux [provide link] from the Perl5 distribution summarizes construction of HP-UX shared libraries.

HP-UX shared libraries can be specified

at linktime with arguments such as "-Wl,+s,+b,/usr/local/lib:." on the command-line or in LDFLAGS, or
at runtime through the environment variable SHLIB_PATH.

Some users have found success by assigning SHLIB_PATH to include the directory (/usr/local/lib, for example) with shared libraries. I've come across reports that it sometimes helps to unsetenv LD_LIBRARY_PATH when relying on SHLIB_PATH.

Several [itemize ...] versions of Tcl [and other common tools--itemize ...] implement a common error with HP-UX dynamic linking. The source file tclLoadShl.c includes the line

     handle = shl_load(fileName, BIND_IMMEDIATE, 0L);

rather than the correct

     handle = shl_load(fileName, DYNAMIC_PATH | BIND_IMMEDIATE, 0L);

Tcl-ers code around this at the script level by doing an absolute load [document ...] rather than package require.

Examination of errno after shl_findsym sometimes is useful.

[Explain ldd.]

Linux's dl library

There are at least two distinct problem symptoms with dynamic-linking that Linux users frequently report: that

the configure for Tcl and/or derivatives think that dynamic linking is not possible; and
a makefile which attempts dynamic linking fails for not resolving "... -ldl ..." adequately.

To the best of my knowledge, all instances of the former problem have been traced back to an inconsistency between configure and other sources, typically the result of incomplete application of a patch, mis-installation of a compiler and its ancillary libraries, or some sort of cross-OS contamination. As far as I know, correcting any compiler problems and doing a "make distclean" to sanitize the Tcl source directory has always been enough to cure the situation.

Some installations (Slackware 2.1? 3.1? others? Red Hat before 4.0?) are reputed to have a missing symbolic link between /lib/libdl.so.1 and /lib/libdl.so. Any application which employs dynamic linking, and which therefore links with "... -ldl ..." under Linux, apparently will fail until one completes the installation by commanding

           ln -s /lib/libdl.so.1 /lib/libdl.so

That's the whole story that I know. However, those unfamiliar with Unix deserve a warning: the previous paragraph used "link" in three distinct senses. If these matters are new to you, you might feel more comfortable substituting

           cp /lib/libdl.so.1 /lib/libdl.so

for the "ln ..." command above. When I'm able to make time, I'll return to explain these matters more fully.

Why isn't there a /lib/libdl.a? That's an involved topic, one that I aim to document later in spring '97.

ldconfig is sometimes an issue for Linux. I'll document it once I understand it better.

Cameron Laird's guide to linking Unix utilities/claird@phaseit.net