“It is time that package management applications free shared libs from artificial shackles and create a more dynamic and versatile distribution.”

The first time I had come across Unix shared libraries was with Unix SVR3.2. The ability to work with multiple versions of libraries concurrently had seemed impressive. As per my recollection, the shared libraries we were using on the mainframes did not offer such capabilities. I found that I needed to use that capability after about a year with Linux.

We were using Slackware distribution and needed to install an additional package. We could get a binary version; however, it needed a lower version of a shared library. It was fairly simple to install the alternate version of the library as well and get on with the work.

Normally, if a shared library is well designed and well maintained, an application developed or linked using a lower version should not have any problem working with the higher version library. The developer has to ensure that the old interfaces continue to be supported on the higher versions. In case the library changes substantially and the old interfaces no longer work, it may even be better to change its name, e.g. libxml2, libX11, libv4l1 or libv4l2.

Over time, repositories maintained by the distributions have made it far easier to deal with library dependencies transparently. We can still manually follow the same approach as above but it is a barrier. GUI desktop packages, in particular, have much more complex dependencies than the server packages with which we were dealing, seemingly, a long time ago.

The distributions have simplified our life and theirs by imposing constraints which limit the flexibility of the underlying shared libraries. Distributions insist that only one version of a library will be managed by the package management software. Sometimes that is just not possible. In that case, additional packages are created, typically prefixed by compat.

In the last few years, I have become convinced that this approach is unduly restrictive. As an example, upgrading to fedora 13 created needless minor failures and friction because of library versions. In case anyone is interested in details – http://sethanil.blogspot.com/2010/06/migrating-to-fedora-13.html. If a version of a library from f12 repository installed is higher than the version available in f13 repository, why should it break an upgrade?

Another example occurred with Arch Linux. Since new xorg and intel display drivers are unstable on some older hardware, I have installed Arch Linux with old intel driver and am holding back the xorg packages to the versions which work with that driver. Recent updates succeeded; however, X stopped running. Xorg needed version libcrypto.so.0.9.8; however, the rest of the system needed the new version libcrypto.so.1.0.0. In this case as expected, the shared library did not change the interfaces for the existing packages. The solution was simple - just link the desired version of the library to the current version.

However, suppose xorg is not updated; so the existing library is still needed. A smarter package management would leave the existing libcrypto in place and install the higher version as well.

It is a standard practice in package management to separate the shared libraries from the executable applications. This is an implicit assumption that libraries are different. What is missing is that the library packages need to be treated differently but aren't.

For example, as we discussed last month, Debian/Ubuntu apt system keeps track of how a package was installed – manually or to satisfy a dependency. In the latter case, when no one needs this package, it can be removed. Libraries are always used by some package. They will normally be installed to satisfy dependencies. Unfortunately, complications can arise as some libraries may be optional and may be explicitly(manually) installed. An example that readily comes to mind is the libdvdcss library.

However, just as multiple versions of kernel can be installed, a package manager should install multiple versions of library packages. Furthermore, if a library package is installed to satisfy a dependency, then as in the case of apt, it should remove this package once no package needs it.

Since there could be unusual problems, it is perfectly reasonable for a distribution to offer a warning like the one firefox gives for 'about:config' - “This may void your warranty!”. The distribution can disown responsibility for the chaos that may result if multiple versions of libraries are installed. But it shouldn't deny us that pleasure!

Once library packages are differentiated, package management can make use of that information. There are already a huge number of packages to choose from and the library packages just add to the noise. A simple estimate - on my system 657 packages out of 2575 installed have 'lib' in them. I neither need to nor want to see the library packages. By default, hiding library packages from view would definitely simplify package management. Shared libraries should normally work silently behind the scenes.

Moral: avoiding a constraint which allows for more complex installation scenarios can result in increased simplicity!