Installing Portable Dynamic (And Static) Libraries On OS X

Last Update: 07-02-2009

I recently decided to install portable Libraries on my Mac, a procedure that has never been proven easy for me for the past years. In order to port my programs from one platform to another, I must have downloaded portable libraries that exist in all platforms, such as PCRE (perl compatible regular expressions library for C and C++).

The problem is that when I try to install the libraries, I mostly encounter errors and problems during compilation while using the “make” tool of UNIX. This is due to the developers not being careful to handle all different system versions. Even when I did manage to install the necessary libraries, I couldn’t find a way to make them work with Xcode.

So what do you do when you need to install a portable Library and use it with Xcode?

This guide is written to guide you through this problem. Note that things will be constantly added to this guide, as I am perfecting my techniques on that.

STEP 1: Download and install Fink

Fink is a UNIX tool for the OS X terminal that brings open source software easily to OS X. The developers of Fink take open source projects, port then so that they are ready to be installed on OS X, and distrubute them using Fink. Fink also supports many open source libraries. Through it, you can download open source libraries, and install them effortlessly. Although Fink is a command line tool, in the package, an application named FinkCommander is provided, which is essentially the Fink command line tool with GUI. So, download Fink from its homepage, and install the package. After that, find the FinkCommander application in the same disk image, and install it in your Applications folder.

Do not run Fink yet! An explanation about environment variables comes first!

STEP 2: Understanding dynamic Libraries, and Header locations

Fink creates a folder named “sw” inside your startup disk. That folder contains the necessary files for the Fink runtime, and also serves as a storage for any Library or open source tool you may install using Fink. So, you have installed Fink, and it said that it will run a script to finish the installation. That script is meant to just do one thing: It opens your .profile (located inside your home directory as an invisible file) and adds a single line:

. /sw/bin/init.sh

That line just tells the computer to run a small script that is located inside /sw/bin/ at the moment you start up your computer. That script sets your environment variables to your computer, and tells Xcode (and other programs) to check for UNIX tools and libraries, in the folder “sw, in addition to the default places (usr/lib, usr/bin, etc). That way Xcode will become aware of any other libraries that may be added to the system. However… this doesn’t work!

Before I explain how this will be solved, I must first make a short reference to dynamic Libraries, and the way the ‘make’ UNIX tool works on OS X (and any UNIX variant).

Dynamic Libraries (.dylibs) contain all object pre-compiled files of a library. They are located in /usr/local/lib , usr/lib and some other places. OS X, using pre-defined environment variables, is predetermined to search those paths for Libraries and Files, it may find. However, dynamic libraries only contain implementation files. In order to use them in your program, you must also install and #include the headers that come with that library. Those header files are installed in /usr/include and usr/local/include on OS X (and I believe in UNIX in general).

Whatever library you install, installs a .dylib file, and a header file in their appropriate locations. When you open Xcode, you are supposed to have the Libraries installed, and just a #include should work for most Libraries. However, I found (the hard way) that this is not the case. So, I used Fink to install ported libraries to Mac. The next part shows how.

STEP 3: Use Fink to download and install a Library

Fink comes with a bunch of command line tools, but I think that users will find using FinkCommander more useful. So, open FinkCommander and the application will automatically fetch a list of available programs (wither already installed or available for install) for your machine.By selecting a library or a program and right clicking it, you can see the available options for that library/program. Here in this article I will talk about installing PCRE library on Fink. Find the pcre library, named “pcre” on Fink, and right click. You must choose on of the two available options, install binary or install source. Installing binary will install a pre-compiled binary for OS . Installing from source means that Fink will install a program or a library by letting your computer to handle compilation.

I chose to install “pcre” from source. The installation files will gointo sw/lib and sw/include . The installation is done, but now you must make the rest of the System work with the newly installed Libraries.

STEP 4: Use Xcode with that Library

There are 2 ways to make the installed libraries work with Xcode. One is setting the environment variables, so that Xcode also searches the folder /sw for libraries and headers. But unfortunately, this didn’t work for me. I chose the second method, which involved manually adding the dynamic library to a project in Xcode each time, and setting the project to look for the inclusion headers inside that.

Firstly, locate the .dylib files for that Library. The files for PCRE are located inside sw/lib/ and are called libpcre.a and libpcrecpp.a ~~(they are the same with .dylib files)~~. The .a files are similar to the dylib files, only that they repsresent static and not dynamic libraries. Although there are many differences, and there are appropriate uses for both types, in the present article you won’t need to know everything about these differences, as the procedure for the inclusion of the two library types is the same. So, drag those two to your project. You can copy the files to your project, or you can leave it alone, in case you need to update it afterwards.

Great, now you did that, and the file is included in your project. Next, you will need to tell your project to link against that library. In the targets folder, where you see the available target, click the enclosing triangle of the target you want to add the library in. Drag the .dylib or .a file inside the “Link Binary With Libraries” build phase. It is possible that you may skip this step, in case Xcode was smart enough to add the library in there on its own. In my case it was, but in case Xcode behaves like stupid, you could do this yourself, using the above method.

Last, but not least, you must tell Xcode to search for the Libraries and their headers to the appropriate paths. Double click your target to open up its configuration window. You are searching for a category named “Search Paths”. In this category rules are included that concern additional search paths for external libraries and headers, in case they are located somewhere else than the default locations. Do the following:

Check the “Always Search User Paths” rule.
In the header search paths, add one more path: “sw/include/” (without the quotes). Also check the “recursive” button to fore checking for headers into sub-folders.
In the Library Search Paths, add one more path: “sw/lib/” (without the quotes). Also check the “recursive” button to fore checking for libraries into sub-folders.
In the “User Header Search Paths”, add one more path: “/sw/” (without the quotes), and check the recursive button.

an alternative way:

Instead of dragging the .dylib file in XCode, you can also open the active target’s preferences, and click the “General” tab. You can add the library to be linked inside that tab, in the “Linked Libraries” section at the botton. I found that this is a better and more organized way of linking libraries.

STEP 5: You are all set!

If all went well, you should be able to use the Library with your project. In your project, create a new source file, and include the appropriate headers for that Library. It should compile correctly.

WHAT ABOUT STATIC LIBRARIES?

OK, so you have made your project, and ran it, but you should remember that the libraries you used are dynamic, and not static. That means that when a user on another computer uses your application, he/she must have the necessary libraries installed on the system! So that actually defeats the purpose, right? Well, not in many cases, but I have a way to remedy the problem there are some problems with .dylib and .a files. One problem is that these types of files cannot include resources such as additional images and such. The other, more important limitation is that they keep the headers separate from the actual Library, and that is extremely annoying because you have to manually add the headers to your project when you want to include a Library. There are also other limitations, that go outside of the scope of this article. The solution is to use FrameworkMaker, a script that converts dynamic Libraries to .framework files, that can be included into any Cocoa application and Carbon, and release the end user from the burden to install a library in his/her system. I didn’t make it. I took it from the subversion of AdiumX.

You can download the script here.

To use the tool, firstly install the rtool (supplied in the .zip). After that, run the following:

python <path/to/python/script> <path/to/file.dylib>  <export/path>

Then, you should have a folder that contains a compiled library into a framework. The final step is to put the headers in it, so that the .framework is complete. In the folder where the compiled dylib file is located make a folder path like this: “include/<name of dylib>/<all public header files>”. Example of such a path is this: UPnPDyld/build/Release/include/libUPnPDyld/<header files> .

After that, you should have a complete framework that you can include in your application.

CONCLUSION

I should note that there are some other ways to achieve the same thing. I just described the way I did it. This article will be updated with more and accurate info about dynamic Libraries on OS X, and a later article will have interesting UNIX environment variables as the main theme.

ADDITIONAL RESOURCES:
Wikipedia on Dynamic Libraries
Apple on porting and using dynamic libraries on OS X when moving from UNIX or Linux
A small article on Using Dynamic Libraries on OS X
Wikipedia on Unix Environment Variables