| Allegro CL version 6.2 Unrevised from 6.1 | |||||||
This document contains the following sections:
1.0 How it WorksAllegro CL supports building shared libraries that can be used by C/C++ programs in the same manner as they use any other function libraries. This document describes how Allegro Common Lisp developers on UNIX machines can create such Lisp based shared libraries. See dll.htm for information on on creating similar files on Windows.
This functionality is available on all Unix platforms except for HP/UX 11.0 (see OS Specific Notes for more information).
The Lisp developer creates a "wrapper" shared library (C is used in the example below) to provide the public functional interface developers will use. The shared library also contains private functions used to transfer control back and forth between C wrappers and Lisp code. A companion Lisp image contains the custom code needed to provide the functionality behind the public interface as well as utility functions needed by the C/Lisp interfacing.
Unix OS thread functionality is used to start the Lisp image on a separate OS thread. Because Unix Allegro CL 6.2 versions are not integrated with OS thread functionality, all Lisp processing must occur in the Lisp thread. (With Windows Allegro CL 6.2, once the Lisp starts, Lisp processing may occur on any thread, including the original main thread, or other threads created outside of Lisp.) Two semaphores are used to synchronize processing between the Lisp thread and the original main thread.
Since Lisp easily handles large valued integer calculations, a Lisp based factorial() function is a good, simple example. To avoid C integer overflow, the answer is placed in a result string, rather than returned as a number.
The example uses a number of files, all in the examples/unix-shared-library/ subdirectory of the Allegro directory on UNIX machines. (The files are not included in the Windows distribution of Allegro CL.) The files are:
The load_lisp() private function runs in a new thread created by the original main thread. It loads the Lisp image and starts it running. It calls the Allegro CL library function lisp_init(), which is described in main.htm.
Note that the shlibs.h file is found in the Allegro CL distribution misc/ directory. Use the compiler -I option to add that directory to the include file search list when compiling fact.c.
The Allegro CL library function tcm() is called to prevent the Lisp from writing to stdout and stderr during initialization and subsequent processing. During your development phase, you may wish to comment this call out, in order to see format or print debugging output.
The wait_on_semaphore() and release_semaphore() private functions are used to synchronize processing between the Lisp thread and the original main thread. This example uses pipes as semaphores. Since file descriptors are a limited resource, and some Unix OS's provide specialized semaphore functionality, you may wish to change this code for your own work. See Section 5.0 OS Specific Notes for more information.
The initialize_factorial() public function creates the Lisp thread. Its argument is a pointer to the main program environment that is available as the third main() argument. Besides initializing the synchronization semaphores, it uses the Allegro CL library function find_file_using_pathstring() to identify the directory containing the fact.dxl Lisp image. It's a good design decision to use the same environmental variable that the OS uses to find shared libraries for the third argument. For example, on Solaris, use LD_LIBRARY_PATH. The wait_on_semaphore() function is called to wait for the Lisp thread to respond that all initialization activities are completed.
The release_end_semaphore() function is provided for the Lisp image to call when initialization steps are complete, releasing the semaphore initialize_factorial() is waiting on, allowing it to return to its caller.
The set_factorial_callback() function is provided for the Lisp image to call during initialization steps to set the callback address for the Lisp 'factorial callback.
The factorial() public function wraps the C/Lisp operations that calculate the factorial result. It sets some global variables that the Lisp thread will access, releases the semaphore that the wait_for_work() function is waiting on in the Lisp thread, and then waits on another semaphore for the Lisp thread to respond that the factorial calculation is complete. It then returns, using a global value set by the Lisp thread.
The wait_for_work() function is called by the Lisp image after initialization steps are complete and it has released the semaphore used to synchronize the initialization phase. It loops forever, responding when the original main thread releases a semaphore that indicates that factorial() has been called. It invokes the factorial callback and then releases a semaphore that tells the main thread that Lisp processing is complete. It then goes back to waiting for more work.
The copy_factorial_result() function is called by Lisp during factorial processing to pass the result string into C data.
The terminate_factorial() public function wakes up wait_for_work() in such a way that it breaks out of its endless loop, causing the Lisp thread to unwind. On Linux, where separate threads have separate process id's, it is imperative that the Lisp thread unwind; otherwise a hang will occur.
The factorial function calculates the factorial result.
The wrapper C shared library is loaded and foreign function definitions are made as necessary.
The 'factorial-callback callback calls 'factorial, converts the result to a string, and passes the result string to C data using the copy_factorial_result() function.
The 'initialize-factorial function passes the callback address back to C, releases the initialization semaphore, and goes into a "wait for work" mode, all using C functions defined in fact.c. When the wait_for_work() call returns, indicating that terminate_factorial() has been called, 'exit is called, to unwind the Lisp thread.
The *restart-init-function* variable is set to
'initialize-factorial, so that the initialization
steps commence when the Lisp image is started.
The ftest.c file is fairly straightforward. It calls the C wrapper shared library public functions initialize_factorial() and factorial(). On the way out, terminate_factorial() is called. Looking at this code, there is no way to tell that the factorial shared library is Lisp based.
See the Section 5.0 OS Specific Notes, below, for OS specific C shared library wrapper generation commands.
generate-application is a useful tool for generating the fact.dxl image for delivery and gathering together all needed files for delivery preparation. You must have an Enterprise license to use generate-application.
Here is the generate-application call we use when testing this example - note that we generate the application after the factorial shared library and ftest module have been built and fact.cl has been compiled to fact.fasl:
(generate-application "fact" "fact/" (list "examples/unix-shared-library/fact.fasl") :application-type :exe :application-files (list "examples/unix-shared-library/ftest") :restart-init-function nil :restart-app-function nil :include-compiler nil :runtime :standard)
Note that the resulting fact/ directory will contain a file fact that is not needed when delivering a shared library application. (It is an executable file, and the executable you actually run is whichever one will be linking in the Lisp shared library.)
Remember when writing your shared library documentation to tell users to add the delivery directory to the OS dynamic library load environmental variable.
This example uses only one lisp callback and passes the address in an unsafe manner, to preserve simplicity. If you wish to provide many callbacks, you must pass all the callback addresses from Lisp to C, and you must be able to determine which callback is appropriate when wait_for_work() wakes up. It is recommended that the addresses be passed by index, using lisp_call_address() to get the actual addresses for each callback before invoking it.
Rather than set up many global argument variables, it may make sense to define a union in C for wait_for_work() to work with. There can be a union member element available for saving the callback address. You can have elements that can be used to store callback arguments.
Put the union in a structure that also contains an 'id' element. Then provide public C wrappers for each callback. In each wrapper, set the 'id' structure element to a unique number and then load the callback arguments into the appropriate union member. Then, when wait_for_work() wakes up, it can use the 'id' element to determine which callback to invoke and retrieve the callback arguments out of the appropriate union member.
If you don't call tcm() in your C shared library wrapper, then you will be able to see print and format that you may put in your Lisp code for debugging purposes. As long as the original main thread is not seeking to read from standard input, you can respond to breaks as you would normally do when starting lisp in a shell.
The following OS specific notes sections assume you are building your shared library in the Allegro CL product installation examples/unix-shared-library/ directory.
On most Unix OS's, the dynamic load environmental variable must include the directories containing the acli61 and fact shared libraries to successfully run the ftest example executable.
As noted, the example does not work with Mac OS X, FreeBSD, or with 32-bit HP-UX 11.0.
cc -mt -c -I.../../misc fact.c ld -mt -G -olibfact.so fact.o -lpthread -L../.. -lacli61
cc -o ftest ftest.c -L. -lfact -L../.. -lacli61
Notes:
cc -xarch=v9 -DAcl64Bit -mt -c -I.../../misc fact.c
ld -mt -G -olibfact.so fact.o -L/usr/lib/sparcv9 -lpthread -L../.. -lacli61
cc -xarch=v9 -DAcl64Bit -o ftest ftest.c \
-L/usr/lib/sparcv9 -L. -lfact -L../.. -lacli61
Notes:
cc -shared -o libfact.so -taso -xtaso -xtaso_short -I../../misc fact.c -lpthread -L../.. -lacli61
cc -taso -o ftest ftest.c -L. -lfact -L../.. -lacli61
Notes
cc -shared -o libfact.so -resumption_safe -ieee_with_no_inexact -DAcl64Bit -I../../misc fact.c -lpthread -L../.. -lacli61
cc -resumption_safe -ieee_with_no_inexact -DAcl64Bit -o ftest ftest.c -L. -lfact -L../.. -lacli61
Notes
cc -shared -o libfact.so -n32 -I../../misc fact.c -lpthread -L../.. -lacli61
cc -o ftest -n32 ftest.c -L. -lfact -L../.. -lacli61
HP/UX 11.0 note: It is not possible to generate a 32-bit Lisp based shared library on HP/UX 11.0.
aCC -DHP1020 -D_REENTRANT -c +z -I../../misc fact.c aCC -b -Wl,+s -Wl,-E -o libfact.sl fact.o -ldce -L../.. -lacli61
aCC -o ftest ftest.c -Wl,+s -L. -lfact -L../.. -lacli61
> Notes
aCC -D_REENTRANT -c -g0 -DAcl64Bit +Z +DA2.0W +DS2.0 -I../../misc fact.c
aCC -b -o libfact.so fact.o -g0 -DAcl64Bit +Z +DA2.0W +DS2.0 \
-Wl,-E -Wl,-B,immediate -l:libpthread.a -L../.. \
-l:libacli61.so -Wl,+b.:.. -Wl,+s
aCC -o ftest ftest.c -g0 -DAcl64Bit +Z +DA2.0W +DS2.0 -Wl,-B,immediate \
-L. -l:libfact.so -L../.. -l:libacli61.so -Wl,+b.:.. -Wl,+s
> Notes: none.
cc -dynamic -c -o fact.o -I../../misc fact.c ld -bundle /usr/lib/bundle1.o -o libfact.dylib -undefined suppress fact.o -lc
cc -o ftest ftest.c
Notes: The example does not currently work on Mac OS X because of the way the test is built. Mac OS dynamically loadable shared-libraries (bundles) can't be relinked with other object files. Thus, the example above doesn't yet work.
cc -g -pthread -shared -o libfact.so -I../../misc fact.c \
-L../.. -lacl61.so -lc_r
cc -g -pthread -o ftest ftest.c -L. -lfact -R.. `pwd`/libacli61.so -lc_r
Notes: The FreeBSD version does not yet work. There are still issue involving the pathname of the shared-library, the compilation of Allegro CL to include threads libraries (i.e. -lc_r) and poor threads support in gdb.
cc -DAIX -c -I../../misc fact.c ld -brtl -bnodelcsect -H4096 -T512 -bM:SRE -bE:fact.exp -o libfact.so fact.o -lpthreads -L../.. -lacli61 -lc
cc -brtl -bnodelcsect -H4096 -T512 -DAIX -o ftest ftest.c -L. -lfact -L../.. -lacli61
Notes
#!
initialize_factorial
release_end_semaphore
wait_on_semaphore
set_factorial_callback
factorial
wait_for_work
copy_factorial_result
lisp_init_func
terminate_factorial
cc -shared -o libfact.so -I../../misc fact.c -L../.. -lacli61
cc -o ftest ftest.c -L. -lfact -L../.. -lacli61
Notes
Copyright (c) 1998-2002, Franz Inc. Oakland, CA., USA. All rights reserved.
Documentation for Allegro CL version 6.2. This page was not revised from the 6.1 page.
Created 2002.2.26.
| Allegro CL version 6.2 Unrevised from 6.1 | |||||||