reap-os-subprocess

Arguments: &key pid wait

This function replaced os-wait, which has the same functionality. This function has a better interface (using keyword rather than optional arguments).

If a process is started by the run-shell-command with the wait keyword argument to that function nil, then the process will remain in the system after it completes until either Lisp exits or Lisp executes reap-os-subprocess to inquire about the exit status. To prevent the system becoming clogged with processes, a program that spawns a number of processes with :wait nil must be sure to call reap-os-subprocess after each process finishes.

Exactly what reap-os-subprocess does depends on the status of spawned processes and the keyword arguments. The pid argument controls what processes might be considered on by reap-os-subprocess. If pid is -1 (the default), all processes are considered. If pid is 0, only processes in the same process group (as the executing Lisp image) are considered. If pid is a positive integer, only the process with that process id is considered. In the rest of this description, processes means `processes considered by reap-os-subprocess'. See the Unix system documentation of the waitpid() system call (you can usually see this by typing "man waitpid" at a Unix prompt).

If there are any processes started by run-shell-command with the argument :wait nil which have exited but for which reap-os-subprocess has not been run, one of them is selected by the operating system and its status and process id are returned in that order as multiple values.

If there are no such processes which have exited but there are processes which are still running, then the behavior of reap-os-subprocess depends on the wait keyword argument. If it is t (the default), sys:reap-os-subprocess will wait (disabling multiprocessing, if necessary) until one of the running processes exits. Then that process's status and id are returned. If wait is nil, reap-os-subprocess will immediately return two values: nil and nil if there are no processes to clean up; a status and a pid if the process with number pid is cleaned up, nil and the pid argument to reap-os-subprocess if processes are still running and none has yet exited.

If there are no running processes, reap-os-subprocess returns immediately with the values nil, nil.

This function simply calls the Unix system function waitpid with the pid and nohang flags. Its behavior is determined by the behavior of that function.

A note on the order of execution when reading from a prorgam to be reaped in a non-multiprocessing environment

Code written similar to the following skeleton of code may hang:

(multiple-value-bind (shell-stream error-stream process)
    (excl:run-shell-command cmd
       :input :stream :output :stream :error :stream)

  (when process
    (loop (when (sys:reap-os-subprocesses :pid process :wait nil)
	     (return))))

  ;; now read from shell-stream and then close the streams

  )

In the code sample, the process is reaped prior to reading the process output. While this often works, because many programs don't bother to wait for all of their writes to complete before exiting, it may cause hanging if the pipe to which the data is sent fills up and thus not all data can be written until some reading is done, or the child program waits until each input has been read before writing more data. Some operating systems will cause select() to not return ready status on the output descriptor if any data at all is in that pipe, regardless of whether a call to write() would have succeeded.

The correct outline for the code is:

(multiple-value-bind (shell-stream error-stream process)
    (excl:run-shell-command cmd
       :input :stream :output :stream :error :stream)

  ;; do all the reading from shell-stream

  (when process
    (loop (when (sys:reap-os-subprocesses :pid process :wait nil)
	     (return))))

  ;;  close the streams  
)

When you are using multiprocessing, you can use multiprocessing tools such as process-wait to ensure that no hanging occurs.

See os-interface.htm for information on running shell programs.

Copyright (c) 1998-2002, Franz Inc. Oakland, CA., USA. All rights reserved.
Documentation for Allegro CL version 6.2. This page was not revised from the 6.1 page.
Created 2002.2.26.