ToCDocOverviewCGDocRelNotesIndexPermutedIndex
Allegro CL version 6.2
Unrevised from 6.1

The Allegro CL imap and pop interface

This document contains the following sections:

1.0 The IMAP interface
   1.1 IMAP facility package
   1.2 Mailboxes
   1.3 Messages
   1.4 Flags
   1.5 Connecting to the server
   1.6 Mailbox manipulation
   1.7 Message manipulation
   1.8 Mailbox Accessors
   1.9 Fetching a Letter
   1.10 Searching for Messages
   1.11 Examples
      1.11.1 Connect to the imap server on the machine holding the email
      1.11.2 Select the inbox, that's where the incoming mail arrives
      1.11.3 Check how many messages are in the mailbox
2.0 The Pop interface
3.0 Conditions signaled by the IMAP and Pop interfaces
4.0 The SMTP interface (used for sending mail)

imap is a client-server protocol for processing electronic mail boxes. imap is the successor to the pop protocol. It is not an upward compatible successor. The main focus of this document is the imap protocol. Only one small section describes the functions in the pop interface.



1.0 The IMAP interface

The imap interface is based on the Imap4rev1 protocol described in rfc2060. Where this document is describing the actions of the imap commands it should be considered a secondary source of information about those commands and rfc2060 should be considered the primary source. (We have not given a link to rfc2060 or other rfc's mentioned in this document because such links tend to become stale. To see them, use a web search site such as Google (tm).)

The advantages of imap over pop are:

  1. imap can work with multiple mailboxes (pop works with a single mailbox)
  2. imap typically allows mail to be read from any machine while pop typically allows mail to be read on one machine only. With imap you're encouraged to leave mail in mailboxes on the server machine, thus it can be read from any machine on the network. With pop you're encouraged to download the mail to the client machine's disk, and it thus becomes inaccessible to all other client machines.
  3. imap parses the headers of messages thus allowing easier analysis of mail messages by the client program.
  4. imap supports searching messages for data and sorting by date.
  5. imap supports annotating messages with flags, thus making subsequent searching easier.

1.1 IMAP facility package

The functions in this interface are defined in the net.post-office package.


1.2 Mailboxes

Mailboxes are repositories for messages. Mailboxes are named by Lisp strings. The mailbox "inbox" always exists and it is the mailbox in which new messages are stored. New mailboxes can be created. They can have simple names, like "foo" or they can have hierarchical names (like "clients/california/widgetco"). After connecting to an imap server you can determine what string of characters you must use between simple names to create a hierarchical name (in this example "/" was the separator character).

Each mailbox has an associated unique number called its uidvalidity. This number won't change as long as imap is the only program used to manipulate the mailbox. In fact if you see that the number has changed then that means that some other program has done something to the mailbox that destroyed the information that imap had been keeping about the mailbox. In particular you can't now retrieve messages by their unique ids that you had used before.


1.3 Messages

Messages in a mailbox can be denoted in one of two ways: message sequence number or unique id.

The message sequence number is the normal way. The messages in a mailbox are numbered from 1 to N where N is the number of messages in the mailbox. There are never any gaps in the sequence numbers. If you tell imap to delete messages 3, 4, and 5 then it will return a value telling you the it has deleted messages 3, 3, and 3. This is because when you deleted message 3, message 4 became the new message 3 just before it was deleted and then message 5 became message 3 just before it was deleted.

A unique id of a message is a number associated with a message that is unique only within a mailbox. As long as the uidvalidity value of a mailbox doesn't change, the unique ids used in deleted messages will never be reused for new messages.


1.4 Flags

A flag is a symbol denoting that a message or mailbox has a certain property. We use keywords in Lisp to denote flags. There are two kinds of flags - System and User flags. System flags begin with the backslash character, which is an unfortunate design decision since that means that in Lisp we have to remember to use two backslashes (e.g. :\\deleted). A subset of the flags can be stored permanently in the mailbox with the messages. When a connection is made to an imap server it will return the list of flags and permanent flags (and these are stored in the mailbox server object returned for access by the program). If the list of permanent flags includes :\\* then the program can create its own flag names (not beginning with a backslash) and can store them permanently in messages.

Some of the important system flags are:


1.5 Connecting to the server

Use the function make-imap-connection to connect to the imap server of a host machine. close-connection closes the connection.


1.6 Mailbox manipulation

These functions work on mailboxes as a whole. The mailbox argument to the functions is is the object returned by make-imap-connection. If a return value isn't specified for a function then the return value isn't important - if something goes wrong an error will be signaled.

The functions are select-mailbox, create-mailbox, delete-mailbox, and rename-mailbox.

The function mailbox-list returns information about a mailbox.


1.7 Message manipulation

The following functions work with the messages in the currently selected mailbox. The mailbox argument is the object returned by make-imap-connection. The messages argument is either a number (denoting a single message), or is the list (:seq N M) denoting messages N through M, or is a list of numbers and :seq forms denoting the messages specified in the list.


1.8 Mailbox Accessors

The mailbox object contains information about the imap server it's connected to as well as the currently selected mailbox. This information can potentially be updated each time a request is made to the imap server. The following functions access values from the mailbox object.


1.9 Fetching a Letter

When using fetch-parts to access letters, you must specify the parts of the messages in which you're interested. There are a wide variety of specifiers, some redundant and overlapping, described in the imap specification in rfc2060. We'll describe the most common ones here. The specification is always a string but it may be specified more than one thing by the use of parentheses in the string, e.g. "(flags envelope)".

The most common specifiers are:

The result of a fetch-parts is a data structure containing all of the requested information. The fetch-field function is then used to extract the particular information for the particular message.


1.10 Searching for Messages

The imap server is able,using search-mailbox, to search for messages matching a search expression. A search-expression is a predicate (described below), or one of these forms:

A predicate is


1.11 Examples

We show an example of using this interface.


1.11.1 Connect to the imap server on the machine holding the email

The mailbox object, the value of mb, will be used in subsequent examples.

user(2): (setq mb (make-imap-connection "mailmachine.franz.com" 
                            :user "myacct" 
                            :password "mypasswd"))
#<mailbox::imap-mailbox @ #x2064ca4a>

1.11.2 Select the inbox, that's where the incoming mail arrives

The value of mb is the mailbox object returned by make-imap-connection in Section 1.11.1 Connect to the imap server on the machine holding the email.

user(3): (select-mailbox mb "inbox")
t

1.11.3 Check how many messages are in the mailbox

The value of mb is the mailbox object returned by make-imap-connection in Section 1.11.1 Connect to the imap server on the machine holding the email.


user(4): (mailbox-message-count mb)

7

There are seven messages at the moment. Fetch the whole 4th message. We could evaluate (fetch-letter mb 4) here (see fetch-letter) instead and then not have to call fetch-field later.

user(5): (setq body (fetch-parts mb 4 "body[]"))

((4
 ("BODY[]" "Return-Path: <jkfmail@tiger.franz.com>
  Received: from tiger.franz.com (jkf@tiger [192.132.95.103])
    by tiger.franz.com (8.8.7/8.8.7) with SMTP id LAA20261
    for <jkfmail@tiger.franz.com>; Mon, 13 Sep 1999 11:36:26 -0700
  Date: Mon, 13 Sep 1999 11:36:26 -0700
  From: jkf mail tester <jkfmail@tiger.franz.com>
  Message-Id: <199909131836.LAA20261@tiger.franz.com>

  message number 5
  ")))

The value was returned inside a data structure designed to hold information about one or more messages. In order to extract the particular information we want we use fetch-field:

user(6): (fetch-field 4 "body[]" body)

 "Return-Path: <jkfmail@tiger.franz.com>
 Received: from tiger.franz.com (jkf@tiger [192.132.95.103])
    by tiger.franz.com (8.8.7/8.8.7) with SMTP id LAA20261
    for <jkfmail@tiger.franz.com>; Mon, 13 Sep 1999 11:36:26 -0700
 Date: Mon, 13 Sep 1999 11:36:26 -0700
 From: jkf mail tester <jkfmail@tiger.franz.com>
 Message-Id: <199909131836.LAA20261@tiger.franz.com>

 message number 5
 "

We use the search function to find all the messages containing the word blitzfig. It turns out there is only one. We then extract the contents of that message.

user(7): (search-mailbox mb '(:text "blitzfig"))
(7)
user(8): (fetch-field 7 "body[]" 
           (fetch-letter mb 7 "body[]"))
"Return-Path: <jkf@verada.com>
Received: from main.verada.com (main.verada.com [208.164.216.3])
    by tiger.franz.com (8.8.7/8.8.7) with ESMTP id NAA20541
    for <jkfmail@tiger.franz.com>; Mon, 13 Sep 1999 13:37:24 -0700
Received: from main.verada.com (IDENT:jkf@localhost [127.0.0.1])
    by main.verada.com (8.9.3/8.9.3) with ESMTP id NAA06121
    for <jkfmail@tiger.franz.com>; Mon, 13 Sep 1999 13:36:54 -0700
Message-Id: <199909132036.NAA06121@main.verada.com>
To: jkfmail@tiger.franz.com
Subject: s test
Date: Mon, 13 Sep 1999 13:36:54 -0700
From: jkf <jkf@verada.com>
secret word: blitzfig

ok?

"

We've been using message sequence numbers up to now. The are the simplest to use but if you're concerned with keeping track of messages when deletions are being done then using unique id's is useful. Here we do the above search example using uids:

user(9): (search-mailbox mb '(:text "blitzfig") :uid t)
(68)
user(10): (fetch-field 68 "body[]" 
            (fetch-letter mb 68 "body[]" :uid t) :uid t)
"Return-Path: <jkf@verada.com>
Received: from main.verada.com (main.verada.com [208.164.216.3])
    by tiger.franz.com (8.8.7/8.8.7) with ESMTP id NAA20541
    for <jkfmail@tiger.franz.com>; Mon, 13 Sep 1999 13:37:24 -0700
Received: from main.verada.com (IDENT:jkf@localhost [127.0.0.1])
    by main.verada.com (8.9.3/8.9.3) with ESMTP id NAA06121
    for <jkfmail@tiger.franz.com>; Mon, 13 Sep 1999 13:36:54 -0700
Message-Id: <199909132036.NAA06121@main.verada.com>
To: jkfmail@tiger.franz.com
Subject: s test
Date: Mon, 13 Sep 1999 13:36:54 -0700
From: jkf <jkf@verada.com>

secret word: blitzfig
ok?
"

We'll delete that letter with the secret word. Note that after we have deleted that one, only six messages are left in the mailbox.

user(11): (delete-letter mb 68 :uid t)
(7)
user(12): (mailbox-message-count mb)
6

Now we assume that a bit of time has passed and we want to see if any new messages have been delivered into the mailbox. In order to find out we have to send a command to the imap server since it will only notify us of new messages when it responds to a command. Since we have nothing to ask the imap server to do we issue the noop command, which does nothing on the server.

user(13): (noop mb)
nil
user(14): (mailbox-message-count mb)
7

The server told us that there are now 7 messages in the inbox, one more than before. Next we create a new mailbox, copy the messages from the inbox to the new mailbox and then delete them from the inbox. Note how we use the :seq form to specify a sequence of messages.

user(15): (create-mailbox mb "tempbox")
t
user(18): (let ((count (mailbox-message-count mb)))
(copy-to-mailbox mb `(:seq 1 ,count) "tempbox")
(delete-letter mb `(:seq 1 ,count)))
(1 1 1 1 1 1 1)
user(19): (mailbox-message-count mb)
0

When we're done there are 0 messages in the currently selected mailbox, which is inbox. We now select the maibox we just created and see that the messages are there.

user(22): (select-mailbox mb "tempbox")
t
user(23): (mailbox-message-count mb)
7

Finally we shut down the connection. Note that imap servers will automatically shut down a connection that's been idle for too long (usually around 10 minutes). When that happens, the next time the client tries to use an imap function to access the mailbox an error will occur. There is nothing that can be done to revive the connection however it is important to call close-imap-connection on the lisp side in order to free up the resources still in use for the now dead connection.

user(24): (close-connection mb)
t


2.0 The Pop interface

The pop protocol is a very simple means for retrieving messages from a single mailbox. The functions in the interface are:



3.0 Conditions signaled by the IMAP and Pop interfaces

When an unexpected event occurs a condition is signaled. This applies to both the imap and pop interfaces. There are two classes of conditions signaled by this package:

Instances of both of these condition classes have these slots in addition to the standard condition slots:

Name Accessor Value
identifier po-condition-identifier keyword describing the kind of condition being signaled. See the table below for the possible values.
server-string po-condition-server-string If the condition was created because of a messages sent from the mailbox server then this is that message.

The meaning of the identifier value is as follows

Identifier Kind Meaning
:problem po-condition The server has responded with a warning message. The most likely warning is that the mailbox can only be opened in read-only mode because another process is using it.
:unknown-ok po-condition The server has sent an informative message that we don't understand. It's probably safe to ignore this.
:unknown-untagged po-condition The server has sent an informative message that we don't understand. It's probably safe to ignore this.
:error-response po-error The server cannot execute the requested command.
:syntax-error po-error The arguments to a function in this package are malformed.
:unexpected po-error The server has responded a way we don't understand and which prevents us from continuing
:server-shutdown-connection po-error The connection to the server has been broken. This usually occurs when the connection has been idle for too long and the server intentionally disconnects. Just before this condition is signaled we close down the socket connection to free up the socket resource on our side. When this condition is signaled the user program should not use the mailbox object again (even to call close-connection on it).
:timeout po-error The server did not respond quickly enough. The timeout value is set in the call to make-imap-connection.
:response-too-large po-error The value returned by a command is too large to fit in a lisp array. When this occurs you should close the connection and reopen it since the imap/pop interface code has gotten out of sync with the imap/pop server.


4.0 The SMTP interface (used for sending mail)

With the SMTP interface, a Lisp program can contact a mail server and send electronic mail. The contents of the message must be a simple text string. There is no provision for encoding binary data and sending it as a Mime attachment.

The smtp module is not loaded automatically when the imap is. To load SMTP functionality into a running image, evaluate:

(require :smtp)

The interface contains two functions:


Copyright (c) 1998-2002, Franz Inc. Oakland, CA., USA. All rights reserved.
Documentation for Allegro CL version 6.2. This page was not revised from the 6.1 page.
Created 2002.2.26.

ToCDocOverviewCGDocRelNotesIndexPermutedIndex
Allegro CL version 6.2
Unrevised from 6.1