Author: Sam Varshavchik

## Transport Modules

The directory `${COURIER_HOME}/lib/modules` contains shared libraries and programs which provide input and output modules to Courier. The actual module itself can actually be installed and invoked anywhere else, the SMTP module, in fact, is installed in bin. The shared library in lib, however, may include functions which rewrite addresses for messages submitted from the input module. All files for module named MODULE are stored in the directory `${COURIER_HOME}/lib/modules/MODULE`.

### The `modules.ctl` file

The file `${COURIER_HOME}/lib/modules/modules.ctl` contains a list of all the modules that Courier loads. `modules.ctl` is only used if Courier is compiled with shared library support. If Courier is compiled with static linkage only (by choice, or by necessity), all the module libraries are statically linked into Courier, and `modules.ctl` is not used. Each line in modules.ctl is in the following form: priority<SP>name<SP>filename, where `<SP>` designates the space character. The lines are sorted in increasing priority order! "priority" is taken from the module's config file, and filename is the filename of the shared library.

### Contents of `${COURIER_HOME}/lib/modules/name` directory

The following files will be found in a module subdirectory under lib. The name of the subdirectory is the name field from the `modules.ctl` file.

#### config

This is the module configuration file. Blank lines in the config file are ignored, also lines that start with the # character, which can be used for comments. The module configuration is specified using the `NAME=VALUE` notation, where `NAME` is the name of the configuration parameter, and `VALUE` is its value.

#### librewrite

This is the shared library that's loaded by `submit` and `courierd`. The shared library provides code to rewrite addresses to and from the canonical format for messages to or from this module. If Courier is compiled with static linkage, this file does not exist - the library is statically linked.

The actual name of the library may vary, and is specified in the `config` file.

If this shared library does not exist, an attempt is made to load `librewrite-old`. This allows an updated version of the shared library to be installed in a live, running, system by renaming the current one to `librewrite-old`, then renaming the new one to `librewrite`.

Although submit will pick up new rewriting rules immediately, `courierd` must be SIGHUPed in order to reload the new shared library.

#### Parameters in config

`NAME=name` - specifies the name of this module. Should be the same as the directory name.

`LIBRARY=filename` - specifies the name of the shared library to load. Not used if Courier was compiled with static libraries.

`VERSION=version` - version of the module interface. Not the actual version of the module, but version of the interface between the module, and Courier. The current version is version 0.

`PRIORITY=n` - priority of the output module. Courier calls all the modules' `rewritedel` functions in the increasing priority order until it finds one which accepts messages addressed to the recipient's address.

`PROG=pathname` - pathname to the output module program. Must be a full path, unless the module itself is in the `lib/MODULE` directory. If the `PROG` parameter is missing, this module is an input only module. If the attempt to execute `PROG` fails, Courier will attempt to execute `PROG-old`, which allows an updated output module to be inserted into a live system by renaming the current one `PROG-old`, then renaming the new output module as `PROG`.

`MAXDELS=n` - maximum concurrent deliveries for this module. No more than these many instances of `PROG` will be started at any given time.

`MAXHOST=n` - maximum instances of `PROG` that will be started with the same `HOST` parameter.

`MAXRCPT=n` - maximum number of addresses that will be given to any single instance of `PROG`.

Please note that although these parameters are reread by `courierd` when it restarts, the individual output module may impose its own restrictions on the valid limits for these parameters.

### Module library functions

The module library contains the following functions. In each case, `XXXX` is used to represent the name of the function in the library for module `XXXX`. For example, in the "local" module, the `rw_install` function is called `local_rw_install`.

`struct rw_list *XXXX_rw_install(const struct rw_install_info *info);`

The `rw_install()` function is called after the shared library is open. It receives a pointer to the following structure:

    struct rw_install_info {
            int rw_verlo;
            int rw_verhi;
            const char *courier_home;
            } ;

`rw_verlo/rw_verhi` - reserved for future expansion. Currently set to 0. `rw_install` function of modules compatible with this Courier interface must check that `rw_verlo` is set to 0, and ignore `rw_verhi`.

`courier_home` - Courier's home directory, the shared library can use this to find its files.

### The `rw_search` function

    struct rw_list *rw_search(const char *);

The `rw_search` function can be called by a library function in order to return the rw_list (see below) structure of a function in another library. `rw_search` may NOT be called form `XXXX_rw_install`, because the library containing this function may not've been installed yet. `rw_search` may be called from the `XXXX_rw_init` function.

`const char *XXXX_rw_init()`

After all modules are installed, each module's `rw_init()` function is called, which can complete any additional setup. `rw_init` should return a null pointer. Otherwise, `rw_init` should return the error message text. Courier will refuse to start if `rw_init` does not return a null pointer.

The library's `rw_install` function must return a pointer to the following structure. If `rw_install` returns a NULL pointer, Courier will refuse to start.

    struct rw_list {
            int rw_version;
            void (*rewrite)(struct rw_info *info, void (*func)(struct rw_info *));
            void (*rewrite_del)(struct rw_info *info, void (*func)(struct rw_info *),
                        void (*delfunc)(struct rw_info *info, const rfc822::tokens &host,
                                    const rfc822::tokens &addr));
            int  (*filter_msg)(const char *,
                            int,
                            const char *,
                            const char *,
                            const char *,

                            char *,
                            unsigned);
    } ;

`rw_version` - shared libraries compatible with this module interface must set rw_version to zero.

`rewrite` - this function is called to rewrite a single address. The first argument points to the following structure:

    struct rw_info {
        int mode;
        rfc822::tokens addr;
        void (*err_func)(int, const char *, struct rw_info *);
        const rfc822::tokens sender;
        const char *smodule;
        void *udata;
    } ;

`mode` contains the following values, ORed into a bitmask: `RW_ENVSENDER` - rewrite envelope sender, `RW_ENVRECIPIENT` - rewrite envelope recipient, `RW_HEADER` - rewrite header. When calling `rewrite()`, one of these three bits will be set. Additional bits that may be set: `RW_OUTPUT` - `rewrite`() should convert canonical format to the transport format. If this bit is not set, rewrite should convert from the transport format to the canonical format. In fact, the main `courierd` does not call rewrite with the `RW_OUTPUT` bit set, because that function is performed by the dedicated output module, which may handle rewriting on its own. Also, the `RW_SUBMIT` can be set together with `RW_ENVSENDER` or `RW_ENVRECIPIENT`, indicating that this call is as a result of a message being submitted for delivery (as opposed to address verification for `EXPN/VRFY` functionality).

It is possible that none of those bits are set, when invoked by another rewrite function.

`addr` is the address to rewrite, as rfc822 tokens.

`udata` contains an arbitrary pointer, for usage by rewrite's caller.

When mode has `RW_ENVRECIPIENT` set, `sender` points to the envelope sender format in the canonical format (previous result of `RW_ENVSENDER`), otherwise this field is unused. If `sender` is NULL, this should be interpreted as an empty envelope sender (or if `rewrite` is being called in test mode.

`smodule` is initialized when mode has the `RW_SUBMIT` bit set. It will point to the name of the module argument to submit - the module sending the message.

`err_func` is the function to call in case of an error.

`rewrite` is expected to call either `func`, (it's second argument), or `err_func`, before returning. If rewriting succeeds, `func` is called. If rewriting failed, `rewrite` must call the `err_func` is function. `errcode` will be the RFC822-style error number, `errmsg` is the error message, which may be multiline (includes newlines). The text in `errmsg` is NOT prefixed by the error number.

After calling `func`, or `err_func`, `rewrite` is expected to immediately terminate. `rewrite` may alter the 'ptr' link list in any form or fashion it wants, except that it may NOT malloc or free any node, or a part thereof. However, it can relink portions of the link list, or modify the link pointers to include tokens created by `rewrite` internally.

After `func` or `err_func` terminates, `rewrite` may deallocate everything it allocated, then terminate itself.

This interface allows rewrite to execute very quickly, without allocating or deallocating any memory. If new RFC822 tokens are required, they can be allocated on the stack, and linked into the original list.

The `rewrite_del` function is called to determine if the module can accept delivery of a message to this address. The `rewrite_del` of all installed libraries are called until one of them calls the `delfunc` function. If `rewrite_del` cannot accept delivery of a message, it should call `func`. The `rewrite_del` function should call the `delfunc` function to indicate that this module can accept mail addressed to the address specified by `info->ptr`. `rewrite_del` receives the pointer to the rw_info structure, then the `host` and the `address` information for the output module, as rfc822::tokens. If the mode field has the `RW_SUBMIT` bit set, rewrite_del can find the message's envelope sender address in canonical format) in `info->sender`.

Like `rewrite`, `rewrite_del` may make arbitrary changes to `info->ptr`, except that it may not deallocate memory used for the existing list. `rewrite_del` may modify the link list, and allocate memory for new rfc822 tokens. After calling either `func` or `delfunc`, `rewrite_del` should terminate immediately, and deallocate any allocated memory. `rewrite_del` must keep track of any allocated memory separate, and cannot assume that `info->ptr` hasn't changed.

When `RW_SUBMIT` bit is set, `rewrite_del` can be used to perform additional recipient-specific code, which may be too expensive to run every time courier goes through the queue. The `udata` field contains a pointer to caller-specific data. The `sender` field contains a pointer to the envelope sender, in canonical format. Like `rewrite`, `rewrite_del` may muck around with the `rfc822::tokens` in `addr`. `rewrite_del` functions are called in order according to the configured module priority. By setting a higher priority, it is possible to have `rewrite_del` rewrite the address so that it would be accepted by another module's `rewrite_del`, down the chain.

The last function, `rw_filter_msg`, is called to run an arbitrary spam filter which can be used to selectively reject messages. `rw_filter_msg` will be called after `rewrite_del` accepted the message for delivery. The arguments are as follows:

- The name of the module that is submitting this message for delivery.
- Either a read-only file descriptor, containing the copy of the message, or -1.
- The 'host' return value from `rewrite_del`, as a text string.
- The 'addr' return value from `rewrite_del`, as a text string.
- The message envelope return address.
- A pointer to a buffer where the error message can be saved.
- How large the buffer is, in bytes.

This function is called twice: once when submit receives the recipient address (file descriptor is -1), and the second time after the message is received, but before it is accepted. Therefore, this function should execute quickly. Also note that ESMTP does not have a well-defined way to selectively reject messages to individual recipients once the contents of the message are received. If a message is addressed to multiple recipients, and at at least one recipient's filter rejects the message, Courier replies with a rejection code, and the sending mail server will assume that the message to all recipients has failed.

`rw_filter_msg` should return 0 if the message is acceptable, a negative value to permanently reject the message to this recipient, and a positive value for a temporary rejection.

#### Running `PROG`

`PROG` is the output module that will be started by Courier when it comes up. PROG can be a shell command, it is executed via "`$SHELL -c`". It will be started under userid mail, group mail, in `${COURIER_HOME}/lib/modules/NAME`. Courier will communicate with `PROG` on standard input and output.

If `PROG` succesfully initializes, it should fork, and the parent should exit with status of 0. Courier waits until `PROG` exits. If it exits with a non-0 exit code, Courier will fail starting up. The child process will then continue to read and write from standard output.

`COURIER_HOME`, `MAXDELS`, `MAXHOST`, and `MAXRCPTS` will be placed in the environment, prior to executing `PROG`.

Tip: if the environment variables are not set, `PROG` can presume that it's being run in test mode, from the command line, and forego the fork.

If `PROG` terminates, Courier will consider it a fatal error (Courier detects the input channel being closed).

If `PROG` gets an end-of-file indication, it can assume that Courier is being shut down, so it should promptly cease all activities, without waiting for pending messages to be delivered.

To start delivery for a particular message, `PROG` will receive a newline-terminated command, specifying the message, and the recipients, and the delivery ID. Once `PROG` finishes delivering messages, `PROG` should write the results of the delivery into the message's control file, then print the delivery ID on its standard output, terminated by newline. If the module's config file specifies that the module can handle multiple deliveries at the same time, `PROG` may receive additional deliveries before it finishes delivering the first message.

The command that prog receives is a newline-terminated line that looks like this:

msgnum<tab>sender<tab>delid<tab>host<tab>num1<tab>addr1<tab>num2<tab>addr2...

<tab> represents the ASCII tab character. This is basically a list of tab-separated fields. The first field is the message id number (the inode number).

sender is the message envelope sender, after it's rewritten to the module format, by the module shared library.

delid is the "delivery ID", which is a small integer, representing this delivery. After `PROG` finishes delivering the message, it should print the message's delivery ID on standard output after saving the delivery status of each recipient in the control file.

The host field specifies the host where the message should be delivered to, as determined by the module's output rewrite rule. Following the host, there will be one or more num/address pairs. address is the recipient's address as determined by the output rewrite rule, and num is the recipient's number in the message's list of recipients (this is used to save the delivery status in the control file). Note that the address can be an empty string, so there will be two consecutive tabs there.