IMAP-Agent

Configuration: Site.XML and the external [user].XML files.

All configuration of IMAP-Agent is performed with as little as a single XML file or can be split up into as many files as suits the administrator. Splitting up the configuration can be done at any time and the additional files can contain information for one or more users and can be placed anywhere in the filesystem that IMAP-Agent has access to open them. Site.xml is the one required file.

In smaller installations it makes sense to have everything in the Site.xml file but in larger more complex installations that becomes cumbersome and it can be simpler to split the configuration up into smaller files.

To simplify maintenance of the configuration file(s), many attributes are optional or conditional with default values. Some attributes have a range of possible values, generally it is not acceptable to add attributes and leave them blank. Optional tags should either have a value or not be used.

SITE.XML Sample File

Here is a sample Site.xml document showing a reasonable configuration.

The example has been color coded to show the main sections.

Red text identifies the config tag.

Yellow hightlight: Everything in this section has site-wide scope defining mailservers and site-wide rules.

Blue hightlight: The user information section defining user specific rules.

The purpose of the ADDITIONAL.XML file:

You should also notice the ADDITIONAL.XML file below the Site.XML file. In the Site.XML sample file there are two users whose information is stored in the additional.xml file. If you look closely, you will see the names of those two highlighted in green and purple.

One or more users information can be located in secondary xml files such as this. Look for the gray highlighted text in the Site.xml and you will see the name of the secondary file. If the gray highlighted text was removed, IMAP-Agent would assume the name of the secondary xml file was identical to the user’s name attribute. So for example, if the gray highlighted text was gone for the user named lister then IMAP-Agent would look for a file named lister.xml.

Note: Secondary xml files cannot refer to tertiary xml files. The Site.xml file can optionally refer to secondary.xml files but those files must contain the users information.

<?xml version="1.0" encoding="UTF-8" ?>

<main>

The pwd attribute can be an actual password or can identify

any console type program to run that will look up the

password. See the password lookup plugin documentation for

more details.

</comment>

<all_users >

<comment>These site wide rules will be appended to any rules in the user's section at runtime.</comment>

</folder>

</rules>

<comment>These aging values will be used site wide unless overridden in the users section.</comment>

<aging>

</aging>

</all_users>

<user mailserverid="holly" name="lister" file="y" >additional.xml</user>

</folder>

</rules>

<aging>

</aging>

</user>

<user mailserverid="holly" name="cat" logon="cat" file="y">additional.xml</user>

</folder>

</rules>

</user>

</folder>

</rules>

<aging>

</aging>

</user>

</users>

</config>

</main>

[ADDITIONAL].XML Sample File

<?xml version="1.0" encoding="UTF-8" ?>

<main>

</folder>

</rules>

<aging>

</aging>

</user>

<rule testtype="custom" to="Keep" searchscope="message">G:\\cvs\\plugins\\CustomRule\\custom_rule_plugin\\bin\\release\\plugin_custom_rule.exe</rule>

</folder>

</folder>

</rules>

<aging>

</aging>

</user>

</users>

</config>

</main>

Element and Attribute Tags

Some of the element tags are can appear in the [additional].xml files. They are shown in the sample above.

<main>

Required. No attributes.

<config>

Required tag.

Attribute	Description	value	Scope
Run	Run IMAP-Agent rules.	“y”, “n”	Optional. Default = “y”
Loglevel	Set loglevel.	1=error 2=warn 3=info 4=verbose	Optional. Default = “1”
Maxlogsize	Set log file max size.	Kbytes.	Optional but advisable. Default = “1000” which is 1Mbyte. When the log file exceeds the maxlogsize then 10% of the file size is cut with oldest entries being removed.
Sleepseconds	IMAP-Agent can sleep between folders. If you are worried IMAP-Agent is hogging the mail server.	Integer.	Optional. Default = “0”
Version	For versioning of the xml file.	Numeric.	Optional. Default = “1”.

<all_users>

Required tag.

<all_users smtb="20" smwa="true">

Attribute	Description	value	Scope
Smtb	Skip message if it’s too big.	Numeric. Kbytes.	Optional. Applies to all users. Default = “0”
Smwa	Skip messages with attachments.	1=True, 0=false	Optional. Applies to all users. Default = “0”

<mailserver>

Required tag. There must be at least one mailserver.

attribute	Description	value	Additional
id	Each mailserver must have a unique id	Any string value.	Required.
host	IMAP host identifier.	Domain name or IP address	Required
port	IMAP port number	(usually 143).	Required
Pwd	Password tag. [plugin] indicates a script to be run for retrieving passwords. See Password plugin section for more details.	String	Optional. Provides a default password for all child <user> elements using this mailserverid.
Extra	The extra tag is passed as one of the parameters in the Password Lookup plugin. Usage is user defined.	String	Optional. Default is empty string. Provides a default extra string for all child <user> elements using this mailserverid.

<rules>

Scope: Optional as a child tag under <all_users>, but required as a child tag under <users>.

Any rules under the <all_users> tag are taken to be site wide rules. Site wide rules are run for each user after that user’s specific rules have been run.

<folder>

Attribute	Description	value	Scope
name	Name of the mailbox folder to run the rules on.	String. Must be a valid mailbox.	Required. Applies to all child rule tags.
Age	Age of the message in days.	Numeric.	Optional. This attribute applies when the folder tag is a child of the <aging> tag. It only applies to aging.
Maxcount	Max count of messages in the mailbox.	Numeric	Optional. This attribute applies when the folder tag is a child of the <aging> tag. It only applies to aging.
Totalsize	Total size quota for the folder.	Numeric. This means Kbytes.	Optional. This attribute applies when the folder tag is a child of the <aging> tag. It only applies to aging.
Smtb	Skip message if it’s too big.	Numeric. Kbytes.	Optional. Applies to all rules in this folder. Default = “0” which means ignore. Overrides site wide (see “all_users” tag).
Smwa	Skip messages with attachments.	1=True, 0=false	Optional. Applies to all rules in this folder. Default = “0” Overrides site wide (see “all_users” tag).
Run	Run IMAP-Agent rules.	“y”, “n”	Optional. Default = “y”

<rule>

A rule to be performed against new mail.

<rule action=”move” to=”Junk” searchscope=”header” testtype=”regex” smtb=20” smwa=”true”>tag value</rule>

Note: Rule tags located under the <all_users> tag are appended to the end of each users set of rules such that user rules get run and then site wide rules get run.

Attribute	Description	value	Scope
Action	Action to perform if the rule test is successful.	“move”, “copy”, “delete”	Optional. Default is “move”.
To	The destination mailbox folder.	String. If folder doesn’t exist it is created.	Required when action is “move” or “copy”.
Searchscope	What part of message to apply rule test against.	“header”, “subject”, “from”, “body”, “message” --------------------- “message” means headers and body but does not include attachments.	Optional. Default is “message”
Testtype	Test type to perform.	“regex” is currently the only test type.	Optional. Default is “regex”. To create a custom rule you set testtype to anything else buy “regex”. See Value attribute below.
Smwa	Skip messages with attachments.	1=True, 0=false	Optional. Applies to this rule. Default = “0” Overrides site wide and folder settings.
Smtb	Skip message if it’s too big.	Numeric. Kbytes.	Optional. Applies to this rule. Default = “0” which means ignore. Overrides site wide and folder settings.
Value	Regex pattern. Rules are fired when regex evaluates to success.	Example: <![CDATA[^X-Header:.(([2-9][5-9]\d)\|[3-9]\d+).*$]]>	Required. If Testtype != “regex” then this value contains the path to the custom rule plugin. Example: C:\\foo\\plugin_custom_rule.exe For more information see the section on Custom Rules.

<users>

Parent tag where individual user parameters are described.

Attribute	Description	value	Scope
Pwd	Password tag. [plugin] indicates a script to be run for retrieving passwords. See Password plugin section for more details.	String	Optional. Provides a default password for all child <user> elements.
Extra	The extra tag is passed as one of the parameters in the Password Lookup plugin. Usage is user defined.	String	Optional. Default is empty string. Provides a default extra string for all child <user> elements.

<user>

Attribute	Description	value	Scope
Name	Unique identity for this user.	String	Required. If file=”y” then this is used to locate the same user in the additional xml file. See Value field below. See See special “Name and Logon Attribute Notes” below.
Mailserverid	Matches the id attribute for a mailserver tag.	String	Required.
Logon	Users mail account code.	String	Optional. Uses name attribute if omitted. See special “Name and Logon Attribute Notes” below.
Pwd	Password tag. [plugin] indicates a script to be run for retrieving passwords. See Password plugin section for more details.	String.	Optional. This may be specified in the <users> tag. Overrides any defaults – see <users> and <mailserver> tags.
Extra	The extra tag is passed as one of the parameters in the Password Lookup plugin. Usage is user defined.	String	Optional. Default is empty string. Overrides any defaults – see <users> and <mailserver> tags.
Run	Run rules for this user when IMAP-Agent runs.	“y”, “n”	Optional. Default = “y”
File	This user’s parameters are located in a different xml file.	“y”, “n”	Optional. Default = “n”
Value	The text value for the <user> tag.	String	If file=”y” then this value is the filename where this users data is located.

Name and Logon Attribute Notes:

The “name” and “logon” attributes have some special characteristics.

1. The name attribute is used as an id in the <users…> tag to uniquely identifying each user. For that reason it needs to be unique.

2. The name attribute is also used as a filename to store that users history. Again, for that reason it needs to be unique.

3. The name attribute can double as the users logon account if the logon attribute is omitted. This was done to simplify configuration setup.

4. You might need to use both the name and logon attribute if IMAP-Agent will be managing more than a single mailserver. In that case, it is possible to have an identical logon account on both mailservers and since the name attribute MUST be unique, at least one of the name attributes would have to be different than it’s logon attribute.

Special Note: If you ever want to change the contents of a “name” attribute, be sure to rename the history (“name”.dat) file to the new name. That will preserve the users history.

File and Value attribute Notes:

If file = “y” and Value = “” then the name attribute will be used as the filename.

<aging>

Aging tags are parent tags containing folder tags. <aging> and <rules> tags are at the same level.

They are separated because aging is only performed periodically such as once per day while the other rules are performed many times per day, perhaps many times per hour and this makes for easier maintenance.

Any combination of the three quotas can be used in the same folder.

The aging process evaluates in this order:

1. totalsize

2. maxcount

3. age

TotalSize and maxcount are the most efficient. Totalsize must fetch the message before deleting it. Maxcount doesn’t need any information from the message and can delete until the maxcount is satisfied.

Age requires fetching each message in the folder to get the message date.

All messages are fetched and/or deleted using their UID. Messages are assigned UID’s when they are added to a folder. UID order of the messages may not coincide with date order. For this reason, it might be better to use the age quota even though it uses more resources.

Note: To clarify something about the age quota and the MIME Date: header in the mail messages, I will describe this specific scenario.

Scenario: Aging using the age quota is set for 15 days on the “keep” folder. A message received now at the INBOX folder is moved to the folder named “keep” by IMAP-Agent based on a rule. So, now the message (less than one day old) that was just received is in the “keep” folder. Then the user manually moves a 20 day old message from some folder into the “keep” folder. When Aging is run that night, the 20 day old message will be deleted. The reason is that the MIME “Date:” header doesn’t get changed by moving the messages around in folders. In fact, there is no reliable way to determine how long a message has been in any given folder because some (perhaps most) IMAP servers preserve INTERNALDATE when copying messages. So, the age quota concerns the age of the messages since they were sent by the sender (which should be minutes before they are received by you if you have an always connected mail server) , not how long they have been in the folder. [I will point out that using maxcount or totalsize, messages are deleted in the order they were added to the folder.]

Note: Subject: IMAP - message date question:

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=a8a0ea.0404021022.369e44cc%40posting.google.com&rnum=3&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26q%3Dspamtrap%26meta%3Dgroup%253Dcomp.mail.imap

Note: In all aging quota cases, only the message header is fetched. The body and any attachments are never fetched.

Note: Deleted messages are not moved to the mailbox Trash folder, but are immediately expunged from the mailbox. There is no ability to retrieve deleted messages.

Attribute	Description	value	Scope
Totalsize	This means the total size quota for the folder in Kbytes.	String	Optional. A value of zero “0” means ignore.
Maxcount	The maximum number of emails quota.	String	Optional. A value of zero “0” means ignore.
Age	Meaning number of days old. All email older than (Age) days is deleted. The MIME “Date:” header is where the email date is taken from.	String	Optional. A value of zero “0” means ignore.