Testing Stalwart
The following subsections cover the different tests available on Stalwart. Before running these tests, make sure you have Rust installed by following the instructions in the compile section.
Protocol tests
JMAP
The JMAP protocol tests ensure protocol compliance. To run these tests execute:
$ cargo test --manifest-path=crates/jmap-proto/Cargo.toml -- --nocapture
IMAP
The IMAP protocol tests ensure protocol compliance. To run these tests execute:
$ cargo test --manifest-path=crates/imap-proto/Cargo.toml -- --nocapture
WebDAV
The WebDAV protocol tests ensure protocol compliance. To run these tests execute:
$ cargo test --manifest-path=crates/dav-proto/Cargo.toml -- --nocapture
Storage layer tests
The store tests ensure that the mail store is working as expected. Before running these tests, you'll have to install and run MinIO in order to be able to run the S3 storage tests:
$ wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio_20230629051228.0.0_amd64.deb -O minio.deb
$ sudo dpkg -i minio.deb
$ mkdir ~/minio
$ minio server ~/minio --console-address :9090 &
$ wget https://dl.min.io/client/mc/release/linux-amd64/mc
$ chmod a+rx mc
$ ./mc alias set myminio http://localhost:9000 minioadmin minioadmin
$ ./mc mb tmp
Once MinIO is up and running, start the store tests by executing:
$ cargo test --manifest-path=tests/Cargo.toml store -- --nocapture
If you would like to test the FoundationDB backend, you'll first have to install the FoundationDB client libraries. Once the FoundationDB client libraries are installed, start the store tests by executing:
$ cargo test --manifest-path=tests/Cargo.toml store --no-default-features --features foundationdb -- --nocapture
Directory tests
The directory tests ensure that the directory service is working as expected. Before running these tests, you'll have to install and run glauth in order to be able to run the LDAP tests:
$ wget https://github.com/glauth/glauth/releases/download/v2.2.0/glauth-linux-arm64
$ chmod a+rx glauth-linux-arm64
$ ./glauth-linux-arm64 -c tests/resources/ldap.cfg &
Once glauth is up and running, run the directory tests by executing:
$ cargo test --manifest-path=tests/Cargo.toml directory -- --nocapture
Integration tests
SMTP
The SMTP tests ensure that the SMTP server is working as expected. To run these tests execute:
$ cargo test --manifest-path=tests/Cargo.toml smtp -- --nocapture
JMAP
The JMAP tests ensure that the JMAP server is working as expected. To run these tests execute:
$ cargo test --manifest-path=tests/Cargo.toml jmap -- --nocapture
IMAP
The IMAP tests ensure that the IMAP server is working as expected. To run these tests execute:
$ cargo test --manifest-path=tests/Cargo.toml imap -- --nocapture
WebDAV
The IMAP tests ensure that the IMAP server is working as expected. To run these tests execute:
$ cargo test --manifest-path=tests/Cargo.toml webdav -- --nocapture
Third party tests
JMAP test suite
Compliance with the JMAP protocol may also be tested using Fastmail's JMAP-TestSuite:
- Clone the JMAP TestSuite repository:
git clone https://github.com/stalwartlabs/jmap-test-suite.git
cd jmap-test-suite/ - Install the recommended Perl dependencies:
cpanm --installdeps .
- Run one of the tests, for example:
JMAP_SERVER_ADAPTER_FILE=eg/stalwart.json perl -I<PATH_TO_PERL_LIB> -I lib t/basic.t
IMAP4 protocol compliance
Stalwart's IMAP protocol compliance may be also tested with Dovecot's ImapTest:
- Download ImapTest.
- Start a test Stalwart instance by running:
$ cargo run --manifest-path=crates/main/Cargo.toml -- --config=./tests/resources/test_config.toml
- Run the compliance tests as follows:
$ ./imaptest host=127.0.0.1 port=9991 \
user=john pass=12345 auth=100 \
test=<PATH_TO_REPO>/tests/resources/imap-test
Note: The tests distributed with ImapTest were slightly modified to support the IMAP4rev2 specification.
IMAP4 Stress tests
Stress testing Stalwart can be done with Dovecot's ImapTest:
- Download ImapTest.
- Start a test Stalwart instance by running:
$ cargo run --manifest-path=crates/main/Cargo.toml -- --config=./tests/resources/test_config.toml
- Create a test
users.txt
file by running:$ echo "john\njane\nbill\n" > users.txt
- Run the stress tests as follows:
./imaptest host=127.0.0.1 port=9991 \
userfile=users.txt \
pass=<12345 \
mbox=<PATH_TO_TEST_MBOX> \
auth=100
WebDAV protocol compliance
Stalwart's WebDAV protocol compliance may be also tested with Litmus' WebDAV test suite.
First, clone the Litmus repository and compile:
$ git clone https://github.com/notroj/litmus.git
$ cd litmus
$ git submodule update --init
$ ./autogen.sh
$ ./configure
$ make
Then, create in Stalwart a test account with the username john
and password 12345
and run the following command to run the WebDAV tests:
$ /usr/local/bin/litmus "http://localhost:8080/dav/file/john" "john" "12345"
Fuzzing
To run the fuzz tests please refer to the Stalwart libraries that handle parsing for the mail server: