In this tutorial, you will learn the process of hosting your own email server. We will be using the open-source hosting control panel CyberPanel as our email server. The reason for this choice is because CyberPanel makes it extremely easy to generate all the necessary DNS mail records like DKIM, MX, and PTR. This is something that typically requires a great deal of knowledge in order to prevent your emails from being marked as spam.
By the end of this tutorial, you will have a full-functioning and properly configured email server. Let’s get started.
Note #1: In this tutorial, I am using a Linode VPS running Ubuntu 20.04 LTS as my server and NameCheap as my domain name registrar for site1.xyz. While you don’t have to use these services, the following examples will be specific to them, but the same concepts will apply wherever your hosting is and domain name resides.
Note #2: If you do choose to use Linode, email ports are disabled by default. Please follow the instructions here to enable email ports on your Linode server.
1. Stand Up a Server
Wether you use a Linode VPS like I do, or have a physical server in your home, you will need a server to host your email. While I am running Ubuntu 20.04 LTS on my email server, CyberPanel also supports Ubuntu 18.04 at the time of this writing.
Minimum systems requirements for CyberPanel are as follows:
- 1 GB or above RAM
- 10GB or above disk space
Take note of the external IP address of your server. You will need that in the next step. The IP address I will be using in this example is 126.96.36.199.
2. Register Personal DNS Servers
What makes hosting an email server with CyberPanel so easy is the fact that CyberPanel manages all of your DNS mail records for you. In order for this to happen, we need to set up personal DNS servers with our registrar and then proceed to create a private nameserver.
To register personal DNS servers with NameCheap, access your domain name from your Domain List. Click on the Advanced DNS tab and add two standard nameservers of ns1 and ns2 with the IP address of your server.
You will end up with two nameservers, both of which are associated with the same IP address.
Next, go to the Domain tab for your domain name. Under the Nameservers section, select Custom DNS and add the two nameservers that you just created.
3. Install CyberPanel on Your Server
Log in to your server via SSH and install CyberPanel with the official installer script.
sh <(curl https://cyberpanel.net/install.sh || wget -O - https://cyberpanel.net/install.sh)
During the installation, choose option 1 to install CyberPanel with OpenLiteSpeed and make sure you install Postfix which will act as our mail server. All other options in the installation are up to you.
Here are the options I chose.
- 1 – Install CyberPanel
- 1 – Install CyberPanel with OpenLiteSpeed
- Y – Full installation
- N – Remote MySQL
- Enter – Press Enter key to continue with latest version
- s – Specify the admin password
- n – Do you wish to install Memcached extension and backend?
- n – Do you wish to install Redis extension and backend?
- Y – Would you like to set up a WatchDog?
4. Email Server Setup
Now that CyberPanel is installed, we can set up the email server.
The CyberPanel hosting control panel runs on port 8090 by default. To access the dashboard, navigate to your IP address followed :8090. For example, since my IP address is 188.8.131.52, I will go to http://184.108.40.206:8090 in a web browser.
At the login screen, enter admin for the username and the password that you created during installation.
Within the hosting control panel, there are three steps we need to take in order to set up our self-hosted mail server.
A. Create a Website Container
The easiest way to associate your domain name with CyberPanel is to create a website. Although it’s possible, you won’t be hosting any content on the website right now. From the perspective of CyberPanel, this website is more like a container for a website.
To create the website container, go to Websites -> Create Website.
Fill out the form with your domain name and make sure you check the DKIM Support box.
B. Create Nameservers
Remember how you registered personal DNS servers at the beginning of this tutorial? Now you must create these nameservers in CyberPanel.
To create nameservers in CyberPanel, navigate to DNS -> Create Nameserver.
Fill out the form with your nameservers from earlier and the IP address of your server.
C. Issue an SSL Certificate
In order to make our mail server secure and encrypted, you must install an SSL certificate. You can do this in one click with a free SSL certificate from Let’s Encrypt.
Go to SSL -> Mailserver SSL.
Select your domain name from the dropdown list and click on the Issue SSL button.
At this point, it’s a good idea to check the status of your DNS settings. You can use a free website like UltraTools to look up your DNS settings. It may take some time for these changes to propagate across the internet (a few minutes to a couple days), so be patient before moving on to the next step.
5. Set Up Reverse DNS
In order for your mail server to work properly as far as DNS is concerned, it is extremely important to properly configure reverse DNS. Reverse DNS is exactly what you’d expect. Rather than associating a domain name with an IP address like is done with DNS, reverse DNS associates a domain name with an IP address.
To set up reverse DNS in Linode, click on the Networking tab. Under the IPv4 section, click the three dots associated with your IP address and click on Edit RDNS.
In the resulting popup window, type in your domain name to associate your domain name with your IP address. In my case, I typed in site1.xyz.
It may take some time for your changes to propagate across the internet (a few minutes to a couple days). You can check the RDNS status with MXToolBox.
If you are not using Linode, you will need to define a DNS record of type PTR while also maps your IP address with your domain name.
6. Create an Email Address
Now the exciting part you’ve been waiting for! Create an email address in CyberPanel by going to Email -> Create Email.
Fill out the form, specifying your domain name, and choose an email address and password. In my case, I picked firstname.lastname@example.org for my email address.
7. Send and Receive a Test Email
You can access your email inbox by navigating to Email -> Access Webmail. The default email client is RainLoop.
Log in with the email credentials you just created to reveal your inbox.
Compose an email, and send it to a recipient of your choice. Within a few seconds, you will see your email pop up in the recipient’s inbox.
Respond to this email to test out the ability to receive emails.
As one final test, you can check to see if your email server is configured properly from the perspective of spam filters. To see if your mail server is configured properly as far as DNS mail records are concerned, you can use a free tool like Mail Tester. Simply send an email to the address, and within a few seconds, you will be given a spamyness score out of 10.
A full video tutorial of hosting your own email server is available below. Please let me know in the comments of the video or in the comments section below if you have any questions. I’ll do my best to help you out.