FeministWiki:Server setup: Difference between revisions
Technician (talk | contribs) No edit summary |
Technician (talk | contribs) No edit summary |
||
Line 4: | Line 4: | ||
== Initial setup of the new server == | == Initial setup of the new server == | ||
This section describes various initialization tasks for the new server that are independent of the old server. | |||
=== Make feministwiki.dev point to the new server === | |||
During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server. | |||
=== Update & upgrade === | === Update & upgrade === | ||
Line 47: | Line 53: | ||
ufw enable | ufw enable | ||
=== | === Enable extra repositories === | ||
We | We might want to add some additional package repositories so we can use the latest version of some of the used software. | ||
Backports: | Backports is always OK to add since the packages don't get priority over the stable ones: | ||
echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list | echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list | ||
Line 67: | Line 73: | ||
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list | echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list | ||
Now we can install | === Install server components === | ||
Now we can install all the software used for the various FeministWiki services: | |||
php_version=7.4 # or whatever version we're on | php_version=7.4 # or whatever version we're on | ||
Line 91: | Line 99: | ||
Example for installing ejabberd from backports instead: | Example for installing ejabberd from backports instead: | ||
apt-get install ejabberd/$(lsb_release -sc)-backports | apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports | ||
=== Enable Apache modules === | === Enable Apache modules === | ||
Line 110: | Line 118: | ||
=== Put config files in place === | === Put config files in place === | ||
The principle is simple: take all the config files from <code>/root/repo/etc</code> and put them where they belong in <code>/etc</code>. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process | The principle is simple: take all the config files from <code>/root/repo/etc</code> and put them where they belong in <code>/etc</code>. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our old config, to figure out what our new config should look like. | ||
There's one special thing to remember, which is that after copying in the new <code>/etc/aliases</code> file, you have to run the <code>newaliases</code> command for the changes to take effect. | There's one special thing to remember, which is that after copying in the new <code>/etc/aliases</code> file, you have to run the <code>newaliases</code> command for the changes to take effect. | ||
=== Initialize LetsEncrypt === | === Initialize LetsEncrypt === | ||
Line 137: | Line 141: | ||
== Copying over live data == | == Copying over live data == | ||
We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data. | |||
=== Copy over LDAP databases === | === Copy over LDAP databases === | ||
Line 147: | Line 149: | ||
=== Copy over SQL databases === | === Copy over SQL databases === | ||
The following command can be used to create a dump of all databases and import it on the new server. You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. (Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want.) | Stop the database server: | ||
systemctl stop mariadb | |||
The following command can be used to create a dump of all databases and import it on the new server. You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. (Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.) | |||
mysqldump -u root -p"$(cat /root/pwd/mysql)" \ | mysqldump -u root -p"$(cat /root/pwd/mysql)" \ | ||
Line 171: | Line 177: | ||
tar -czf - . | ssh root@feministwiki.dev 'cd /var/www; tar -xzf -' | tar -czf - . | ssh root@feministwiki.dev 'cd /var/www; tar -xzf -' | ||
== | === Reboot === | ||
We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. | |||
reboot | |||
=== Test! === | |||
=== | |||
At this point you should test everything using the feministwiki.dev domain name. | |||
== Finishing up == | |||
'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.''' | |||
=== Update DNS entries === | === Update DNS entries === | ||
Line 210: | Line 212: | ||
For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP. | For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP. | ||
=== Open ports === | |||
We are almost ready to serve: | |||
for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777 | |||
do ufw allow proto tcp to 0.0.0.0/0 port $port | |||
done | |||
=== Update the certificate === | === Update the certificate === | ||
Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev. | Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev. | ||
(This requires at least port 80 to be opened via the command in the last step.) | |||
After this, everything should be functional. If not, it's time for some debugging! |
Revision as of 19:06, 17 August 2021
!!! WORK IN PROGRESS !!!
These are the steps required to set up a new FeministWiki Debian server.
Initial setup of the new server
This section describes various initialization tasks for the new server that are independent of the old server.
Make feministwiki.dev point to the new server
During setup and testing of the new server, we want to make it accessible under the feministwiki.dev domain. So change the A
entry of the feministwiki.dev DNS settings to point to the IP address of the new server.
Update & upgrade
First of all, let's make sure the system is up to date.
apt-get update apt-get upgrade apt-get dist-upgrade
Install miscellaneous tools
Some of these are needed further down, some are just good to have.
apt-get install certbot apt-get install dnsutils apt-get install emacs apt-get install git apt-get install mg apt-get install moreutils apt-get install net-tools apt-get install nmap apt-get install software-properties-common apt-get install tree
Fetch scripts & config repo
Set up GitHub ssh access by copying the .ssh/id_rsa
from the old server. After that:
cd ~ git clone git@github.com:FeministWiki/FeministWiki.git repo cp -a repo/root/* repo/root/.??* . sh repo/decrypt-pwd.sh
The decryption script will prompt you for a password the first time it's used. Enter the password stored in /root/pwd/meta
on the old server.
Set up firewall
For now, block everything but SSH.
apt-get install ufw ufw allow proto tcp to 0.0.0.0/0 port 22 ufw enable
Enable extra repositories
We might want to add some additional package repositories so we can use the latest version of some of the used software.
Backports is always OK to add since the packages don't get priority over the stable ones:
echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list
PHP repo only if a very new version is needed:
wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list
MariaDB repo only if a very new version is needed:
wget https://mariadb.org/mariadb_release_signing_key.asc apt-key add mariadb_release_signing_key.asc rm mariadb_release_signing_key.asc echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list
Install server components
Now we can install all the software used for the various FeministWiki services:
php_version=7.4 # or whatever version we're on apt-get install apache2 apt-get install dovecot-core apt-get install dovecot-imapd apt-get install dovecot-pop3d apt-get install ejabberd # good candidate for backports apt-get install fail2ban apt-get install inspircd apt-get install mailman apt-get install mariadb-server apt-get install opendkim apt-get install php${php_version} apt-get install php${php_version}-mbstring apt-get install php${php_version}-xml apt-get install postfix apt-get install slapd
If any installation asks you for a password, remember that most passwords are found in /root/pwd
.
Example for installing ejabberd from backports instead:
apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports
Enable Apache modules
We need a number of Apache modules to be enabled which might not be enabled by default:
a2enmod expires a2enmod headers a2enmod macro a2enmod rewrite a2enmod ssl
Create vmail user
groupadd -g 5000 vmail useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail
Put config files in place
The principle is simple: take all the config files from /root/repo/etc
and put them where they belong in /etc
. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our old config, to figure out what our new config should look like.
There's one special thing to remember, which is that after copying in the new /etc/aliases
file, you have to run the newaliases
command for the changes to take effect.
Initialize LetsEncrypt
First, initialize the certbot configuration:
certbot register -n --agree-tos -m technician@feministwiki.org
Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:
ufw allow 80 letsencrypt-refresh --dev-only ufw delete allow 80
Our letsencrypt-refresh
script makes sure that the cert files are found in /etc/fw-certs
and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:
adduser ejabberd ssl-cert adduser irc ssl-cert
Copying over live data
We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.
Copy over LDAP databases
TODO
Copy over SQL databases
Stop the database server:
systemctl stop mariadb
The following command can be used to create a dump of all databases and import it on the new server. You can use the show databases;
command in the SQL console to make sure that the list of databases is complete. (Unfortunately they have to be listed manually, because using the --all-databases
option includes system databases that we don't want to copy.)
mysqldump -u root -p"$(cat /root/pwd/mysql)" \ --databases blogs \ feministblog \ feministfiles \ feministforum \ feministmail \ feministsocial \ feministwiki \ feministwiki_de \ feministwiki_es \ feministwiki_it \ feministwiki_pt \ fff \ | ssh root@feministwiki.dev 'mysql -u root -p"$(cat /root/pwd/mysql)"'
Copy over /var/www
This is very simple but takes a lot of time to finish:
cd /var/www tar -czf - . | ssh root@feministwiki.dev 'cd /var/www; tar -xzf -'
Reboot
We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot.
reboot
Test!
At this point you should test everything using the feministwiki.dev domain name.
Finishing up
Now, all services on the old server should be stopped, because we will begin the final transfer of live data.
Update DNS entries
You have to change the configuration of the following domains:
- feministwiki.org
- feministwiki.net
- feministwiki.de
- fem.wiki
- fffrauen.de
feministwiki.org
You only have to change three DNS entries, since most of the subdomains work via CNAME entries:
- The main
A
entry for@
(self-reference i.e. feministwiki.org) - The
A
entry forsmtp
since this is not allowed to be a CNAME - The
A
entry forxmpp
since this is not allowed to be a CNAME
feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de
For these, you only have to change the main A
entry, since they don't use SMTP or XMPP.
Open ports
We are almost ready to serve:
for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777 do ufw allow proto tcp to 0.0.0.0/0 port $port done
Update the certificate
Run the letsencrypt-refresh
script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.
(This requires at least port 80 to be opened via the command in the last step.)
After this, everything should be functional. If not, it's time for some debugging!