FeministWiki:Server setup: Difference between revisions
Technician (talk | contribs) |
Technician (talk | contribs) |
||
| (18 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
This section describes various initialization tasks for the new server that are independent of the old server. | This section describes various initialization tasks for the new server that are independent of the old server. | ||
=== Configure reverse DNS === | |||
In the settings of the VPS host (e.g. Strato AG), you can configure reverse-DNS for the IP address of the server. Set the FQDN for the IP address to {{C|feministwiki.org}}. It's good to do this early since it can take some time to propagate. | |||
=== Make feministwiki.dev point to the new server === | === Make feministwiki.dev point to the new server === | ||
During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the | During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server. | ||
=== Update & upgrade === | === Update & upgrade === | ||
| Line 21: | Line 25: | ||
Some of these are needed further down, some are just good to have. | Some of these are needed further down, some are just good to have. | ||
apt-get install automysqlbackup | apt-get install automysqlbackup \ | ||
certbot \ | |||
dnsutils \ | |||
emacs \ | |||
git \ | |||
mg \ | |||
moreutils \ | |||
net-tools \ | |||
nmap \ | |||
rsync \ | |||
software-properties-common \ | |||
tree | |||
=== Fetch scripts & config repo === | === Fetch scripts & config repo === | ||
Set up GitHub ssh access by copying the | Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that: | ||
cd ~ | cd ~ | ||
| Line 43: | Line 47: | ||
sh repo/decrypt-pwd.sh | sh repo/decrypt-pwd.sh | ||
The decryption script will prompt you for a password the first time it's used. Enter the password stored in | The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server. | ||
=== Set up firewall === | === Set up firewall === | ||
| Line 77: | Line 81: | ||
Now we can install all the software used for the various FeministWiki services: | Now we can install all the software used for the various FeministWiki services: | ||
apt-get install apache2 | apt-get install apache2 \ | ||
dovecot-core \ | |||
dovecot-imapd \ | |||
dovecot-ldap \ | |||
dovecot-pop3d \ | |||
ejabberd \ | |||
fail2ban \ | |||
inspircd \ | |||
mailman \ | |||
mariadb-server \ | |||
opendkim \ | |||
postfix \ | |||
postfix-ldap \ | |||
slapd | |||
If any installation asks you for a password, remember that most passwords are found in | If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}. | ||
Example for installing ejabberd from backports instead: | Example for installing ejabberd from backports instead: | ||
| Line 104: | Line 108: | ||
php_version=7.4 # or whatever version we're on | php_version=7.4 # or whatever version we're on | ||
apt-get install php${php_version} | apt-get install php${php_version} \ | ||
php${php_version}-apcu \ | |||
php${php_version}-bcmath \ | |||
php${php_version}-cli \ | |||
php${php_version}-ctype \ | |||
php${php_version}-curl \ | |||
php${php_version}-gd \ | |||
php${php_version}-gmp \ | |||
php${php_version}-iconv \ | |||
php${php_version}-imagick \ | |||
php${php_version}-intl \ | |||
php${php_version}-json \ | |||
php${php_version}-ldap \ | |||
php${php_version}-mbstring \ | |||
php${php_version}-mysql \ | |||
php${php_version}-opcache \ | |||
php${php_version}-readline \ | |||
php${php_version}-xml \ | |||
php${php_version}-zip | |||
=== Put config files in place === | === Put config files in place === | ||
| Line 131: | Line 135: | ||
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect | * After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect | ||
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts | * After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts | ||
* Likewise, don't forget {{C|chmod +x}} for <code>/etc/cron.{hourly,daily,weekly,monthly}</code> and {{C|/etc/boot.d}} | |||
=== Enable Apache modules, config, and sites === | === Enable Apache modules, config, and sites === | ||
| Line 170: | Line 175: | ||
ufw delete allow 80 | ufw delete allow 80 | ||
Our | Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files: | ||
adduser ejabberd ssl-cert | adduser ejabberd ssl-cert | ||
| Line 193: | Line 198: | ||
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1' | slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1' | ||
==== Breaking changes in OpenLDAP ==== | |||
There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}. | |||
Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message: | Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message: | ||
| Line 203: | Line 208: | ||
The solution is as follows: | The solution is as follows: | ||
# On the new server, open | # On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class. | ||
# On the old server, save the output of | # On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it. | ||
The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!) | The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!) | ||
| Line 212: | Line 217: | ||
This is very simple but takes a lot of time to finish. '''Run it from the old server:''' | This is very simple but takes a lot of time to finish. '''Run it from the old server:''' | ||
rsync - | rsync -az --delete /var/www/ root@feministwiki.dev:/var/www | ||
Note that the trailing slash in | Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server. | ||
=== SQL databases === | === SQL databases === | ||
| Line 232: | Line 237: | ||
feministwiki_pt \ | feministwiki_pt \ | ||
fff \ | fff \ | ||
| ssh root@feministwiki.dev /root/bin/sql | | gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql' | ||
You can use the | You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy. | ||
=== Emails === | === Emails === | ||
| Line 240: | Line 245: | ||
This is a simple one. Run this command from the old server: | This is a simple one. Run this command from the old server: | ||
rsync - | rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail | ||
Note that the trailing slash in {{C|/home/vmail/}} is important. | Note that the trailing slash in {{C|/home/vmail/}} is important. | ||
| Line 249: | Line 254: | ||
cd /var/lib/mailman | cd /var/lib/mailman | ||
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman | |||
And then this on the new server: | |||
The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing | check_perms -f | ||
The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions. | |||
== Recreate SQL users == | == Recreate SQL users == | ||
| Line 272: | Line 281: | ||
create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)'; | create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)'; | ||
grant all on feministwiki.* to feministwiki@localhost; | grant all on feministwiki.* to feministwiki@localhost; | ||
grant all on feministwiki_de.* to feministwiki@localhost; | |||
grant all on feministwiki_es.* to feministwiki@localhost; | |||
grant all on feministwiki_it.* to feministwiki@localhost; | |||
grant all on feministwiki_pt.* to feministwiki@localhost; | |||
create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)'; | create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)'; | ||
| Line 298: | Line 311: | ||
Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well. | Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well. | ||
=== Deactivate again === | |||
After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over: | |||
for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777 | |||
do ufw delete allow proto tcp to 0.0.0.0/0 port $port | |||
done | |||
systemctl stop apache2 | |||
systemctl stop dovecot | |||
systemctl stop ejabberd | |||
systemctl stop inspircd | |||
systemctl stop mailman | |||
systemctl stop postfix | |||
systemctl stop slapd | |||
== Finishing up == | == Finishing up == | ||
| Line 323: | Line 352: | ||
=== Copy over the live data one more time === | === Copy over the live data one more time === | ||
'''Simply repeat the whole section ''Copying over live data''.''' | |||
The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this. | |||
So just repeat the steps from that section exactly one more time. | |||
=== Reboot the new server === | |||
At this point we can reboot the new server again, to make sure all services are properly restarted. | |||
=== Open ports on the new server === | |||
Now we can open the ports again on the new server: | |||
for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777 | |||
do ufw allow proto tcp to 0.0.0.0/0 port $port | |||
done | |||
=== Update DNS entries === | === Update DNS entries === | ||
| Line 341: | Line 384: | ||
You only have to change three DNS entries, since most of the subdomains work via CNAME entries: | You only have to change three DNS entries, since most of the subdomains work via CNAME entries: | ||
* The main | * The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org) | ||
* The | * The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME | ||
* The | * The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME | ||
==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ==== | ==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ==== | ||
For these, you only have to change the main | For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP. | ||
=== Update the certificate === | === Update the certificate === | ||
Run the | Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev. | ||
After this, everything should be functional. If not, it's time for some debugging! | After this, everything should be functional. If not, it's time for some debugging! | ||