« FeministWiki:Server setup » : différence entre les versions

    De FeministWiki
     
    (25 versions intermédiaires par le même utilisateur non affichées)
    Ligne 1 : Ligne 1 :
    '''!!! WORK IN PROGRESS !!!'''
    These are the steps required to set up a new FeministWiki Debian server.
    These are the steps required to set up a new FeministWiki Debian server.


    Ligne 6 : Ligne 4 :


    This section describes various initialization tasks for the new server that are independent of the old server.
    This section describes various initialization tasks for the new server that are independent of the old server.
    === Configure reverse DNS ===
    In the settings of the VPS host (e.g. Strato AG), you can configure reverse-DNS for the IP address of the server.  Set the FQDN for the IP address to {{C|feministwiki.org}}.  It's good to do this early since it can take some time to propagate.


    === Make feministwiki.dev point to the new server ===
    === Make feministwiki.dev point to the new server ===


    During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain.  So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.
    During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain.  So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.


    === Update & upgrade ===
    === Update & upgrade ===
    Ligne 23 : Ligne 25 :
    Some of these are needed further down, some are just good to have.
    Some of these are needed further down, some are just good to have.


      apt-get install automysqlbackup
      apt-get install automysqlbackup \
    apt-get install certbot
                    certbot \
    apt-get install dnsutils
                    dnsutils \
    apt-get install emacs
                    emacs \
    apt-get install git
                    git \
    apt-get install mg
                    mg \
    apt-get install moreutils
                    moreutils \
    apt-get install net-tools
                    net-tools \
    apt-get install nmap
                    nmap \
    apt-get install rsync
                    rsync \
    apt-get install software-properties-common
                    software-properties-common \
    apt-get install tree
                    tree


    === Fetch scripts & config repo ===
    === Fetch scripts & config repo ===
        
        
    Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server.  After that:
    Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server.  After that:
        
        
      cd ~
      cd ~
    Ligne 45 : Ligne 47 :
      sh repo/decrypt-pwd.sh
      sh repo/decrypt-pwd.sh


    The decryption script will prompt you for a password the first time it's used.  Enter the password stored in <code>/root/pwd/meta</code> on the old server.
    The decryption script will prompt you for a password the first time it's used.  Enter the password stored in {{C|/root/pwd/meta}} on the old server.


    === Set up firewall ===
    === Set up firewall ===
    Ligne 79 : Ligne 81 :
    Now we can install all the software used for the various FeministWiki services:
    Now we can install all the software used for the various FeministWiki services:


      apt-get install apache2
      apt-get install apache2 \
    apt-get install dovecot-core
                    dovecot-core \
    apt-get install dovecot-imapd
                    dovecot-imapd \
    apt-get install dovecot-ldap
                    dovecot-ldap \
    apt-get install dovecot-pop3d
                    dovecot-pop3d \
    apt-get install ejabberd # good candidate for backports
                    ejabberd \
    apt-get install fail2ban
                    fail2ban \
    apt-get install inspircd
                    inspircd \
    apt-get install mailman
                    mailman \
    apt-get install mariadb-server
                    mariadb-server \
    apt-get install opendkim
                    opendkim \
    apt-get install postfix
                    postfix \
    apt-get install slapd
                    postfix-ldap \
                    slapd


    If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.
    If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.


    Example for installing ejabberd from backports instead:
    Example for installing ejabberd from backports instead:
    Ligne 105 : Ligne 108 :
      php_version=7.4 # or whatever version we're on
      php_version=7.4 # or whatever version we're on
       
       
      apt-get install php${php_version}
      apt-get install php${php_version} \
    apt-get install php${php_version}-apcu
                    php${php_version}-apcu \
    apt-get install php${php_version}-bcmath
                    php${php_version}-bcmath \
    apt-get install php${php_version}-cli
                    php${php_version}-cli \
    apt-get install php${php_version}-ctype
                    php${php_version}-ctype \
    apt-get install php${php_version}-curl
                    php${php_version}-curl \
    apt-get install php${php_version}-gd
                    php${php_version}-gd \
    apt-get install php${php_version}-gmp
                    php${php_version}-gmp \
    apt-get install php${php_version}-iconv
                    php${php_version}-iconv \
    apt-get install php${php_version}-imagick
                    php${php_version}-imagick \
    apt-get install php${php_version}-intl
                    php${php_version}-intl \
    apt-get install php${php_version}-json
                    php${php_version}-json \
    apt-get install php${php_version}-ldap
                    php${php_version}-ldap \
    apt-get install php${php_version}-mbstring
                    php${php_version}-mbstring \
    apt-get install php${php_version}-mysql
                    php${php_version}-mysql \
    apt-get install php${php_version}-opcache
                    php${php_version}-opcache \
    apt-get install php${php_version}-readline
                    php${php_version}-readline \
    apt-get install php${php_version}-xml
                    php${php_version}-xml \
    apt-get install php${php_version}-zip
                    php${php_version}-zip
     
    === Put config files in place ===
     
    The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}.  However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config.  Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.
     
    There's a number of things important to remember however:
    * After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
    * After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts
    * Likewise, don't forget {{C|chmod +x}} for <code>/etc/cron.{hourly,daily,weekly,monthly}</code> and {{C|/etc/boot.d}}


    === Enable Apache modules, config, and sites ===
    === Enable Apache modules, config, and sites ===
    Ligne 138 : Ligne 150 :
       
       
      a2ensite 000-wiki
      a2ensite 000-wiki
    a2ensite account
      a2ensite blogs
      a2ensite blogs
      a2ensite chat
      a2ensite chat
    Ligne 143 : Ligne 156 :
      a2ensite forum
      a2ensite forum
      a2ensite mail
      a2ensite mail
    a2ensite xmpp


    === Create vmail user ===
    === Create vmail user ===
    Ligne 148 : Ligne 162 :
      groupadd -g 5000 vmail
      groupadd -g 5000 vmail
      useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail
      useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail
    === Put config files in place ===
    The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}.  However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config.  Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.
    There's a number of things important to remember however:
    * After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
    * After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts


    === Initialize LetsEncrypt ===
    === Initialize LetsEncrypt ===
    Ligne 169 : Ligne 175 :
      ufw delete allow 80
      ufw delete allow 80


    Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members.  A number of users have to be added to this group so they can read said files:
    Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members.  A number of users have to be added to this group so they can read said files:


      adduser ejabberd ssl-cert
      adduser ejabberd ssl-cert
    Ligne 176 : Ligne 182 :
    == Copying over live data ==
    == Copying over live data ==


    We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.
    We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.
     
    Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.


    === LDAP databases ===
    === LDAP databases ===
    Ligne 194 : Ligne 198 :
      slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'
      slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'


    Note:
    ==== Breaking changes in OpenLDAP ====


    '''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''
    There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.


    Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:
    Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:
    Ligne 204 : Ligne 208 :
    The solution is as follows:
    The solution is as follows:


    # On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>.  You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
    # On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}.  You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
    # On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined.  It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK).  There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition.  Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.
    # On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined.  It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK).  There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition.  Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.


    The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide.  It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version.  (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)
    The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide.  It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version.  (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)
    Ligne 213 : Ligne 217 :
    This is very simple but takes a lot of time to finish.  '''Run it from the old server:'''
    This is very simple but takes a lot of time to finish.  '''Run it from the old server:'''


      rsync -a --delete /var/www/ root@feministwiki.dev:/var/www
      rsync -az --delete /var/www/ root@feministwiki.dev:/var/www


    Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.
    Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.


    === SQL databases ===
    === SQL databases ===
    Ligne 233 : Ligne 237 :
                   feministwiki_pt \
                   feministwiki_pt \
                   fff \
                   fff \
       | ssh root@feministwiki.dev /root/bin/sql
       | gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'


    You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete.  Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.
    You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete.  Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.


    === Emails ===
    === Emails ===
    Ligne 241 : Ligne 245 :
    This is a simple one.  Run this command from the old server:
    This is a simple one.  Run this command from the old server:


      rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail
      rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail
     
    Note that the trailing slash in {{C|/home/vmail/}} is important.


    === Mailman data ===
    === Mailman data ===
    Ligne 248 : Ligne 254 :


      cd /var/lib/mailman
      cd /var/lib/mailman
      tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'
      rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman
     
    And then this on the new server:
     
    check_perms -f


    The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.
    The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.


    == Recreate SQL users ==
    == Recreate SQL users ==
    Ligne 271 : Ligne 281 :
      create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
      create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
      grant all on feministwiki.* to feministwiki@localhost;
      grant all on feministwiki.* to feministwiki@localhost;
    grant all on feministwiki_de.* to feministwiki@localhost;
    grant all on feministwiki_es.* to feministwiki@localhost;
    grant all on feministwiki_it.* to feministwiki@localhost;
    grant all on feministwiki_pt.* to feministwiki@localhost;
       
       
      create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
      create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
    Ligne 297 : Ligne 311 :


    Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name.  This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.
    Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name.  This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.
    === Deactivate again ===
    After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:
    for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
    do ufw delete allow proto tcp to 0.0.0.0/0 port $port
    done
    systemctl stop apache2
    systemctl stop dovecot
    systemctl stop ejabberd
    systemctl stop inspircd
    systemctl stop mailman
    systemctl stop postfix
    systemctl stop slapd


    == Finishing up ==
    == Finishing up ==
    Ligne 322 : Ligne 352 :
    === Copy over the live data one more time ===
    === Copy over the live data one more time ===


    The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server.  (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)
    '''Simply repeat the whole section ''Copying over live data''.'''
     
    The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server.  For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.
     
    So just repeat the steps from that section exactly one more time.


    So in short, just repeat the steps from that section exactly one more time.
    === Reboot the new server ===
     
    At this point we can reboot the new server again, to make sure all services are properly restarted.
     
    === Open ports on the new server ===
     
    Now we can open the ports again on the new server:
     
    for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
    do ufw allow proto tcp to 0.0.0.0/0 port $port
    done


    === Update DNS entries ===
    === Update DNS entries ===
    Ligne 340 : Ligne 384 :
    You only have to change three DNS entries, since most of the subdomains work via CNAME entries:
    You only have to change three DNS entries, since most of the subdomains work via CNAME entries:


    * The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
    * The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
    * The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
    * The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
    * The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME
    * The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME


    ==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====
    ==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====


    For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.
    For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.


    === Update the certificate ===
    === Update the certificate ===


    Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.
    Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.


    After this, everything should be functional.  If not, it's time for some debugging!
    After this, everything should be functional.  If not, it's time for some debugging!

    Dernière version du 31 août 2021 à 19:44

    These are the steps required to set up a new FeministWiki Debian server.

    Initial setup of the new server

    This section describes various initialization tasks for the new server that are independent of the old server.

    Configure reverse DNS

    In the settings of the VPS host (e.g. Strato AG), you can configure reverse-DNS for the IP address of the server. Set the FQDN for the IP address to feministwiki.org. It's good to do this early since it can take some time to propagate.

    Make feministwiki.dev point to the new server

    During setup and testing of the new server, we want to make it accessible under the feministwiki.dev domain. So change the A entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

    Update & upgrade

    First of all, let's make sure the system is up to date.

    apt-get update
    apt-get upgrade
    apt-get dist-upgrade
    

    Install miscellaneous tools

    Some of these are needed further down, some are just good to have.

    apt-get install automysqlbackup \
                    certbot \
                    dnsutils \
                    emacs \
                    git \
                    mg \
                    moreutils \
                    net-tools \
                    nmap \
                    rsync \
                    software-properties-common \
                    tree
    

    Fetch scripts & config repo

    Set up GitHub ssh access by copying the .ssh/id_rsa from the old server. After that:

    cd ~
    git clone git@github.com:FeministWiki/FeministWiki.git repo
    cp -a repo/root/* repo/root/.??* .
    sh repo/decrypt-pwd.sh
    

    The decryption script will prompt you for a password the first time it's used. Enter the password stored in /root/pwd/meta on the old server.

    Set up firewall

    For now, block everything but SSH.

    apt-get install ufw
    ufw allow proto tcp to 0.0.0.0/0 port 22
    ufw enable
    

    Enable extra repositories

    We might want to add some additional package repositories so we can use the latest version of some of the used software.

    Backports is always OK to add since the packages don't get priority over the stable ones:

    echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list
    

    PHP repo only if a very new version is needed:

    wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
    echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list
    

    MariaDB repo only if a very new version is needed:

    wget https://mariadb.org/mariadb_release_signing_key.asc
    apt-key add mariadb_release_signing_key.asc
    rm mariadb_release_signing_key.asc
    echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list
    

    Install server components

    Now we can install all the software used for the various FeministWiki services:

    apt-get install apache2 \
                    dovecot-core \
                    dovecot-imapd \
                    dovecot-ldap \
                    dovecot-pop3d \
                    ejabberd \
                    fail2ban \
                    inspircd \
                    mailman \
                    mariadb-server \
                    opendkim \
                    postfix \
                    postfix-ldap \
                    slapd
    

    If any installation asks you for a password, remember that most passwords are found in /root/pwd.

    Example for installing ejabberd from backports instead:

    apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports
    

    Install PHP and modules

    This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

    php_version=7.4 # or whatever version we're on
    
    apt-get install php${php_version} \
                    php${php_version}-apcu \
                    php${php_version}-bcmath \
                    php${php_version}-cli \
                    php${php_version}-ctype \
                    php${php_version}-curl \
                    php${php_version}-gd \
                    php${php_version}-gmp \
                    php${php_version}-iconv \
                    php${php_version}-imagick \
                    php${php_version}-intl \
                    php${php_version}-json \
                    php${php_version}-ldap \
                    php${php_version}-mbstring \
                    php${php_version}-mysql \
                    php${php_version}-opcache \
                    php${php_version}-readline \
                    php${php_version}-xml \
                    php${php_version}-zip
    

    Put config files in place

    The principle is simple: take all the config files from /root/repo/etc and put them where they belong in /etc. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

    There's a number of things important to remember however:

    • After copying in the new /etc/aliases file, run newaliases for the changes to take effect
    • After populating /etc/letsencrypt/renewal-hooks, remember to chmod +x all the scripts
    • Likewise, don't forget chmod +x for /etc/cron.{hourly,daily,weekly,monthly} and /etc/boot.d

    Enable Apache modules, config, and sites

    We need a number of Apache modules to be enabled which might not be enabled by default:

    a2enmod expires
    a2enmod headers
    a2enmod macro
    a2enmod rewrite
    a2enmod ssl
    
    a2enconf 99-feministwiki
    
    a2ensite 000-wiki
    a2ensite account
    a2ensite blogs
    a2ensite chat
    a2ensite files
    a2ensite forum
    a2ensite mail
    a2ensite xmpp
    

    Create vmail user

    groupadd -g 5000 vmail
    useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail
    

    Initialize LetsEncrypt

    First, initialize the certbot configuration:

    certbot register -n --agree-tos -m technician@feministwiki.org
    

    Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

    ufw allow 80
    letsencrypt-refresh --dev-only
    ufw delete allow 80
    

    Our letsencrypt-refresh script makes sure that the cert files are found in /etc/fw-certs and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

    adduser ejabberd ssl-cert
    adduser irc ssl-cert
    

    Copying over live data

    We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

    LDAP databases

    Stop the LDAP server and delete the existing configuration and data on the new server (careful!):

    # Commands to run on the NEW (fresh) server:
    systemctl stop slapd
    rm -r /etc/ldap/slapd.d/*
    rm /var/lib/ldap/data.mdb
    

    Then copy over the config and data by running these commands from the old server:

    slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
    slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'
    

    Breaking changes in OpenLDAP

    There might be incompatible changes between OpenLDAP (aka slapd) versions which require manual editing of the slapcat output before it's read in on the new server with slapadd.

    Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

    User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined
    

    The solution is as follows:

    1. On the new server, open /etc/ldap/schema/ppolicy.ldif and search for pwdMaxRecordedFailure. You will note that there is a olcAttributeTypes: ... entry that defines it, and also it's listed in the MAY attributes block of the olcObjectClasses: ... entry that defines the pwdPolicy object class.
    2. On the old server, save the output of slapcat -n 0 to a file, open the file, and search for the block where the ppolicy schema is defined. It should start with the line {{{1}}} (the {4} part might contain a different integer, that's OK). There, note that the olcAttributeTypes: ... entry for pwdMaxRecordedFailure is missing, and also it's not listed in the MAY list of the pwdPolicy object class definition. Copy over the attribute type definition from the ppolicy.ldif file on the new server, and amend the MAY list to include it.

    The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

    Contents of /var/www

    This is very simple but takes a lot of time to finish. Run it from the old server:

    rsync -az --delete /var/www/ root@feministwiki.dev:/var/www
    

    Note that the trailing slash in /var/www/ is important; if not provided, it will copy the directory to /var/www/www on the new server.

    SQL databases

    Run the following command from the old server:

    mysqldump -u root -p"$(cat /root/pwd/mysql)" \
      --add-drop-database \
      --databases blogs \
                  feministfiles \
                  feministforum \
                  feministmail \
                  feministwiki \
                  feministwiki_de \
                  feministwiki_es \
                  feministwiki_it \
                  feministwiki_pt \
                  fff \
      | gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'
    

    You can use the show databases; command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the --all-databases option includes system databases that we don't want to copy.

    Emails

    This is a simple one. Run this command from the old server:

    rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail
    

    Note that the trailing slash in /home/vmail/ is important.

    Mailman data

    GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

    cd /var/lib/mailman
    rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman
    

    And then this on the new server:

    check_perms -f
    

    The check_perms command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

    Recreate SQL users

    If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the mysql.user table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

    /root/bin/sql << EOF
    create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
    grant all on blogs.* to blogs@localhost;
    
    create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
    grant all on feministfiles.* to feministfiles@localhost;
    
    create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
    grant all on feministforum.* to feministforum@localhost;
    
    create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
    grant all on feministmail.* to feministmail@localhost;
    
    create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
    grant all on feministwiki.* to feministwiki@localhost;
    grant all on feministwiki_de.* to feministwiki@localhost;
    grant all on feministwiki_es.* to feministwiki@localhost;
    grant all on feministwiki_it.* to feministwiki@localhost;
    grant all on feministwiki_pt.* to feministwiki@localhost;
    
    create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
    grant all on fff.* to fff@localhost;
    EOF
    

    Test

    It's important to test the new server to make sure everything works well!

    Reboot

    We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

    Open ports

    We need to open all the ports used by the various FeministWiki services:

    for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
    do ufw allow proto tcp to 0.0.0.0/0 port $port
    done
    

    Test!

    At this point you should test everything using the feministwiki.dev domain name.

    Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

    Deactivate again

    After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:

    for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
    do ufw delete allow proto tcp to 0.0.0.0/0 port $port
    done
    
    systemctl stop apache2
    systemctl stop dovecot
    systemctl stop ejabberd
    systemctl stop inspircd
    systemctl stop mailman
    systemctl stop postfix
    systemctl stop slapd
    

    Finishing up

    Now, all services on the old server should be stopped, because we will begin the final transfer of live data.

    Stop services on the old server

    Stop all the services that interface with users and/or are responsible for modifying live data:

    systemctl stop apache2
    systemctl stop dovecot
    systemctl stop ejabberd
    systemctl stop inspircd
    systemctl stop mailman
    systemctl stop postfix
    systemctl stop slapd
    

    Close all the relevant ports just to be double-sure:

    for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
    do ufw delete allow proto tcp to 0.0.0.0/0 port $port
    done
    

    Copy over the live data one more time

    Simply repeat the whole section Copying over live data.

    The techniques and commands described above in the section Copying over live data are idempotent, meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. For instance, the --delete argument to the rsync command and the --add-drop-database argument to the mysqldump command help to make sure of this.

    So just repeat the steps from that section exactly one more time.

    Reboot the new server

    At this point we can reboot the new server again, to make sure all services are properly restarted.

    Open ports on the new server

    Now we can open the ports again on the new server:

    for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
    do ufw allow proto tcp to 0.0.0.0/0 port $port
    done
    

    Update DNS entries

    You have to change the configuration of the following domains:

    • feministwiki.org
    • feministwiki.net
    • feministwiki.de
    • fem.wiki
    • fffrauen.de

    feministwiki.org

    You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

    • The main A entry for @ (self-reference i.e. feministwiki.org)
    • The A entry for smtp since this is not allowed to be a CNAME
    • The A entry for xmpp since this is not allowed to be a CNAME

    feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de

    For these, you only have to change the main A entry, since they don't use SMTP or XMPP.

    Update the certificate

    Run the letsencrypt-refresh script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

    After this, everything should be functional. If not, it's time for some debugging!