FeministWiki

FeministWiki:Server setup: Difference between revisions

Project page Discussion

FeministWiki:Server setup (view source)

Revision as of 20:28, 18 January 2024

2,376 bytes added ,  18 January
→‎Install miscellaneous tools
(30 intermediate revisions by the same user not shown)
Line 28: Line 28:
                 bsdutils \
                 bsdutils \
                 certbot \
                 certbot \
                composer \
                 curl \
                 curl \
                 dnsutils \
                 dnsutils \
Line 33: Line 34:
                 git \
                 git \
                 imagemagick \
                 imagemagick \
                iotop \
                 ldap-utils \
                 ldap-utils \
                 mg \
                 mg \
Line 116: Line 118:
  Pin: origin packages.sury.org
  Pin: origin packages.sury.org
  Pin-Priority: 700
  Pin-Priority: 700
Elasticsearch, if you want CirrusSearch for MediaWiki:
curl https://artifacts.elastic.co/GPG-KEY-elasticsearch | gpg --dearmor -o /etc/apt/trusted.gpg.d/elasticsearch.gpg
# As of January 2024, CirrusSearch only supports Elasticsearch 7.x
echo 'deb https://artifacts.elastic.co/packages/7.x/apt stable main' > /etc/apt/sources.list.d/elastic.list


=== Create vmail user ===
=== Create vmail user ===
Line 126: Line 134:
Now we can install all the software used for the various FeministWiki services:
Now we can install all the software used for the various FeministWiki services:


  apt-get install apache2 \
  apt-get install
                dovecot-core \
    apache2 \
                dovecot-imapd \
    dovecot-core \
                dovecot-ldap \
    dovecot-imapd \
                dovecot-pop3d \
    dovecot-ldap \
                ejabberd \
    dovecot-pop3d \
                fail2ban \
    ejabberd \
                inspircd \
    elasticsearch \
                mariadb-server \
    fail2ban \
                nginx-extras \
    inspircd \
                opendkim \
    mariadb-server \
                postfix \
    nginx-extras \
                postfix-ldap \
    opendkim \
                slapd
    postfix \
    postfix-ldap \
    slapd


If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.
If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.
Line 146: Line 156:


  apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/bookworm-backports
  apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/bookworm-backports
=== Make sure Postfix can connect to OpenDKIM ===
mkdir -p /var/spool/postfix/opendkim
chown opendkim:opendkim /var/spool/postfix/opendkim
adduser postfix opendkim


=== Install PHP and modules ===
=== Install PHP and modules ===
Line 156: Line 172:
                 php${php_version}-apcu \
                 php${php_version}-apcu \
                 php${php_version}-bcmath \
                 php${php_version}-bcmath \
                php${php_version}-bz2 \
                 php${php_version}-cli \
                 php${php_version}-cli \
                 php${php_version}-curl \
                 php${php_version}-curl \
Line 170: Line 187:
                 php${php_version}-xml \
                 php${php_version}-xml \
                 php${php_version}-zip
                 php${php_version}-zip
We also want {{C|php-luasandbox}}, which may not have a PHP version attached to the package name, in which case you'll have to make sure it supports the PHP version currently in use. If not, you can use the standalone Lua binary instead by setting {{C|$wgScribuntoDefaultEngine = 'luastandalone';}} in MediaWiki's {{C|LocalSettings.php}} configuration file.
# See if this works first:
apt-get install php${php_version}-luasandbox
# Otherwise...
apt-get install php-luasandbox
# Check the package contents to see which PHP versions are supported
dpkg -L php-luasandbox


=== Copy over certificates ===
=== Copy over certificates ===
Line 175: Line 202:
Copy over the certs from the old server:
Copy over the certs from the old server:


  tar -czPf- /etc/fw-certs | ssh feministwiki.dev 'tar -xzPf-'
  # Run on old server
rsync -avz /etc/fw-certs feministwiki.dev:/etc/fw-certs


The {{C|/etc/fw-certs}} directory and its contents should be owned by the group {{C|ssl-cert}}.  Make sure this is the case on the new server after running the command above, since the group ID might be different on the new server.  If the group doesn't exist at all, just create it.
The {{C|/etc/fw-certs}} directory and its contents should be owned by the group {{C|ssl-cert}}.  Make sure this is the case on the new server after running the command above, since the group ID might be different on the new server.  If the group doesn't exist at all, just create it.
Line 183: Line 211:
Then, to allow certain services to read those files containing the private key, add them to the {{C|ssl-cert}} group:
Then, to allow certain services to read those files containing the private key, add them to the {{C|ssl-cert}} group:


# Run on new server
  adduser ejabberd ssl-cert
  adduser ejabberd ssl-cert
  adduser irc ssl-cert
  adduser irc ssl-cert
Also copy over the certificates stored directly in {{C|/etc/letsencrypt}}:
# Run on old server
rsync -avz /etc/letsencrypt/{archive,live} feministwiki.dev:/etc/letsencrypt


=== Put config files in place ===
=== Put config files in place ===
Line 201: Line 235:
Enable PHP FPM and other Apache modules:
Enable PHP FPM and other Apache modules:


  a2enmod expires headers proxy_fcgi
  a2enmod expires headers proxy_fcgi rewrite
  a2enconf php${php_version}-fpm
  a2enconf php${php_version}-fpm


Line 209: Line 243:


  # Run on old server
  # Run on old server
  rsync -r /etc/ldap/slapd.d/ feministwiki.dev:/tmp/slapd.d
  rsync -az /etc/ldap/slapd.d/ feministwiki.dev:/tmp/slapd.d


  # Run on new server
  # Run on new server
Line 235: Line 269:
Then copy over the configuration database, by running the following commands from the old server:
Then copy over the configuration database, by running the following commands from the old server:


# Run on old server
  slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
  slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'


Line 241: Line 276:
''Note: This is an '''alternative''' method to that described in the '''previous''' section. See above for which one to choose.''
''Note: This is an '''alternative''' method to that described in the '''previous''' section. See above for which one to choose.''


Running the following sequence of commands, taken from [[FeministWiki:LDAP Schema]], should work:
First run {{C|dpkg-reconfigure slapd}} to fill in some basic information such as the domain name and admin password.  You can reuse the old admin password found in {{C|/root/pwd/ldap}}.
 
Then, running the following sequence of commands, taken from [[FeministWiki:LDAP Schema]], should do the rest:


  # Create fwMember object class
  # Create fwMember object class
Line 365: Line 402:


  # Run on old server
  # Run on old server
  slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'
  slapcat -n 1 | zstd | ssh feministwiki.dev 'zstd -d | sudo -u openldap slapadd -n 1'


Start slapd again in the new server afterwards:
Start slapd again in the new server afterwards:
Line 376: Line 413:
This is very simple but takes a lot of time to finish.  '''Run it from the old server:'''
This is very simple but takes a lot of time to finish.  '''Run it from the old server:'''


  rsync -az --delete /var/www/ feministwiki.dev:/var/www
  rsync -azP --delete /var/www/ feministwiki.dev:/var/www


Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.
Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.
There's actually a systemd service found in {{C|/var/www/fw/wiki}} that you'll want to enable on the new server:
systemctl enable /var/www/fw/wiki/fw-wiki-job-runner.service
No need to actually start it yet.


=== SQL databases ===
=== SQL databases ===
Line 384: Line 427:
Run the following command from the old server:
Run the following command from the old server:


  mysqldump -u root -p"$(cat /root/pwd/mysql)" \
  mariadb-dump -u root -p"$(cat /root/pwd/mariadb)" \
   --add-drop-database \
   --add-drop-database \
   --databases blogs \
   --databases feministblogs \
               feministfiles \
               feministfiles \
               feministforum \
               feministforum \
Line 397: Line 440:
               feministwiki_pt \
               feministwiki_pt \
               fff \
               fff \
   | gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'
   | zstd | ssh feministwiki.dev 'zstd -d | /root/bin/sql'


You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete.  Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.
You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete.  Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.
Line 408: Line 451:


Note that the trailing slash in {{C|/home/vmail/}} is important.
Note that the trailing slash in {{C|/home/vmail/}} is important.
=== Elasticsearch ===
Temporarily stop Elasticsearch on the old server and copy over the data:
systemctl stop elasticsearch
rsync -az --delete /var/lib/elasticsearch/ feministwiki.dev:/var/lib/elasticsearch
systemctl start elasticsearch


=== Mailman data ===
=== Mailman data ===
Line 427: Line 478:


  /root/bin/sql << EOF
  /root/bin/sql << EOF
  create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
  create user 'feministblogs'@localhost identified by '$(cat ~/pwd/mariadb-feministblogs)';
  create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
  create user 'feministfiles'@localhost identified by '$(cat ~/pwd/mariadb-feministfiles)';
  create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
  create user 'feministforum'@localhost identified by '$(cat ~/pwd/mariadb-feministforum)';
  create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
  create user 'feministmail'@localhost identified by '$(cat ~/pwd/mariadb-feministmail)';
  create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
  create user 'feministwiki'@localhost identified by '$(cat ~/pwd/mariadb-feministwiki)';
  create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
  create user 'fff'@localhost identified by '$(cat ~/pwd/mariadb-fff)';
  EOF
  EOF


Line 438: Line 489:


  /root/bin/sql << EOF
  /root/bin/sql << EOF
  grant all on blogs.* to blogs@localhost;
  grant all on feministblogs.* to feministblogs@localhost;
  grant all on feministfiles.* to feministfiles@localhost;
  grant all on feministfiles.* to feministfiles@localhost;
  grant all on feministforum.* to feministforum@localhost;
  grant all on feministforum.* to feministforum@localhost;
Line 488: Line 539:
  systemctl stop dovecot
  systemctl stop dovecot
  systemctl stop ejabberd
  systemctl stop ejabberd
systemctl stop elasticsearch
systemctl stop fw-wiki-job-runner
  systemctl stop inspircd
  systemctl stop inspircd
  systemctl stop mailman
  systemctl stop nginx
systemctl stop opendkim
  systemctl stop postfix
  systemctl stop postfix
  systemctl stop slapd
  systemctl stop slapd
Note that we leave MariaDB running, since it needs to be live for data transfer.


== Finishing up ==
== Finishing up ==
Line 501: Line 557:
Stop all the services that interface with users and/or are responsible for modifying live data:
Stop all the services that interface with users and/or are responsible for modifying live data:


for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done
  systemctl stop apache2
  systemctl stop apache2
  systemctl stop dovecot
  systemctl stop dovecot
  systemctl stop ejabberd
  systemctl stop ejabberd
systemctl stop elasticsearch
systemctl stop fw-wiki-job-runner
  systemctl stop inspircd
  systemctl stop inspircd
  systemctl stop mailman
  systemctl stop nginx
systemctl stop opendkim
  systemctl stop postfix
  systemctl stop postfix
  systemctl stop slapd
  systemctl stop slapd


Close all the relevant ports just to be double-sure:
As with the old server, we leave MariaDB running since it will be needed for data transfer.
 
for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done


=== Copy over the live data one more time ===
=== Copy over the live data one more time ===
Line 519: Line 578:
'''Simply repeat the whole section ''Copying over live data''.'''
'''Simply repeat the whole section ''Copying over live data''.'''


The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server.  For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.
The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server.  For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mariadb-dump}} command help to make sure of this.


So just repeat the steps from that section exactly one more time.
So just repeat the steps from that section exactly one more time.
Retrieved from "https://feministwiki.org/wiki/Special:MobileDiff/1336...1372"