Jump to content

FeministWiki:Technical documentation: Difference between revisions

(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page documents the FeministWiki's technical infrastructure, the target audience being technicians.
This page documents the FeministWiki's technical infrastructure, the target audience being technicians.  Note that while this page will help you understand the inner workings of the FeministWiki, it's not optimal for guiding you through the setup of a new server.  For that, see the page [[FW:Server setup]] instead.


== Hosts ==
== Hosts ==


The following table documents the basic DNS configuration, and can be used as part of the <code>/etc/hosts</code> file on each server to obviate the need for DNS lookups when connecting to one another.
The following table documents the basic DNS configuration, and can be used as part of the {{C|/etc/hosts}} file on each server to obviate the need for DNS lookups when connecting to one another.


{| class="wikitable"
{| class="wikitable"
!IP            !! FQDN                        !! Host          !! Purpose                !! Ports
!IP            !! FQDN                        !! Host          !! Purpose                !! Ports
|-
|-
|85.214.101.34 ||            feministwiki.org ||              || Wiki                    || 80, 443
|85.214.208.60 ||            feministwiki.org ||              || Wiki                    || 80, 443
|-
|-
|85.214.101.34 ||        www.feministwiki.org || www          || Wiki                    || 80, 443
|85.214.208.60 ||        www.feministwiki.org || www          || Wiki                    || 80, 443
|-
|-
|85.214.101.34 ||      ldap.feministwiki.org || ldap          || LDAP                    || -
|85.214.208.60 ||      ldap.feministwiki.org || ldap          || LDAP                    || -
|-
|-
|85.214.101.34 ||      blogs.feministwiki.org || blogs        || Blogging                || 80, 443
|85.214.208.60 ||      blogs.feministwiki.org || blogs        || Blogging                || 80, 443
|-
|-
|85.214.101.34 ||      chat.feministwiki.org || chat          || Web-client for XMPP    || 80, 443
|85.214.208.60 ||      chat.feministwiki.org || chat          || Web-client for XMPP    || 80, 443
|-
|-
|85.214.101.34 ||      forum.feministwiki.org || forum        || BBS Forum              || 80, 443
|85.214.208.60 ||      forum.feministwiki.org || forum        || BBS Forum              || 80, 443
|-
|-
|85.214.101.34 ||      mail.feministwiki.org || mail          || Web-client for Mail    || 80, 443
|85.214.208.60 ||      mail.feministwiki.org || mail          || Web-client for Mail    || 80, 443
|-
|-
|85.214.101.34 ||      files.feministwiki.org || files        || File storage            || 80, 443
|85.214.208.60 ||      files.feministwiki.org || files        || File storage            || 80, 443
|-
|-
|85.214.101.34 ||      imap.feministwiki.org || imap          || IMAP                    || 993
|85.214.208.60 ||      imap.feministwiki.org || imap          || IMAP                    || 993
|-
|-
|85.214.101.34 ||      pop3.feministwiki.org || pop3          || POP3                    || 995
|85.214.208.60 ||      pop3.feministwiki.org || pop3          || POP3                    || 995
|-
|-
|85.214.101.34 ||      smtp.feministwiki.org || smtp          || SMTP                    || 25, 465, 587
|85.214.208.60 ||      smtp.feministwiki.org || smtp          || SMTP                    || 25, 465, 587
|-
|-
|85.214.101.34 ||      xmpp.feministwiki.org || xmpp          || XMPP                    || 5222, 5269, 5280
|85.214.208.60 ||      xmpp.feministwiki.org || xmpp          || XMPP                    || 5222, 5223, 5269, 5270, 5443, 7777
|-
|-
|85.214.101.34 ||        irc.feministwiki.org || irc          || IRC                    || 6697
|85.214.208.60 ||        irc.feministwiki.org || irc          || IRC                    || 6697
|-
|-
|85.214.101.34 || add-member.feministwiki.org || add-member    || Add a member            || 80, 443
|85.214.208.60 ||   account.feministwiki.org || account      || Account operations      || 80, 443
|}
|}


As you can see, all services are on the same server for now.  However, it should be kept as an open possibility that the hosts are split across different IPs.  When done so, the <code>ldap</code> host should listen on 636 for LDAPS connections.
As you can see, all services are on the same server for now.  However, it should be kept as an open possibility that the hosts are split across different IPs.  When done so, the {{C|ldap}} host should listen on 636 for LDAPS connections.


== Firewall ==
There are no AAAA entries in the DNS because we only allow IPv4 for incoming connections.  This simplifies security auditing.
 
The simple <code>ufw</code> firewall-frontend is used to trivially limit all network I/O to the ports you can see in the host table above, plus port 22 for ssh and scp.


== Special DNS entries ==
== Special DNS entries ==
Line 68: Line 66:
|TXT  || _dmarc          || v=DMARC1; p=reject; rua=<nowiki>mailto:admin</nowiki>@feministwiki.org || DMARC
|TXT  || _dmarc          || v=DMARC1; p=reject; rua=<nowiki>mailto:admin</nowiki>@feministwiki.org || DMARC
|}
|}
'''Note:''' There must be a direct A or AAAA record (not a CNAME record) for the domain name specified in the MX record.


For XMPP:
For XMPP:
Line 77: Line 77:
|-
|-
|SRV  || _xmpp-server || _tcp    || @    || xmpp.feministwiki.org || 5269
|SRV  || _xmpp-server || _tcp    || @    || xmpp.feministwiki.org || 5269
|-
|SRV  || _xmpps-client || _tcp    || @    || xmpp.feministwiki.org || 5223
|-
|SRV  || _xmpps-server || _tcp    || @    || xmpp.feministwiki.org || 5270
|}
|}


Line 86: Line 90:
|TXT  || @              || google-site-verification=<key>
|TXT  || @              || google-site-verification=<key>
|}
|}
== Firewall ==
The simple {{C|ufw}} firewall-frontend is used to trivially limit all network I/O to the ports you can see in the host table above, plus port 22 for ssh and scp.
UFW adds IPv6 rules by default, which can be prevented by using more explicit rules.  Consider the following rule:
  ufw allow 12345/tcp  # will allow TCP connections to port 12345 via IPv4 and IPv6
To limit this to IPv4 you can instead use this:
  ufw allow proto tcp to 0.0.0.0/0 port 12345  # will allow TCP connections to port 12345 via IPv4 only
== Fail2ban ==
We use {{C|fail2ban}} to detect brute force attempts on some services.  The git repository for scripts and config contains the relevant Fail2ban configuration under {{C|etc/fail2ban}}.


== SSH access ==
== SSH access ==


FeministWiki hosts have ssh enabled for <code>root</code> access, but password login is disabled.  You must own a valid private key to log in.
FeministWiki hosts have ssh enabled for {{C|root}} access, but password login is disabled.  You must own a valid private key to log in.


== Git repo of scripts and configuration ==
== Git repo of scripts and configuration ==
Line 101: Line 121:
The FeministWiki uses LetsEncrypt to acquire digital certificates for encrypted communication.
The FeministWiki uses LetsEncrypt to acquire digital certificates for encrypted communication.


To ease use of the letsencrypt command, the file <code>/root/etc/domains</code> contains all the FQDNs used by the FeministWiki, one per line. Given that, the preferred way to populate the <code>/etc/letsencrypt/live/feministwiki.org</code> directory with fresh certs is to run the following commands while TCP port 80 is free (e.g. stop Apache first):
After certificates are generated with {{C|certbot}}, copies of them are put into {{C|/etc/fw-certs}}, and the group ownership and permissions of the {{C|privkey.pem}} and {{C|bundle.pem}} files are set such that any user who's in the {{C|ssl-cert}} group can read the private key and bundle.  (The others can be read by anyone anyway.)  A script in {{C|/etc/letsencrypt/renewal-hooks/post}} is responsible for taking care of this after automatic executions of certbot scheduled by the operating system.


  # Use $() to eliminate the terminating newline, if any.
The file {{C|/etc/fw-certs/bundle.pem}} is useful for programs that don't have the capability of reading a separate cert and key file; it combines the full certificate chain ({{C|fullchain.pem}}) with the private key in a single file.
  domains=$(cat /root/etc/domains)
 
  domains=$(printf '%s' "$domains" | tr '\n' ',')
 
  letsencrypt certonly --authenticator standalone --keep --expand -d "$domains"


Additionally, for programs that require a cert file and its private key in a single combined <code>.pem</code> file, run the following commands to generate such a file:
If you ever add a new domain under which the FeministWiki server will be reachable, add it as a line to the file {{C|/etc/feministwiki/domains}} and run the script {{C|/root/bin/letsencrypt-refresh}}. This script takes care of running {{C|certbot}} to refresh the cert files, and populating the {{C|/etc/fw-certs}} directory with updated files.


cd /etc/letsencrypt/live/feministwiki.org
Note that whenever you run the {{C|letsencrypt-refresh}} script, it will momentarily stop the web serverThis means you're causing a short outage of the web-based services of the FeministWiki whenever you run the script.
  cat fullchain.pem privkey.pem > certbundle.pem


The script <code>/root/bin/letsencrypt-refresh</code> '''does all of the above''', so in practice you just need to run the following commands to recreate the cert:
== Ubuntu package repositories ==


  service apache2 stop  # Assuming Apache is running on the machine
Currently, FeministWiki runs on Ubuntu 16.04 LTS (Xenial), which has rather old Apache and PHP packages. We use the Ubuntu PPAs {{C|ondrej/apache2}} and {{C|ondrej/php}} to get newer versions.
letsencrypt-refresh
service apache2 start


The above can be used not only to refresh a cert that's running out, but also to add a new domain to the cert.  Just add the domain to <code>/root/etc/domains</code> and run the commands.
For F-Droid packages, we use the {{C|fdroid/fdroidserver}} Ubuntu PPA.
 
Note that the letsencrypt command may not work well on a "dumb" terminal such as an Emacs shell buffer.  If in doubt, run it from within a proper terminal emulator.
 
=== Readability of the key files ===
 
To ensure that processes running under unprivileged users can read key files, ensure that the users they run under are members of the <code>ssl-cert</code> group, which should have read access to the files in <code>/etc/letsencrypt/live/feministwiki.org</code>.


== Services ==
== Services ==
Line 138: Line 144:
Software: OpenLDAP
Software: OpenLDAP


The LDAP service contains the central database of FeministWiki members.  The structure looks like this:
The LDAP service contains the central database of FeministWiki members.  For details on the LDAP schema, see [[FeministWiki:LDAP Schema]].


* dc=feministwiki,dc=org
=== Wiki ===
** ou=members
*** cn=''username'' <br/> objectClass: inetOrgPerson <br/> cn: ''username'' <br/> uid: ''username'' <br/> sn: - <br/> userPassword: {SSHA}''saltedhash'' <br/> mail: ''username''@feministwiki.org
*** cn=''username2'' <br/> objectClass: inetOrgPerson <br/> cn: ''username2'' <br/> uid: ''username2'' <br/> sn: - <br/> userPassword: {SSHA}''saltedhash2'' <br/> mail: ''username2''@feministwiki.org <br/> manager: cn=''username'',ou=members,dc=feministwiki,dc=org
*** ...
** ou=groups
*** cn=members <br/> objectClass: groupOfNames <br/> cn: members <br/> member: ''username'' <br/> member: ''username2'' <br/> member: ...


Notes:
Host: feministwiki.org, www.feministwiki.org, fem.wiki, feministwiki.de, www.feministwiki.de <br/>
* The <code>cn</code> (common name) and <code>uid</code> (user ID) fields both contain the username. This is because some software is preconfigured to look at <code>uid</code>, while most look at <code>cn</code>.
Software: MediaWiki
* The <code>sn</code> (surname) field simply contains a minus character as a placeholder, because it's a mandatory field.
* The <code>manager</code> field is optional and we use it to record the member who added the member in question.
 
To make sure passwords are stored with the <code>{SSHA}</code> scheme rather than plain text, the <code>ppolicy</code> "password policy overlay" is used.  ZYTRAX has a very nice book about LDAP which documents how to enable this: http://www.zytrax.com/books/ldap/ch6/ppolicy.html
 
In short, the steps go as follows (these commands ''should'' work verbatim):
 
# Add the ppolicy schema
ldapadd -Y external -H ldapi:/// < /etc/ldap/schema/ppolicy.ldif
# Enable the ppolicy dynamic module
ldapmodify -Y external -H ldapi:/// <<EOF
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: ppolicy
EOF
# Add the ppolicy overlay with olcPPolicyHashCleartext set to TRUE
ldapadd -Y external -H ldapi:/// <<EOF
dn: olcOverlay=ppolicy,olcDatabase={1}mdb,cn=config
objectClass: olcPPolicyConfig
olcOverlay: ppolicy
olcPPolicyHashCleartext: TRUE
EOF


=== Wiki ===
The wiki uses a [https://www.mediawiki.org/wiki/MediaWiki MediaWiki] installation located at {{C|/var/www/wiki/w}}.  It uses the [https://www.mediawiki.org/wiki/LDAP_Stack LDAP Stack] extension for login management, and the "Short URL" feature is enabled.  The wiki uses the SQL database called "feministwiki" and the SQL user of the same name.


Host: feministwiki.org, www.feministwiki.org <br/>
The default wiki is in English. Parallel wiki installations for different languages are supported via a combination of Apache's URL rewriting, and conditional branches in the {{C|LocalSettings.php}} file of the MediaWiki installation:
Software: MediaWiki


The wiki uses a [https://www.mediawiki.org/wiki/MediaWiki MediaWiki] installation located at <code>/var/www/wiki/w</code>. It uses the [https://www.mediawiki.org/wiki/LDAP_Stack LDAP Stack] for login management, and the "Short URL" feature enabled with help of Apache's vhost configuration.
* The rewrite rules ensure that, for any supported language {{C|xy}}, URLs beginning with {{C|/xy/wiki/...}} internally resolve to {{C|/w/index.php}} and URLs beginning with {{C|/xy/w/...}} resolve to {{C|/w/...}}.
* The LocalSettings file checks {{C|$_SERVER['REQUEST_URI']}} to determine the language code prefix in the requested URL, and configures things accordingly:
** It sets {{C|$wgArticlePath}} to {{C|/xy/wiki/$1}} so that {{C|/w/index.php}} knows that when it's invoked via such a URL, it should serve an article.
** It sets {{C|$wgScriptPath}} to {{C|/xy/w}} so that links to scripts served by MediaWiki are correct for the language.
** It sets the SQL database name to {{C|feministwiki_xy}}.
** It configures a "foreign" image database via the [https://www.mediawiki.org/wiki/Manual:$wgForeignFileRepos#Using_files_from_a_database_that_you_can_access_:_ForeignDBRepo,_ForeignDBViaLBRepo ForeignDBRepo] method so that the media upload database of the default wiki is used by all, instead of every parallel wiki having its own media database.


The wiki uses the SQL database called "feministwiki" and the SQL user of the same name.
The domain names feministwiki.de and www.feministwiki.de redirect to {{C|https://feministwiki.org/de}}.  For instance, requesting {{C|https://feministwiki.de/wiki/Hauptseite}} will result in an HTTP redirect to {{C|https://feministwiki.org/de/wiki/Hauptseite}}.


=== Blogs ===
=== Blogs ===
Line 190: Line 169:
Software: WordPress (multisite)
Software: WordPress (multisite)


This is an installation of WordPress in <code>/var/www/blogs</code>, with the "multisite network" feature enabled on a path-basis, so users can have their own blogs on URLs like <code>blogs.feministwiki.org/janedoe</code>.  LDAP authentication is enabled via the AD/LDAP plugin from miniOrange.
This is an installation of WordPress in {{C|/var/www/blogs}}, with the "multisite network" feature enabled on a path-basis, so users can have their own blogs on URLs like {{C|blogs.feministwiki.org/janedoe}}.  LDAP authentication is enabled via the AD/LDAP plugin from miniOrange.


Users from LDAP who log in for the first time are automatically registered as "Subscriber" accounts, and the admin can change their WordPress role "Author" to allow publishing.
Users from LDAP who log in for the first time are automatically registered as "Subscriber" accounts, and the admin can change their WordPress role "Author" to allow publishing.
Line 201: Line 180:
Software: Converse.js
Software: Converse.js


The web-interface for the FeministChat uses the full-screen "Impress" variant of the [https://conversejs.org/ Converse.js] XMPP client.  The hosted HTML and JS files are located at <code>/var/www/chat</code>, although they load Converse.js as an external script from upstream, which is why the self-hosted HTML and JS are very minimal.
The web-interface for the FeministChat uses the full-screen "Impress" variant of the [https://conversejs.org/ Converse.js] XMPP client.  The hosted HTML and JS files are located at {{C|/var/www/chat}}, although they load Converse.js as an external script from upstream, which is why the self-hosted HTML and JS are very minimal.


=== Forum ===
=== Forum ===
Line 208: Line 187:
Software: phpBB
Software: phpBB


The forum uses a [https://www.phpbb.com/ phpBB] installation located at <code>/var/www/forum</code>.  Most configuration of phpBB, including LDAP authentication, is done through its administration panel.  The style used by the forum is essentially Basic Orange, though the logo is changed via an inheriting style called FeministWiki.
The forum uses a [https://www.phpbb.com/ phpBB] installation located at {{C|/var/www/forum}}.  Most configuration of phpBB, including LDAP authentication, is done through its administration panel.  The style used by the forum is a minimally changed "ProSilver Dark". The forum uses the SQL database called "feministforum" and the SQL user of the same name.
 
The forum uses the SQL database called "feministforum" and the SQL user of the same name.


=== Mail (web interface) ===
=== Mail (web interface) ===
Line 217: Line 194:
Software: Roundcube
Software: Roundcube


The web-interface for the FeministMail uses the [https://roundcube.net/ Roundcube] mail client, installed at <code>/var/www/mail</code>.  It uses the standard "larry" style, but with some tweaks to logos and images.
The web-interface for the FeministMail uses the [https://roundcube.net/ Roundcube] mail client, installed at {{C|/var/www/mail}}.  It uses a FeministWiki-branded modification of the new "elastic" style.


=== Files ===
=== Files ===
Line 224: Line 201:
Software: Nextcloud
Software: Nextcloud


FeministFiles is a [https://nextcloud.com/ Nextcloud] installation with some branding, and LDAP authentication, installed at <code>/var/www/files</code>.
FeministFiles is a [https://nextcloud.com/ Nextcloud] installation with some branding, and LDAP authentication, installed at {{C|/var/www/files}}.


=== IMAP ===
=== IMAP ===
Line 231: Line 208:
Software: Dovecot
Software: Dovecot


FeministMail uses the [https://www.dovecot.org/ Dovecot] IMAP server, configured for LDAP authentication and using virtual mail boxes under <code>/home/vmail</code>.
FeministMail uses the [https://www.dovecot.org/ Dovecot] IMAP server, configured for LDAP authentication and using virtual mail boxes under {{C|/home/vmail}}.


=== POP3 ===
=== POP3 ===
Line 245: Line 222:
Software: Postfix, OpenDKIM
Software: Postfix, OpenDKIM


FeministMail uses the [http://www.postfix.org/ Postfix] SMTP server, using SASL authentication through Dovecot, LDAP-based virtual mail boxes under <code>/home/vmail</code>, and DKIM signing via OpenDKIM.  Send a mail to a Gmail account and use the "Show original" feature of Gmail to see if the mail passes SPF, DKIM, and DMARC tests.
FeministMail uses the [http://www.postfix.org/ Postfix] SMTP server, using SASL authentication through Dovecot, LDAP-based virtual mail boxes under {{C|/home/vmail}}, and DKIM signing via OpenDKIM.  Send a mail to a Gmail account and use the "Show original" feature of Gmail to see if the mail passes SPF, DKIM, and DMARC tests.


There are also various tools on the web to automatically test the DNS settings for correctness, to check if the domain/IP is on blacklists, etc., which you can find via Google.  All in all, FeministMail is probably the most complicated service of the FeministWiki, as far as technical background goes.
There are also various tools on the web to automatically test the DNS settings for correctness, to check if the domain/IP is on blacklists, etc., which you can find via Google.  All in all, FeministMail is probably the most complicated service of the FeministWiki, as far as technical background goes.
=== Mailing lists ===
Email domain: lists.feministwiki.org <br/>
Software: GNU Mailman, Postfix
Postfix is configured to recognize {{C|lists.feministwiki.org}} as a "local" domain.  This means it uses the file specified in the {{C|alias_maps}} configuration directive (typically {{C|/etc/aliases}}) to decide the final recipient of an e-mail sent to this domain.  Correspondingly, we populate {{C|/etc/aliases}} with the aliases needed by Mailman to operate each mailing list it controls.
Note that there is no DNS entry for {{C|lists.feministwiki.org}} because e-mail software just checks the MX record for {{C|feministwiki.org}} when the recipient is from the domain {{C|<em>anything</em>.feministwiki.org}}.


=== XMPP ===
=== XMPP ===
Line 261: Line 247:
Software: InspIRCd
Software: InspIRCd


FeministIRC uses the [http://www.inspircd.org/ InspIRCd] IRC server with the <code>ldapauth</code> module for LDAP authentication.
FeministIRC uses the [http://www.inspircd.org/ InspIRCd] IRC server with the {{C|ldapauth}} module for LDAP authentication.  The client is authenticated via the combination of the NICK and PASS provided upon connection, which must correspond to the FeministWiki credentials.


=== Add a member ===
=== Account operations ===


Host: add-member.feministwiki.org <br/>
Host: account.feministwiki.org <br/>
Software: custom
Software: custom


The page to add a new member, hosted at <code>/var/www/add-member</code>, uses a bit of self-written HTML, PHP, and a setuid-root C program to invoke the shell script located at <code>/root/bin/fw-adduser</code> with root privileges.
This custom web interface hosted at {{C|/var/www/account}} lets you complete several tasks related to FeministWiki membership, such as changing your account settings, resetting your password, or adding a new member.  It also contains a form to request membership.  The interface is written in HTML, CSS, PHP, and C.