FeministWiki:Technical documentation: Difference between revisions

    From FeministWiki
    (31 intermediate revisions by 2 users not shown)
    Line 6: Line 6:


    {| class="wikitable"
    {| class="wikitable"
    !IP            !! FQDN                        !! Host          !! Purpose
    !IP            !! FQDN                        !! Host          !! Purpose                 !! Ports
    |-
    |-
    |85.214.101.34 ||            feministwiki.org ||              || Wiki
    |85.214.101.34 ||            feministwiki.org ||              || Wiki                   || 80, 443
    |-
    |-
    |85.214.101.34 ||        www.feministwiki.org || www          || Wiki
    |85.214.101.34 ||        www.feministwiki.org || www          || Wiki                   || 80, 443
    |-
    |-
    |85.214.101.34 ||      ldap.feministwiki.org || ldap          || LDAP
    |85.214.101.34 ||      ldap.feministwiki.org || ldap          || LDAP                   || -
    |-
    |-
    |85.214.101.34 ||       chat.feministwiki.org || chat          || Web-client for XMPP
    |85.214.101.34 ||     blogs.feministwiki.org || blogs        || Blogging                || 80, 443
    |-
    |-
    |85.214.101.34 ||     forum.feministwiki.org || forum        || BBS Forum
    |85.214.101.34 ||       chat.feministwiki.org || chat          || Web-client for XMPP    || 80, 443
    |-
    |-
    |85.214.101.34 ||       mail.feministwiki.org || mail          || Web-client for Mail
    |85.214.101.34 ||     forum.feministwiki.org || forum        || BBS Forum              || 80, 443
    |-
    |-
    |85.214.101.34 ||     files.feministwiki.org || files        || File storage
    |85.214.101.34 ||       mail.feministwiki.org || mail          || Web-client for Mail    || 80, 443
    |-
    |-
    |85.214.101.34 ||       imap.feministwiki.org || imap          || IMAP
    |85.214.101.34 ||     files.feministwiki.org || files        || File storage            || 80, 443
    |-
    |-
    |85.214.101.34 ||      smtp.feministwiki.org || smtp         || SMTP
    |85.214.101.34 ||      imap.feministwiki.org || imap         || IMAP                    || 993
    |-
    |-
    |85.214.101.34 ||      xmpp.feministwiki.org || xmpp         || XMPP
    |85.214.101.34 ||      pop3.feministwiki.org || pop3         || POP3                    || 995
    |-
    |-
    |85.214.101.34 ||       irc.feministwiki.org || irc          || IRC
    |85.214.101.34 ||       smtp.feministwiki.org || smtp          || SMTP                    || 25, 465, 587
    |-
    |-
    |85.214.101.34 ||     social.feministwiki.org || social        || GNU social
    |85.214.101.34 ||       xmpp.feministwiki.org || xmpp          || XMPP                    || 5222, 5269, 5280
    |-
    |-
    |85.214.101.34 || add-member.feministwiki.org || add-member   || Add a member
    |85.214.101.34 ||       irc.feministwiki.org || irc          || IRC                    || 6697
    |-
    |85.214.101.34 ||   account.feministwiki.org || account      || Account operations      || 80, 443
    |}
    |}


    (As you can see, all services are on the same server for now.)
    As you can see, all services are on the same server for now. However, it should be kept as an open possibility that the hosts are split across different IPs.  When done so, the <code>ldap</code> host should listen on 636 for LDAPS connections.
     
    == Firewall ==
     
    The simple <code>ufw</code> firewall-frontend is used to trivially limit all network I/O to the ports you can see in the host table above, plus port 22 for ssh and scp.


    == Special DNS entries ==
    == Special DNS entries ==
    Line 61: Line 67:
    |-
    |-
    |TXT  || _dmarc          || v=DMARC1; p=reject; rua=<nowiki>mailto:admin</nowiki>@feministwiki.org || DMARC
    |TXT  || _dmarc          || v=DMARC1; p=reject; rua=<nowiki>mailto:admin</nowiki>@feministwiki.org || DMARC
    |-
    |TXT  || @              || google-site-verification=<key>                                        || Google Site Verification
    |}
    |}
    '''Note:''' There must be a direct A or AAAA record (not a CNAME record) for the domain name specified in the MX record.


    For XMPP:
    For XMPP:
    Line 73: Line 79:
    |-
    |-
    |SRV  || _xmpp-server || _tcp    || @    || xmpp.feministwiki.org || 5269
    |SRV  || _xmpp-server || _tcp    || @    || xmpp.feministwiki.org || 5269
    |}
    Google Site Verification:
    {| class="wikitable"
    !Type !! Host            !! Data
    |-
    |TXT  || @              || google-site-verification=<key>
    |}
    |}


    Line 89: Line 103:
    The FeministWiki uses LetsEncrypt to acquire digital certificates for encrypted communication.
    The FeministWiki uses LetsEncrypt to acquire digital certificates for encrypted communication.


    To ease use of the letsencrypt command, the file <code>/root/etc/domains</code> contains all the FQDNs used by the FeministWiki, one per line.  Given that, the preferred way to populate the <code>/etc/letsencrypt/live/feministwiki.org</code> directory with fresh certs is to run the following command while TCP port 80 is free (e.g. stop Apache first):
    To ease use of the letsencrypt command, the file <code>/root/etc/domains</code> contains all the FQDNs used by the FeministWiki, one per line.  Given that, the preferred way to populate the <code>/etc/letsencrypt/live/feministwiki.org</code> directory with fresh certs is to run the following commands while TCP port 80 is free (e.g. stop Apache first):


    letsencrypt certonly --authenticator standalone -d "$(tr '\n' ',' < /root/etc/domains)"
      # Use $() to eliminate the terminating newline, if any.
      domains=$(cat /root/etc/domains)
     
      domains=$(printf '%s' "$domains" | tr '\n' ',')
     
      letsencrypt certonly --authenticator standalone --keep --expand -d "$domains"


    Additionally, for programs that require a cert file and its private key in a single combined <code>.pem</code> file, run the following commands to generate such a file:
    Additionally, for programs that require a cert file and its private key in a single combined <code>.pem</code> file, run the following commands to generate such a file:
    Line 98: Line 117:
      cat fullchain.pem privkey.pem > certbundle.pem
      cat fullchain.pem privkey.pem > certbundle.pem


    The script <code>/root/bin/letsencrypt-refresh</code> '''does all of the above''', so in practice you just need to run the following commands to recreate the cert:
    The script <code>/root/bin/letsencrypt-refresh</code> '''does all of the above''' automatically.  It stops Apache, runs the commands described above, and starts Apache again.  This can be used not only to refresh a cert that's running out, but also to add a new domain to the cert.  Just add the domain to <code>/root/etc/domains</code> and run the commands.  Note that the letsencrypt command may not work well on a "dumb" terminal such as an Emacs shell buffer.  If in doubt, run it from within a proper terminal emulator.


    service apache2 stop  # Assuming Apache is running on the machine
    === Readability of the key files ===
    letsencrypt-refresh
    service apache2 start


    The above can be used not only to refresh a cert that's running out, but also to add a new domain to the cert.  Just add the domain to <code>/root/etc/domains</code> and run the commands.
    To ensure that processes running under unprivileged users can read key files, ensure that the users they run under are members of the <code>ssl-cert</code> group, which should have read access to the files in <code>/etc/letsencrypt/live/feministwiki.org</code>.


    Note that the letsencrypt command doesn't work well on a "dumb" terminal such as an Emacs shell buffer.  Make sure to run it from within a proper terminal emulator.
    == Ubuntu package repositories ==


    === Readability of the key files ===
    Currently, FeministWiki runs on Ubuntu 16.04 LTS (Xenial), which has rather old Apache and PHP packages.  We use the Ubuntu PPAs <code>ondrej/apache2</code> and <code>ondrej/php</code> to get newer versions.


    To ensure that processes running under unprivileged users can read key files, ensure that the users they run under are members of the <code>ssl-cert</code> group, which should have read access to the files in <code>/etc/letsencrypt/live/feministwiki.org</code>.
    For F-Droid packages, we use the <code>fdroid/fdroidserver</code> Ubuntu PPA.


    == Services ==
    == Services ==
    Line 121: Line 138:
    Software: OpenLDAP
    Software: OpenLDAP


    The LDAP service contains the central database of FeministWiki members.  The structure looks like this:
    The LDAP service contains the central database of FeministWiki members.  For details on the LDAP schema, see [[FeministWiki:LDAP Schema]].


    * dc=feministwiki,dc=org
    === Wiki ===
    ** ou=members
    *** cn=''username'' <br/> objectClass: inetOrgPerson <br/> cn: ''username'' <br/> uid: ''username'' <br/> sn: - <br/> userPassword: {SSHA}''saltedhash'' <br/> mail: ''username''@feministwiki.org
    *** cn=''username2'' <br/> objectClass: inetOrgPerson <br/> cn: ''username2'' <br/> uid: ''username2'' <br/> sn: - <br/> userPassword: {SSHA}''saltedhash2'' <br/> mail: ''username2''@feministwiki.org <br/> manager: cn=''username'',ou=members,dc=feministwiki,dc=org
    *** ...
    ** ou=groups
    *** cn=members <br/> objectClass: groupOfNames <br/> cn: members <br/> member: ''username'' <br/> member: ''username2'' <br/> member: ...


    Notes:
    Host: feministwiki.org, www.feministwiki.org <br/>
    * The <code>cn</code> (common name) and <code>uid</code> (user ID) fields both contain the username. This is because some software is preconfigured to look at <code>uid</code>, while most look at <code>cn</code>.
    Software: MediaWiki
    * The <code>sn</code> (surname) field simply contains a minus character as a placeholder, because it's a mandatory field.
    * The <code>manager</code> field is optional and we use it to record the member who added the member in question.


    To make sure passwords are stored with the <code>{SSHA}</code> scheme rather than plain text, the <code>ppolicy</code> "password policy overlay" is usedZYTRAX has a very nice book about LDAP which documents how to enable this: http://www.zytrax.com/books/ldap/ch6/ppolicy.html
    The wiki uses a [https://www.mediawiki.org/wiki/MediaWiki MediaWiki] installation located at <code>/var/www/wiki/w</code>.  It uses the [https://www.mediawiki.org/wiki/LDAP_Stack LDAP Stack] for login management, and the "Short URL" feature enabled with help of Apache's vhost configuration.


    In short, the steps go as follows (these commands ''should'' work verbatim):
    The wiki uses the SQL database called "feministwiki" and the SQL user of the same name.


    # Add the ppolicy schema
    === Blogs ===
    ldapadd -Y external -H ldapi:/// < /etc/ldap/schema/ppolicy.ldif
    # Enable the ppolicy dynamic module
    ldapmodify -Y external -H ldapi:/// <<EOF
    dn: cn=module{0},cn=config
    changetype: modify
    add: olcModuleLoad
    olcModuleLoad: ppolicy
    EOF
    # Add the ppolicy overlay with olcPPolicyHashCleartext set to TRUE
    ldapadd -Y external -H ldapi:/// <<EOF
    dn: olcOverlay=ppolicy,olcDatabase={1}mdb,cn=config
    objectClass: olcPPolicyConfig
    olcOverlay: ppolicy
    olcPPolicyHashCleartext: TRUE
    EOF


    === Wiki ===
    Host: blogs.feministwiki.org <br/>
    Software: WordPress (multisite)


    Host: feministwiki.org, www.feministwiki.org <br/>
    This is an installation of WordPress in <code>/var/www/blogs</code>, with the "multisite network" feature enabled on a path-basis, so users can have their own blogs on URLs like <code>blogs.feministwiki.org/janedoe</code>.  LDAP authentication is enabled via the AD/LDAP plugin from miniOrange.
    Software: MediaWiki


    The wiki uses a [https://www.mediawiki.org/wiki/MediaWiki MediaWiki] installation located at <code>/var/www/wiki/w</code> with the [https://www.mediawiki.org/wiki/Extension:LDAP_Authentication LDAP Authentication] plugin for login management and the "Short URL" feature enabled with help of Apache's vhost configuration, which is located at <code>/etc/apache2/sites-available/wiki.conf</code>.
    Users from LDAP who log in for the first time are automatically registered as "Subscriber" accounts, and the admin can change their WordPress role "Author" to allow publishing.


    The wiki uses the SQL database called "feministwiki" and the SQL user of the same name.
    This WordPress installation uses the SQL database called "blogs" and an SQL user of the same name.


    === Chat (web interface) ===
    === Chat (web interface) ===
    Line 204: Line 196:


    FeministMail uses the [https://www.dovecot.org/ Dovecot] IMAP server, configured for LDAP authentication and using virtual mail boxes under <code>/home/vmail</code>.
    FeministMail uses the [https://www.dovecot.org/ Dovecot] IMAP server, configured for LDAP authentication and using virtual mail boxes under <code>/home/vmail</code>.
    === POP3 ===
    Host: pop3.feministwiki.org <br/>
    Software: Dovecot
    While Dovecot is primarily an IMAP server, it also offers POP3 support, which the FeministWiki installation has enabled.


    === SMTP ===
    === SMTP ===
    Line 213: Line 212:


    There are also various tools on the web to automatically test the DNS settings for correctness, to check if the domain/IP is on blacklists, etc., which you can find via Google.  All in all, FeministMail is probably the most complicated service of the FeministWiki, as far as technical background goes.
    There are also various tools on the web to automatically test the DNS settings for correctness, to check if the domain/IP is on blacklists, etc., which you can find via Google.  All in all, FeministMail is probably the most complicated service of the FeministWiki, as far as technical background goes.
    === Mailing lists ===
    Email domain: lists.feministwiki.org <br/>
    Software: GNU Mailman, Postfix
    Postfix is configured to recognize <code>lists.feministwiki.org</code> as a "local" domain.  This means it uses the file specified in the <code>alias_maps</code> configuration directive (typically <code>/etc/aliases</code>) to decide the final recipient of an e-mail sent to this domain.  Correspondingly, we populate <code>/etc/aliases</code> with the aliases needed by Mailman to operate each mailing list it controls.
    Note that there is no DNS entry for <code>lists.feministwiki.org</code> because e-mail software just checks the MX record for <code>feministwiki.org</code> when the recipient is from the domain <code><em>anything</em>.feministwiki.org</code>.


    === XMPP ===
    === XMPP ===
    Line 228: Line 236:
    FeministIRC uses the [http://www.inspircd.org/ InspIRCd] IRC server with the <code>ldapauth</code> module for LDAP authentication.
    FeministIRC uses the [http://www.inspircd.org/ InspIRCd] IRC server with the <code>ldapauth</code> module for LDAP authentication.


    === Add a member ===
    === Account operations ===


    Host: add-member.feministwiki.org <br/>
    Host: account.feministwiki.org <br/>
    Software: custom
    Software: custom


    The page to add a new member, hosted at <code>/var/www/add-member</code>, uses a bit of self-written HTML, PHP, and a setuid-root C program to invoke the shell script located at <code>/root/bin/fw-adduser</code> with root privileges.
    This custom web interface hosted at <code>/var/www/account</code> lets you complete several tasks related to FeministWiki membership, such as changing your account settings, resetting your password, or adding a new member.  It also contains a form to request membership.  The interface is written in HTML, CSS, PHP, and C.

    Revision as of 20:15, 21 December 2019

    This page documents the FeministWiki's technical infrastructure, the target audience being technicians.

    Hosts

    The following table documents the basic DNS configuration, and can be used as part of the /etc/hosts file on each server to obviate the need for DNS lookups when connecting to one another.

    IP FQDN Host Purpose Ports
    85.214.101.34 feministwiki.org Wiki 80, 443
    85.214.101.34 www.feministwiki.org www Wiki 80, 443
    85.214.101.34 ldap.feministwiki.org ldap LDAP -
    85.214.101.34 blogs.feministwiki.org blogs Blogging 80, 443
    85.214.101.34 chat.feministwiki.org chat Web-client for XMPP 80, 443
    85.214.101.34 forum.feministwiki.org forum BBS Forum 80, 443
    85.214.101.34 mail.feministwiki.org mail Web-client for Mail 80, 443
    85.214.101.34 files.feministwiki.org files File storage 80, 443
    85.214.101.34 imap.feministwiki.org imap IMAP 993
    85.214.101.34 pop3.feministwiki.org pop3 POP3 995
    85.214.101.34 smtp.feministwiki.org smtp SMTP 25, 465, 587
    85.214.101.34 xmpp.feministwiki.org xmpp XMPP 5222, 5269, 5280
    85.214.101.34 irc.feministwiki.org irc IRC 6697
    85.214.101.34 account.feministwiki.org account Account operations 80, 443

    As you can see, all services are on the same server for now. However, it should be kept as an open possibility that the hosts are split across different IPs. When done so, the ldap host should listen on 636 for LDAPS connections.

    Firewall

    The simple ufw firewall-frontend is used to trivially limit all network I/O to the ports you can see in the host table above, plus port 22 for ssh and scp.

    Special DNS entries

    For CAA:

    Type Name Flag Tag Value
    CAA @ 0 issue letsencrypt.org
    CAA @ 0 iodef admin@feministwiki.org

    For email:

    Type Host Data Purpose
    MX @ smtp.feministwiki.org Mail server
    TXT @ v=spf1 mx -all SPF
    TXT mail._domainkey v=DKIM1; k=rsa; p=<pubkey> DKIM
    TXT _dmarc v=DMARC1; p=reject; rua=mailto:admin@feministwiki.org DMARC

    Note: There must be a direct A or AAAA record (not a CNAME record) for the domain name specified in the MX record.

    For XMPP:

    Type Service Protocol Name Destination Port
    SRV _xmpp-client _tcp @ xmpp.feministwiki.org 5222
    SRV _xmpp-server _tcp @ xmpp.feministwiki.org 5269

    Google Site Verification:

    Type Host Data
    TXT @ google-site-verification=<key>

    SSH access

    FeministWiki hosts have ssh enabled for root access, but password login is disabled. You must own a valid private key to log in.

    Git repo of scripts and configuration

    The following GitHub account hosts repositories with scripts and configuration used by the FeministWiki:

    https://github.com/FeministWiki

    Certs

    The FeministWiki uses LetsEncrypt to acquire digital certificates for encrypted communication.

    To ease use of the letsencrypt command, the file /root/etc/domains contains all the FQDNs used by the FeministWiki, one per line. Given that, the preferred way to populate the /etc/letsencrypt/live/feministwiki.org directory with fresh certs is to run the following commands while TCP port 80 is free (e.g. stop Apache first):

     # Use $() to eliminate the terminating newline, if any.
     domains=$(cat /root/etc/domains)
     
     domains=$(printf '%s' "$domains" | tr '\n' ',')
     
     letsencrypt certonly --authenticator standalone --keep --expand -d "$domains"
    

    Additionally, for programs that require a cert file and its private key in a single combined .pem file, run the following commands to generate such a file:

    cd /etc/letsencrypt/live/feministwiki.org
    cat fullchain.pem privkey.pem > certbundle.pem
    

    The script /root/bin/letsencrypt-refresh does all of the above automatically. It stops Apache, runs the commands described above, and starts Apache again. This can be used not only to refresh a cert that's running out, but also to add a new domain to the cert. Just add the domain to /root/etc/domains and run the commands. Note that the letsencrypt command may not work well on a "dumb" terminal such as an Emacs shell buffer. If in doubt, run it from within a proper terminal emulator.

    Readability of the key files

    To ensure that processes running under unprivileged users can read key files, ensure that the users they run under are members of the ssl-cert group, which should have read access to the files in /etc/letsencrypt/live/feministwiki.org.

    Ubuntu package repositories

    Currently, FeministWiki runs on Ubuntu 16.04 LTS (Xenial), which has rather old Apache and PHP packages. We use the Ubuntu PPAs ondrej/apache2 and ondrej/php to get newer versions.

    For F-Droid packages, we use the fdroid/fdroidserver Ubuntu PPA.

    Services

    This section documents the individual services of the FeministWiki. They should work regardless of what server they're on. I.e. every service could in theory be hosted on its own server.

    LDAP

    Host: ldap.feministwiki.org
    Software: OpenLDAP

    The LDAP service contains the central database of FeministWiki members. For details on the LDAP schema, see FeministWiki:LDAP Schema.

    Wiki

    Host: feministwiki.org, www.feministwiki.org
    Software: MediaWiki

    The wiki uses a MediaWiki installation located at /var/www/wiki/w. It uses the LDAP Stack for login management, and the "Short URL" feature enabled with help of Apache's vhost configuration.

    The wiki uses the SQL database called "feministwiki" and the SQL user of the same name.

    Blogs

    Host: blogs.feministwiki.org
    Software: WordPress (multisite)

    This is an installation of WordPress in /var/www/blogs, with the "multisite network" feature enabled on a path-basis, so users can have their own blogs on URLs like blogs.feministwiki.org/janedoe. LDAP authentication is enabled via the AD/LDAP plugin from miniOrange.

    Users from LDAP who log in for the first time are automatically registered as "Subscriber" accounts, and the admin can change their WordPress role "Author" to allow publishing.

    This WordPress installation uses the SQL database called "blogs" and an SQL user of the same name.

    Chat (web interface)

    Host: chat.feministwiki.org
    Software: Converse.js

    The web-interface for the FeministChat uses the full-screen "Impress" variant of the Converse.js XMPP client. The hosted HTML and JS files are located at /var/www/chat, although they load Converse.js as an external script from upstream, which is why the self-hosted HTML and JS are very minimal.

    Forum

    Host: forum.feministwiki.org
    Software: phpBB

    The forum uses a phpBB installation located at /var/www/forum. Most configuration of phpBB, including LDAP authentication, is done through its administration panel. The style used by the forum is essentially Basic Orange, though the logo is changed via an inheriting style called FeministWiki.

    The forum uses the SQL database called "feministforum" and the SQL user of the same name.

    Mail (web interface)

    Host: mail.feministwiki.org
    Software: Roundcube

    The web-interface for the FeministMail uses the Roundcube mail client, installed at /var/www/mail. It uses the standard "larry" style, but with some tweaks to logos and images.

    Files

    Host: files.feministwiki.org
    Software: Nextcloud

    FeministFiles is a Nextcloud installation with some branding, and LDAP authentication, installed at /var/www/files.

    IMAP

    Host: imap.feministwiki.org
    Software: Dovecot

    FeministMail uses the Dovecot IMAP server, configured for LDAP authentication and using virtual mail boxes under /home/vmail.

    POP3

    Host: pop3.feministwiki.org
    Software: Dovecot

    While Dovecot is primarily an IMAP server, it also offers POP3 support, which the FeministWiki installation has enabled.

    SMTP

    Host: smtp.feministwiki.org
    Software: Postfix, OpenDKIM

    FeministMail uses the Postfix SMTP server, using SASL authentication through Dovecot, LDAP-based virtual mail boxes under /home/vmail, and DKIM signing via OpenDKIM. Send a mail to a Gmail account and use the "Show original" feature of Gmail to see if the mail passes SPF, DKIM, and DMARC tests.

    There are also various tools on the web to automatically test the DNS settings for correctness, to check if the domain/IP is on blacklists, etc., which you can find via Google. All in all, FeministMail is probably the most complicated service of the FeministWiki, as far as technical background goes.

    Mailing lists

    Email domain: lists.feministwiki.org
    Software: GNU Mailman, Postfix

    Postfix is configured to recognize lists.feministwiki.org as a "local" domain. This means it uses the file specified in the alias_maps configuration directive (typically /etc/aliases) to decide the final recipient of an e-mail sent to this domain. Correspondingly, we populate /etc/aliases with the aliases needed by Mailman to operate each mailing list it controls.

    Note that there is no DNS entry for lists.feministwiki.org because e-mail software just checks the MX record for feministwiki.org when the recipient is from the domain anything.feministwiki.org.

    XMPP

    Host: xmpp.feministwiki.org
    Software: ejabberd

    FeministChat uses the ejabberd XMPP server, configured to use LDAP authentication and an LDAP-based shared roster group for all members.

    IRC

    Host: irc.feministwiki.org
    Software: InspIRCd

    FeministIRC uses the InspIRCd IRC server with the ldapauth module for LDAP authentication.

    Account operations

    Host: account.feministwiki.org
    Software: custom

    This custom web interface hosted at /var/www/account lets you complete several tasks related to FeministWiki membership, such as changing your account settings, resetting your password, or adding a new member. It also contains a form to request membership. The interface is written in HTML, CSS, PHP, and C.