FeministWiki - Contributions [fr]

Accueil

2024-11-13T11:58:11Z

Technician : /* Who's behind the project? */

{{PageSeo
|description = Welcome to the FeministWiki, a digital platform for feminism consisting of a wiki, forum, chat, blog hosting, and more.
}}
''Services: [https://blogs.feministwiki.org FeministBlog] - [https://files.feministwiki.org/ FeministFiles] - [https://mail.feministwiki.org/ FeministMail] - [https://forum.feministwiki.org/ FeministForum] - [https://chat.feministwiki.org/ FeministChat] - [[Help:IRC|FeministIRC]]''

Welcome to '''FeministWiki''', a wiki and integrated digital platform for feminism.

A wiki is a knowledge-base like an encyclopedia, but managed by the public. The FeministWiki specializes on feminism, and is managed by feminists and their supporters. It aims to archive and present information relevant to feminism, and explain feminist perspectives to the reader.

Further, the FeministWiki aims to offer a complete integrated digital platform for feminists, with components such as a blogging platform, a discussion forum, a messaging/chat system, private and shared online storage for large files, and more. You don't need to provide any personally identifying information to become a member, nor do you need to pay; the modest number of users makes the platform cheap to run.

There are a few ways to become a member:
* Use the [https://account.feministwiki.org/register.html Registration Request] form to ask for an account.
* Contact an existing member and have them create an account for you by using the [https://account.feministwiki.org/add-member.html Add Member] form.
* Ask for an account by manually contacting technician@feministwiki.org or by messaging an official social media account of the FeministWiki.

Spinster: [https://spinster.xyz/@FeministWiki @FeministWiki] (preferred) 
Twitter: [https://twitter.com/@FeministWiki @FeministWiki] 
Facebook: [https://www.facebook.com/FeministWiki/ FeministWiki]

Once you are a member, you will be given a username and password with which you can log in to all FeministWiki services. For example, to log in to the wiki itself (which you are reading right now), use the small "Log in" link at the top right of the page.

See the '''[[FW:Welcome|Welcome page]]''' for further information on the FeministWiki, written with new members in mind.

Heads up: '''the FeministWiki needs you'''. All of the technical infrastructure of the FeministWiki is only useful if there's a community making use of it, and content on the wiki doesn't write itself! Be bold, don't shy off of asking for membership, and let the community and the world benefit from your added knowledge. You can become a member even if you have no intention to contribute to the actual wiki; feel free to chat away with other members, discuss matters important to you on the forum, or use the file storage to have a central place to store your favorite information-material on feminism.

== Check out some articles ==

=== Notable feminists ===

* [[Magdalen Berns]]

=== Sex, gender, and trans activism ===

* [[Gender]]
* [[Transgender ideology]]
* [[Cisgender]]
* [[Autogynephilia]]
* [[TERF]]
* [[Transwomen in women's sports]]

=== Sex industry ===

* [[Nordic Model]]

=== Black feminism ===

* [[Black feminism]]

== What is feminism? ==

There are a variety of ideological groupings which call themselves feminism, and some of them are in contradiction with each other. As such, one cannot support all types of feminism at the same time. The FeministWiki is for people who adhere to a relatively straightforward and classical interpretation of feminism: the liberation of female people from male supremacism. This is sometimes called ''radical feminism'' because male supremacism is a radical notion for many people, and its elimination requires radical changes to society.

Male supremacism refers to social and political systems that use stereotypes, myths, discrimination, belittlement, violence, and other means to keep men in power and women oppressed. Women are then exploited for reproductive and domestic labor, men's sexual gratification, and more. Male supremacism also causes collateral damage to some men and boys. For instance, effeminate boys and gay men are often targeted with punishment for failing to uphold the myth that men are inherently masculine.

The FeministWiki promotes second-wave feminist literature:

* [https://radfem.org/ Radical Feminist Archives]

The FeministWiki is in full support of the Women's Human Rights Campaign:

* [https://www.womensdeclaration.com/ Declaration on Women's Sex-Based Rights]

The FeministWiki recommends Spinster as a feminism-friendly social media platform:

* [https://spinster.xyz Spinster.xyz]

Further, the FeministWiki promotes and stands in solidarity with the following groups and organizations:

* [http://lgballiance.org.uk LGB Alliance]: The LGB Alliance
* [http://womensliberationfront.org/ WoLF]: The Women's Liberation Front
* [https://feministcurrent.com/ Feminist Current]: Canadian feminist news, commentary, and podcasts
* [https://www.rapereliefshelter.bc.ca/ Vancouver Rape Relief]: Female-only rape-relief and women's shelter
* [https://nordicmodelnow.org/ Nordic Model Now]: Educational movement for the abolition of prostitution
* [http://www.spaceintl.org/ SPACE International]: Survivors of Prostitution Abuse Calling for Enlightenment
* [https://womansplaceuk.org/ Woman's Place UK]: Women's campaigning group scrutinizing gender self-identification
* [https://speakupforwomen.nz/ Speak Up for Women]: New Zealand group opposing gender self-identification
* [https://feministstruggle.org/ Feminists in Struggle - FIST]: US-based female-only radical feminist network
* [https://pussychurchofmodernwitchcraft.com/ The Pussy Church of Modern Witchcraft]: Lesbian-led Church for Women and Girls

== Who's behind the project? ==

The FeministWiki platform is offered to the public for free by the German LLC '''WikiWiz UG (haftungsbeschränkt)'''.

For legal inquiries please contact info@feministwiki.org or feministwiki@gmail.com.



[[de:Hauptseite]] [[en:Main_Page]] [[es:Página_principal]] [[it:Pagina_principale]] [[pt:Página_principal]]

Accueil

2022-08-30T09:04:48Z

Technician :

{{PageSeo
|description = Welcome to the FeministWiki, a digital platform for feminism consisting of a wiki, forum, chat, blog hosting, and more.
}}
''Services: [https://blogs.feministwiki.org FeministBlog] - [https://files.feministwiki.org/ FeministFiles] - [https://mail.feministwiki.org/ FeministMail] - [https://forum.feministwiki.org/ FeministForum] - [https://chat.feministwiki.org/ FeministChat] - [[Help:IRC|FeministIRC]]''

Welcome to '''FeministWiki''', a wiki and integrated digital platform for feminism.

A wiki is a knowledge-base like an encyclopedia, but managed by the public. The FeministWiki specializes on feminism, and is managed by feminists and their supporters. It aims to archive and present information relevant to feminism, and explain feminist perspectives to the reader.

Further, the FeministWiki aims to offer a complete integrated digital platform for feminists, with components such as a blogging platform, a discussion forum, a messaging/chat system, private and shared online storage for large files, and more. You don't need to provide any personally identifying information to become a member, nor do you need to pay; the modest number of users makes the platform cheap to run.

There are a few ways to become a member:
* Use the [https://account.feministwiki.org/register.html Registration Request] form to ask for an account.
* Contact an existing member and have them create an account for you by using the [https://account.feministwiki.org/add-member.html Add Member] form.
* Ask for an account by manually contacting technician@feministwiki.org or by messaging an official social media account of the FeministWiki.

Spinster: [https://spinster.xyz/@FeministWiki @FeministWiki] (preferred) 
Twitter: [https://twitter.com/@FeministWiki @FeministWiki] 
Facebook: [https://www.facebook.com/FeministWiki/ FeministWiki]

Once you are a member, you will be given a username and password with which you can log in to all FeministWiki services. For example, to log in to the wiki itself (which you are reading right now), use the small "Log in" link at the top right of the page.

See the '''[[FW:Welcome|Welcome page]]''' for further information on the FeministWiki, written with new members in mind.

Heads up: '''the FeministWiki needs you'''. All of the technical infrastructure of the FeministWiki is only useful if there's a community making use of it, and content on the wiki doesn't write itself! Be bold, don't shy off of asking for membership, and let the community and the world benefit from your added knowledge. You can become a member even if you have no intention to contribute to the actual wiki; feel free to chat away with other members, discuss matters important to you on the forum, or use the file storage to have a central place to store your favorite information-material on feminism.

== Check out some articles ==

=== Notable feminists ===

* [[Magdalen Berns]]

=== Sex, gender, and trans activism ===

* [[Gender]]
* [[Transgender ideology]]
* [[Cisgender]]
* [[Autogynephilia]]
* [[TERF]]
* [[Transwomen in women's sports]]

=== Sex industry ===

* [[Nordic Model]]

=== Black feminism ===

* [[Black feminism]]

== What is feminism? ==

There are a variety of ideological groupings which call themselves feminism, and some of them are in contradiction with each other. As such, one cannot support all types of feminism at the same time. The FeministWiki is for people who adhere to a relatively straightforward and classical interpretation of feminism: the liberation of female people from male supremacism. This is sometimes called ''radical feminism'' because male supremacism is a radical notion for many people, and its elimination requires radical changes to society.

Male supremacism refers to social and political systems that use stereotypes, myths, discrimination, belittlement, violence, and other means to keep men in power and women oppressed. Women are then exploited for reproductive and domestic labor, men's sexual gratification, and more. Male supremacism also causes collateral damage to some men and boys. For instance, effeminate boys and gay men are often targeted with punishment for failing to uphold the myth that men are inherently masculine.

The FeministWiki promotes second-wave feminist literature:

* [https://radfem.org/ Radical Feminist Archives]

The FeministWiki is in full support of the Women's Human Rights Campaign:

* [https://www.womensdeclaration.com/ Declaration on Women's Sex-Based Rights]

The FeministWiki recommends Spinster as a feminism-friendly social media platform:

* [https://spinster.xyz Spinster.xyz]

Further, the FeministWiki promotes and stands in solidarity with the following groups and organizations:

* [http://lgballiance.org.uk LGB Alliance]: The LGB Alliance
* [http://womensliberationfront.org/ WoLF]: The Women's Liberation Front
* [https://feministcurrent.com/ Feminist Current]: Canadian feminist news, commentary, and podcasts
* [https://www.rapereliefshelter.bc.ca/ Vancouver Rape Relief]: Female-only rape-relief and women's shelter
* [https://nordicmodelnow.org/ Nordic Model Now]: Educational movement for the abolition of prostitution
* [http://www.spaceintl.org/ SPACE International]: Survivors of Prostitution Abuse Calling for Enlightenment
* [https://womansplaceuk.org/ Woman's Place UK]: Women's campaigning group scrutinizing gender self-identification
* [https://speakupforwomen.nz/ Speak Up for Women]: New Zealand group opposing gender self-identification
* [https://feministstruggle.org/ Feminists in Struggle - FIST]: US-based female-only radical feminist network
* [https://pussychurchofmodernwitchcraft.com/ The Pussy Church of Modern Witchcraft]: Lesbian-led Church for Women and Girls

== Who's behind the project? ==

The FeministWiki platform is offered to the public by the German non-profit organization '''FeministWiki gemeinnützige UG (haftungsbeschränkt)'''.

The FeministWiki g. UG is bound by German law to abide by the following mission statement (translated from German):

<blockquote>
1. The company exclusively and directly pursues charitable purposes within the meaning of the section "tax-privileged purposes" of the tax code (AO). The purpose of the company is to promote equality between women and men.

2. The purpose of the statutes is achieved by operating an online platform called feministwiki.org. The platform serves as an information center on the subject of feminism, as well as a communication platform for people who want to stand up for the rights of women.
</blockquote>

The technical infrastructure is managed by [[FW:Technician|the technician]], who also offers user support. For support, please contact technician@feministwiki.org.

For legal inquiries please contact info@feministwiki.org or feministwiki@gmail.com.



[[de:Hauptseite]] [[en:Main_Page]] [[es:Página_principal]] [[it:Pagina_principale]] [[pt:Página_principal]]

Accueil

2022-08-30T07:19:09Z

Technician : Technician a déplacé la page Main Page vers Accueil sans laisser de redirection

{{PageSeo
|description = Welcome to the FeministWiki, a digital platform for feminism consisting of a wiki, forum, chat, blog hosting, and more.
}}
''Services: [https://blogs.feministwiki.org FeministBlog] - [https://files.feministwiki.org/ FeministFiles] - [https://mail.feministwiki.org/ FeministMail] - [https://forum.feministwiki.org/ FeministForum] - [https://chat.feministwiki.org/ FeministChat] - [[Help:IRC|FeministIRC]]''

Welcome to '''FeministWiki''', a wiki and integrated digital platform for feminism.

A wiki is a knowledge-base like an encyclopedia, but managed by the public. The FeministWiki specializes on feminism, and is managed by feminists and their supporters. It aims to archive and present information relevant to feminism, and explain feminist perspectives to the reader.

Further, the FeministWiki aims to offer a complete integrated digital platform for feminists, with components such as a blogging platform, a discussion forum, a messaging/chat system, private and shared online storage for large files, and more. You don't need to provide any personally identifying information to become a member, nor do you need to pay; the modest number of users makes the platform cheap to run.

There are a few ways to become a member:
* Use the [https://account.feministwiki.org/register.html Registration Request] form to ask for an account.
* Contact an existing member and have them create an account for you by using the [https://account.feministwiki.org/add-member.html Add Member] form.
* Ask for an account by manually contacting technician@feministwiki.org or by messaging an official social media account of the FeministWiki.

Spinster: [https://spinster.xyz/@FeministWiki @FeministWiki] (preferred) 
Twitter: [https://twitter.com/@FeministWiki @FeministWiki] 
Facebook: [https://www.facebook.com/FeministWiki/ FeministWiki]

Once you are a member, you will be given a username and password with which you can log in to all FeministWiki services. For example, to log in to the wiki itself (which you are reading right now), use the small "Log in" link at the top right of the page.

See the '''[[FW:Welcome|Welcome page]]''' for further information on the FeministWiki, written with new members in mind.

Heads up: '''the FeministWiki needs you'''. All of the technical infrastructure of the FeministWiki is only useful if there's a community making use of it, and content on the wiki doesn't write itself! Be bold, don't shy off of asking for membership, and let the community and the world benefit from your added knowledge. You can become a member even if you have no intention to contribute to the actual wiki; feel free to chat away with other members, discuss matters important to you on the forum, or use the file storage to have a central place to store your favorite information-material on feminism.

== Check out some articles ==

=== Notable feminists ===

* [[Magdalen Berns]]

=== Sex, gender, and trans activism ===

* [[Gender]]
* [[Transgender ideology]]
* [[Cisgender]]
* [[Autogynephilia]]
* [[TERF]]
* [[Transwomen in women's sports]]

=== Sex industry ===

* [[Nordic Model]]

=== Black feminism ===

* [[Black feminism]]

== What is feminism? ==

There are a variety of ideological groupings which call themselves feminism, and some of them are in contradiction with each other. As such, one cannot support all types of feminism at the same time. The FeministWiki is for people who adhere to a relatively straightforward and classical interpretation of feminism: the liberation of female people from male supremacism. This is sometimes called ''radical feminism'' because male supremacism is a radical notion for many people, and its elimination requires radical changes to society.

Male supremacism refers to social and political systems that use stereotypes, myths, discrimination, belittlement, violence, and other means to keep men in power and women oppressed. Women are then exploited for reproductive and domestic labor, men's sexual gratification, and more. Male supremacism also causes collateral damage to some men and boys. For instance, effeminate boys and gay men are often targeted with punishment for failing to uphold the myth that men are inherently masculine.

The FeministWiki promotes second-wave feminist literature:

* [https://radfem.org/ Radical Feminist Archives]

The FeministWiki is in full support of the Women's Human Rights Campaign:

* [https://www.womensdeclaration.com/ Declaration on Women's Sex-Based Rights]

The FeministWiki recommends Spinster as a feminism-friendly social media platform:

* [https://spinster.xyz Spinster.xyz]

Further, the FeministWiki promotes and stands in solidarity with the following groups and organizations:

* [http://lgballiance.org.uk LGB Alliance]: The LGB Alliance
* [http://womensliberationfront.org/ WoLF]: The Women's Liberation Front
* [https://feministcurrent.com/ Feminist Current]: Canadian feminist news, commentary, and podcasts
* [https://www.rapereliefshelter.bc.ca/ Vancouver Rape Relief]: Female-only rape-relief and women's shelter
* [https://nordicmodelnow.org/ Nordic Model Now]: Educational movement for the abolition of prostitution
* [http://www.spaceintl.org/ SPACE International]: Survivors of Prostitution Abuse Calling for Enlightenment
* [https://womansplaceuk.org/ Woman's Place UK]: Women's campaigning group scrutinizing gender self-identification
* [https://speakupforwomen.nz/ Speak Up for Women]: New Zealand group opposing gender self-identification
* [https://feministstruggle.org/ Feminists in Struggle - FIST]: US-based female-only radical feminist network
* [https://pussychurchofmodernwitchcraft.com/ The Pussy Church of Modern Witchcraft]: Lesbian-led Church for Women and Girls

== Who's behind the project? ==

The FeministWiki platform is offered to the public by the German non-profit organization '''FeministWiki gemeinnützige UG (haftungsbeschränkt)'''.

The FeministWiki g. UG is bound by German law to abide by the following mission statement (translated from German):

<blockquote>
1. The company exclusively and directly pursues charitable purposes within the meaning of the section "tax-privileged purposes" of the tax code (AO). The purpose of the company is to promote equality between women and men.

2. The purpose of the statutes is achieved by operating an online platform called feministwiki.org. The platform serves as an information center on the subject of feminism, as well as a communication platform for people who want to stand up for the rights of women.
</blockquote>

The technical infrastructure is managed by [[FW:Technician|the technician]], who also offers user support. For support, please contact technician@feministwiki.org.

For legal inquiries please contact info@feministwiki.org or feministwiki@gmail.com.


[[de:Hauptseite]] [[pt:Página_principal]]

FeministWiki:Privacy policy

2022-06-19T13:03:00Z

Technician : /* Activity data */

= Privacy Policy =

The entity responsible in the context of data protection laws, in particular the EU General Data Protection Regulation (GDPR), is:

<pre>
FeministWiki gemeinnützige UG (haftungsbeschränkt)
Richard-Wagner-Str. 25
DE-35321 Laubach

District court Giessen, HRB 10100

Legal correspondent: Taylan Ulrich Kammer

Email: info@feministwiki.org
</pre>

=== Your rights as a data subject ===

You can exercise the following rights at any time using the contact details of our data protection officer:

* Information about your data stored by us and their processing (Art. 15 GDPR),
* Correction of incorrect personal data (Art. 16 GDPR),
* Deletion of your data stored by us (Art. 17 GDPR),
* Restriction of data processing if we are not yet allowed to delete your data due to legal obligations (Art. 18 GDPR),
* Objection to the processing of your data by us (Art. 21 GDPR) and
* Data portability, provided you have consented to the data processing or have concluded a contract with us (Art. 20 GDPR).

If you have given us your consent, you can revoke it at any time with effect for the future.

You can contact a supervisory authority with a complaint at any time, e.g. to the responsible supervisory authority of the federal state of your place of residence or to the authority responsible for us as the responsible body.

A list of the supervisory authorities (for the non-public area) with addresses can be found at: https://www.bfdi.bund.de/DE/Infothek/Anschriften_Links/anschriften_links-node.html.

=== Changes to our privacy policy ===

We reserve the right to adapt this data protection declaration so that it always complies with current legal requirements or to implement changes to our services in the data protection declaration, e.g. when introducing new services. The new data protection declaration will then apply to your next visit.

=== Embedded content from other websites ===

Articles on this website may refer to other content (links).

These websites can collect data about you, use cookies, embed additional tracking services from third parties and your interaction with the embedded content, if you have an account and are logged in to this website.

=== Liability for content ===

The articles that can be called up on the website are for general information only and not for advice in specific cases. We endeavor to ensure that all information and data contained on the website are correct and up to date in accordance with Section 7 (1) of the German Telemedia Act. For the correctness, completeness, up-to-dateness or quality of the information and data provided, no guarantee is given according to Sections 8 to 10 of the Act. Liability for the content of the information that can be accessed is excluded unless it is intentional or grossly negligent incorrect information. Obligations to remove or block the use of information under general law remain unaffected. Liability in this regard is only possible from the point in time at which we become aware of a specific legal violation. As soon as we become aware of such legal violations, we will remove this content immediately.

=== Liability for links ===

We are not responsible for the content of websites that can be reached via a hyperlink. The operators of the linked pages are solely responsible for their content. We expressly do not adopt the content of these Internet pages as our own and can therefore not guarantee the correctness, completeness and availability of the content. When the link was first established, we checked the third-party content to see whether it might trigger civil or criminal liability. However, we are not obliged to constantly check the content to which we refer in our offer for changes that could give rise to new responsibility. Only if we discover or are informed by others that a specific offer to which we have provided a link triggers civil or criminal liability, we will remove the reference to this offer, insofar as this is technically possible and reasonable for us.

=== Copyright ===

The content and works on this website created by the operator of this website are subject to German copyright law. All contributions from third parties are marked as such. The duplication, processing, distribution and any kind of exploitation outside the limits of copyright require the written consent of the respective author or creator. Copies of these pages are only permitted for private use, but not for commercial purposes.

=== Data protection ===

As a rule, our website can be used without providing personal data. Insofar as personal data (e.g. name, address or email address) is collected on our website, this is always done on a voluntary basis as far as possible. These data will not be passed on to third parties without your express consent.

In connection with your access, data is temporarily stored in our server for the purpose of data security, which may allow identification (website visited, time of access, amount of data sent in bytes, source / reference from which you came to the page, browser used, operating system used, visible IP address). The data collected are only used for statistical evaluations. However, we reserve the right to check server log files retrospectively if there are concrete indications of illegal use. An evaluation, with the exception of statistical purposes in anonymous form, does not take place.

= In lay terms =

This section does not constitute a legal privacy policy. This is an honest summary of everything you might want to know about the FeministWiki and privacy, explained in lay terms instead of legalese.

Data stored by the FeministWiki could be split in two categories: data stored about your account, and data generated by your usage of FeministWiki services like opening pages, editing pages, using the chat system, or uploading files.

== Account data ==

At its core, the FeministWiki doesn't save anything about you other than your chosen username, a "cryptographic hash" of your current password, and the username of the member who added you. In addition, it can save an e-mail address and a "display name" of your choosing, if you enter these in the account settings page or provide them in the account request form. A detailed listing of all data stored in relation to your FeministWiki account follows.

=== The username ===

Your username doesn't need to correspond to your real identity in any way. If it does, note that other members can see it in some places such as the list of names in the chat service. It's almost impossible to keep 100% of malicious people from getting a FeministWiki membership, so a malicious person could end up seeing your username too. Furthermore, if you edit the wiki, post on public parts of the forum, publish a FeministWiki blog post etc., then your username will be publicly visible. If this is a problem for you, please contact the technician to change your username to one that doesn't relate to your real identity.

=== The password ===

Your password is not saved in plain text. Rather, a "salted SHA1 hash" of your password is saved. In layperson terms, this means: even in case of a data leak, attackers won't immediately know your password, though it's technically possible for them to figure it out if they spend a lot of time processing the leaked data. For this reason, the password you use here should not be a very important one, such as the password you use for online banking. (This issue applies to almost all websites that use passwords; the FeministWiki is not any less secure in this regard than other websites.) All FeministWiki services use encrypted communication, so your password doesn't travel over the network in plain text either.

=== The member who added you ===

The username of the FeministWiki member who created your account is visible to the [[FW:Technician|technician]], if she/he cares to look it up from the internally kept database.

=== Your regular e-mail address ===

In the account settings, you can set an e-mail address that should be associated with your FeministWiki account instead of the default address of ''(username)@feministwiki.org''. (In the account request form, this corresponds to the address you provide in the first e-mail input field.) While this address won't be listed publicly anywhere, it's possible that other FeministWiki members may see it. Given that it's practically impossible to keep out 100% of malicious persons from getting a FeministWiki member account, you should consider the risk that a malicious person will see this e-mail address. As such, consider not providing one and just using the ''(username)@feministwiki.org'' address provided by the FeministWiki, or providing one that cannot be traced to your real identity, if keeping your identity hidden is important to you.

=== Your recovery e-mail address ===

Also in the account settings, you can provide a secondary, hidden e-mail address that will be used solely for account recovery in case you forget your FeministWiki password. This address is only visible to the [[FW:Technician|technician]]. However, despite state of the art security measures, please note that a data leak can never be ruled out 100%, so if keeping your identity secret is very important to you, consider leaving this empty and instead taking good care of your password, or using an address that can't be traced to your real identity.

=== Data provided in the account request form ===

If you use the account registration/request form, if you leave out the primary e-mail address field, you are asked to fill out a secondary e-mail field so your account request can be responded to. While this address is not recorded in the member database (unless you explicitly opt in), it will appear in the automatic e-mail sent to admin@feministwiki.org. Likewise with the text you write in the "personal declaration" field of the form. The e-mail containing this data is stored on the mail server of the FeministWiki, meaning that the same data leak concerns as explained in the previous section might apply. That said, the technician will try to make sure that these e-mails are deleted after the account request is processed. (So far, a technical guarantee of this is not provided; this will come in a future rework of the account request system.)

== Activity data ==

When you use any of the FeministWiki services, like visiting any part of the website, editing wiki pages, posting to the forum, sending chat messages, sending FeministWiki e-mail, uploading files, or writing or commenting on blog posts, you generate data that is stored on the server, some of which is also publicly shown. Details follow.

=== Visiting pages ===

Every new page of the FeministWiki that you open or navigate to (including forum pages, blog posts, the on-site chat or email clients, etc.) generates an "access log" entry on the web server. These entries contain your IP address, the time of access, and optional information sent by your web browser. This helps with alleviating abuse of the website, and searching for technical problems.

Note: This is a standard feature of web servers and is done by most websites you visit on the Internet.

Usually, your IP address cannot easily be traced back to your real identity. However, it reveals information about your Internet Service Provider and an approximate geographic location (such as the nearest large city). If this is an issue for you, please look into software such as the [https://www.torproject.org/ Tor Project], a VPN Provider, or other ways to hide your IP address from websites you visit.

The FeministWiki will never reveal the contents of the access logs to anyone, unless mandated by law enforcement.

=== Editing the wiki ===

All individual edits made to the wiki (including talk pages and other types of special pages) are permanently recorded, with the username of the editor and the date and time of the edit. These records remain even if the page is later edited by someone else so thoroughly that none of the content written by you remains. This recording of all edits helps to resolve issues with vandalism by malicious editors. (E.g. if someone removes all content on a page and replaces it with garbage, the original content can be recovered in a few clicks, and the edit logs will show who did the vandalism.)

Note: This is a standard feature of the wiki software ''MediaWiki'' that is also used by Wikipedia.

=== Posting on the forum ===

Most sections of the forum are publicly visible. Your username will appear aside your forum post, as well as the date and time of the post. There are sections of the forum that are only visible to members, but please remember that it's difficult to keep out 100% of malicious persons from getting a FeministWiki account. All forum posts are also stored on the server, and data leaks '''may''' happen despite state of the art security measures.

=== Using the chat system ===

Chat messages sent through the FeministWiki chat service are only visible to the recipient(s) of the message. However, they are stored on the server to provide you your chat history when you log in to the chat with a different device. Ideally, don't ever send personal information through the FeministWiki chat service if you want it to remain absolutely secret.

Furthermore, any device you use to log in to the chat system leads to an access log entry to be stored by the software. This helps identifying the cause of technical problems, like when someone is having difficulties logging in.

=== Writing blog posts or comments ===

The blog posts as well as the comments beneath them are publicly visible, and have your username attached to them, as well as the date and time of the comment, much like public posts on the forum.

=== Uploading files to the file storage system ===

The files you upload to the FeministWiki file storage are private by default. You have the option of sharing them with others by creating a sharing link or making files or whole folders accessible to other members. Since the files are stored on the server however, you should ideally never upload any files with personal information that you want to keep absolutely secret.

=== Sending and receiving FeministWiki e-mail ===

The e-mails you send and receive with your ''(username)@feministwiki.org'' account are stored on the server. Ideally, don't send any e-mail that provides personal information which you want to keep absolutely secret.

Furthermore, any e-mail client software you've configured to automatically fetch your received FeministWiki e-mail leads to access log entries to be stored by the e-mail server whenever the software in question opens a connection to look for any incoming mail. This helps with identifying technical problems like when someone is having difficulty receiving their FeministWiki e-mail, and is a standard feature of all e-mail servers, meaning your regular e-mail provider is doing it as well.

== Contact for further questions ==

Please contact technician@feministwiki.org if you have any privacy related concerns or questions that aren't answered above.

TERF

2022-06-03T21:51:49Z

Technician : /* Trans activist Dana Rivers commits triple homicide */ Not sure if adopted, one of them could be the biological mother.

{{PageSeo
|description = TERF is a slur used by transgender activists to debase anyone who criticizes their movement on the basis of feminist concerns.
|image = Sonicfox.png
}}
[[File:Terf-slur-02.png|thumb|300px|A typical tweet containing "terf"]]

The word ''TERF'' (or ''terf''; pl ''terfs'' or ''terves'') is a slur that is used predominantly by transgender activists and their allies against people who criticize the transgender movement on the basis of feminist concerns. Since the slur is used for people with feminist concerns, the main target tend to be women. As such, it's usually understood to be an anti-feminist, sexist and misogynist slur.

The word was invented as an acronym for ''Trans-Exclusionary Radical Feminist'', where the "trans-exclusionary" part referred to those holding roughly the position that transwomen should not be included under a feminist definition of womanhood, and the "radical feminist" part was meant neutrally, i.e. for people who would indeed describe themselves as [[Radical feminism|radical feminists]] in the true sense.

Over time, the acronym pretty much became a [[Wikipedia:Four-letter word|four-letter word]]. Nowadays the capitalization is frequently omitted, and the already ambiguous original meaning ignored entirely. Still, users of the term tend to claim that it's a neutral description. The "trans-exclusionary" part may now refer to anyone who thinks transwomen should not have unfettered access to all female-only spaces (e.g. changing rooms), should not partake in women's sports where they have [[Transwomen in women's sports|unfair advantages]], should not be considered a natural part of the lesbian dating pool, etc. Although most members of the public would see these as rather sensible positions (especially when considering that a "transwoman" may have intact male anatomy), transgender activists nevertheless see all of these types of "exclusion" as unacceptable.

A closely associated term is ''SWERF'', which is supposed to stand for ''Sex-Worker-Exclusionary Radical Feminist'' and is used for those who see the sex industry (prostitution, pornography, etc.) as highly exploitative and sexist. Like ''TERF'', the term is almost always applied as a slur, and to misrepresent the political position of the person it's used against. Ironically, some of those who have to face the term most commonly are women who worked in prostitution and became anti-prostitution activists as a result of their own experiences as so-called sex workers.

== History and evolution towards hate speech ==

=== Origin ===

The oldest known use of the term is by Viv Smythe aka "tigtog" in a blog post from 2008.<ref name=terf-origin/> She defended the term as late as 2018, in an article for The Guardian.<ref name=guardian-smythe/> Transgender activists frequently try to defend the term on the grounds that Viv Smythe is a woman who herself claims to be a radical feminist, and seems to have first used the term in a way that is not derogatory. Of course, the benign origins of a term does not mean that it cannot evolve into a slur.

=== Early years and first principled critique ===

The evolution of the term from 2008 into the early to mid 2010s is not well documented. Mostly, feminists had to face the term on social media, where it began to be used regularly to debase their position. In July 2014, [https://www.feministcurrent.com/ Feminist Current] published two articles referencing the term. The first, written by C. K. Egbert and titled ''Defending the 'TERF': Gender as political'', explains and defends in length the political theory underlining the ideas supported by feminists who are slurred as "terf."<ref name=fc-egbert/> The second, written by [[Sarah Ditum]] and titled ''How 'TERF' works'', shortly analyses a situation in which a woman is pressured to retract a statement opposing violence against women, on the grounds that the statement originally stems from a feminist who is considered a "terf".<ref name=fc-ditum/> Since Feminist Current is highly acclaimed among radical-leaning feminists, its decision to support the women slurred with "terf" could be seen as a turning point.

In August 2014, Vice published an article titled ''I Am Now Officially a Transphobic Twitter Troll'' (subtitle: ''At least according to the 'Block Bot' I am'') by author Martin Robbins.<ref name=vice-robbins/> In the article, Robbins talks about how the "Block Bot" project on Twitter, which is supposed to help people avoid abusive trolls, has included feminist authors and journalists such as [[Caroline Criado-Perez]] and [[Helen Lewis]] among the people who should be blocked. Ironically, Lewis seems to have made it to the list for complaining about abusive trolls, as the "evidence" for the reason to ban her includes objections to tweets such as "kill TERFS", "burn TERFS", or hateful jokes such as "what's better than 1 dead terf? 2 dead terfs."

Another Feminist Current article defending those targeted with the slur was published in November 2015, written by [[Penny White]] and titled ''Why I no longer hate ‘TERFs’''.<ref name=fc-white/> In the article, White explains how she herself used to be convinced that so-called "TERFs" are worthy of contempt, but changed her mind after starting to look closer into the issue. This experience seems to resonate with many women and some socially liberal men to this day, who start out being supportive of the transgender movement, only to start becoming skeptical after negative experiences and observations, ultimately leading them to be also labeled "terf" and shunned by transgender activists and their allies. After that, Feminist Current started publishing articles critical of the transgender movement with some frequency, much to the enragement of transgender activists.

=== The mainstreaming of hatred ===

[[File:Mya-byrne-punches-terfs.jpg|thumb|left|150px|Totally not a slur!]]

In June 2017, transgender activist [[Wikipedia:Mya Byrne|Mya Byrne]] came to the San Francisco Pride Parade with a t-shirt reading "I PUNCH TERFS", decorated with a large fake blood-stain. Byrne uploaded a selfie of him wearing the t-shirt at the parade, captioned "This is what gay liberation looks like #pride #yesallterfs" which sparked many negative reactions.<ref name=fc-tra-violence/> The t-shirt would later be displayed at an "art exhibit" at the San Francisco Public Library, set up by the trans activist group ''The Degenderettes''. After complaints, the library removed the t-shirt from the exhibition, though similar items showcasing a violent mentality remained, such as baseball bats wrapped in barbed wire and painted in the colors of the transgender pride flag.<ref name=fc-tra-violence/>

[[File:Sonicfox.png|thumb|right|250px|Dominique McLean's tweet glorifying violence against women.]]

In April 2019, professional video gamer [[Wikipedia:SonicFox|Dominique McLean]] aka "SonicFox" uploaded a video on Twitter, captioned "what I do to terfs," in which a male video game character strikes a female character's neck so hard that her skin comes off, while the camera shows her agony-filled face. McLean can be heard yelling "terf!" into his microphone with each blow dealt to the female character, reminiscent of a lynching.<ref name=sonicfox/> The tweet garnered hundreds of thousands of views, and many thousands of retweets and likes.

McLean's tweet is made even worse by the fact that the hatred is not just directed at an imagined "terf" character, but at the voice actress behind the character, [[Wikipedia:Ronda Rousey|Ronda Rousey]]. Rousey, who is an MMA (mixed martial arts) fighter, is deemed a "TERF" for having stated, with blunt wording, that it's unfair for male MMA fighter [[Transwomen in women's sports#Fallon_Fox|Fallon Fox]] to compete against women.<ref name=rousey-fox/><ref name=rousey-marysue/><ref name=rousey-reddit/> A year after Rousey's remarks, female MMA fighter Tamikka Brents had suffered a concussion and an orbital bone fracture during a fight with Fallon Fox,<ref name=fallon-fox/> but this does not seem to have changed the opinion of trans activists.

After McLean's tweet was reported for hate speech, Twitter initially decided that it [[:File:Sonicfox2.png|did not break their rules]]. Only after continued protest and many more reports did Twitter react, by removing the tweet and issuing SonicFox a mere 24-hour ban.

{{clear}}

<gallery caption="Aftermath of McLean's tweet" widths=300px heights=150px>
File:Sonicfox3.jpg | McLean defending his tweet on the grounds that Ronda Rousey is a "TERF" and as such deserving of the violent contempt
File:Sonicfox-copycat1.png | A copycat tweet showing another video game character dying brutally, with the caption "FUCK TERFS"
File:Sonicfox-copycat2.png | Another copycat tweet. The GIF is of [[Wikipedia:Mortal Kombat 11|Mortal Kombat 11]] character [[Wikipedia:Scorpion (Mortal Kombat)|Scorpion]] burning his opponent alive.
</gallery>

=== New trend: Threaten women with your face attached ===

In late 2019, a social media trend dubbed "POV you're a TERF in my mentions" started, in which trans activists would pose with a weapon (baseball bat, sword, machete, or the like), sometimes preparing to strike or showing them mid-strike, taken as a [[Wikipedia:Point-of-view shot|point-of-view (POV) shot]] and captioned with the phrase "POV you're a TERF in my mentions" or variations thereof.<ref name=pov-terf/> The pictures are supposed to represent the point-of-view (POV) of a "TERF" being assaulted by the depicted person. In other words, women called "TERF" who look at the pictures are supposed to imagine themselves being violently assaulted.

<gallery caption="Some examples of the trend" widths=300px heights=150px>
File:Pov-terf-1.webp
File:Pov-terf-2.webp
File:Pov-terf-3.webp
File:Pov-terf-4.webp
File:Pov-terf-5.webp
File:Pov-terf-6.webp
</gallery>

=== Political justification of physical violence ===

In May 2020, a blog post was published on Medium.com equating so-called TERFs with Nazis in a credible-sounding tone and explaining how various methods such as infiltration, censorship, property damage, and physical violence should be used against them.<ref name=medium-izaguirre/> The account posting the article was suspended from Medium.com on the same day and the content re-uploaded on WordPress.com soon after.<ref name=wp-izaguirre/> (Still online as of February 20th, 2021)

=== Not even pretending that this isn't a witch hunt? ===

[[File:Spain-witch-puppet.jpg|thumb|right|150px|Effigy of [[Wikipedia:Carmen Calvo|Carmen Calvo]] hanging from a tree.]]

In February 2021, Spanish news reported about the appearance of a puppet hanging from a tree, with its face made to resemble female politician, author, and vice president [[Wikipedia:Carmen Calvo|Carmen Calvo]] of the Socialist Workers' Party.<ref name=elcomun-calvo/> Calvo had come under fire from trans activists for her opposition to a proposed [[gender self-identification]] law. The puppet was hanging from a tree in Plaza 8 de Marzo in Santiago de Compostela, with a sign hanging from its neck saying "Me he perdido…por dónde queda el patriarcado" which could be translated as "I'm lost... where's the patriarchy again?" referring clearly to trans activist claim that it's not them who uphold patriarchal norms, but rather the feminists who oppose them.

{{clear}}

== Real-life assault, vandalism, and harassment ==

This section lists known incidents of real-life (offline) assault, vandalism, verbal abuse and other harassment against feminist organizations or women, by trans activists who are radicalized by the notion that so-called "TERFs" are after their human rights. (The only case in which the motivation is not entirely clear is the triple homicide of a lesbian couple and their son, committed by formerly prominent trans activist Dana Rivers.)

=== Vandalism of Vancouver Women's Library ===

In February 2017, the newly opened Vancouver Women's Library was met with a small group of vandals, including a very obviously male person who claimed to be a female sex worker. They ripped off a poster, poured wine on a book, and harassed those trying to partake in the opening celebration. Their stated reason was that the library included books which they deemed to support so-called "TERF" and "SWERF" ideology.<ref name=vwl/>

<gallery caption="Images of the vandalism" widths=210px heights=175px>
File:VWL-Vandalized-1.jpg | Crude graffiti sprayed at the entrance
File:VWL-Vandalized-2.jpg | Close-up of some of the sprayed text
File:VWL-Vandalized-3.jpg | Wine poured on one of the books
File:VWL-Defamation.jpg | The supposed reason for the vandalism
</gallery>

=== Physical assault at Speakers' Corner, London ===

In September 2017, Maria MacLachlan was physically assaulted by a transgender activist.

Initially, a group of feminists wanted to hold a meeting to discuss proposed changes to the [https://www.feministcurrent.com/2018/09/14/never-mind-reforming-gender-recognition-act-theres-no-need-gender-recognition-certificates/ Gender Recognition Act] (GRA) in the [https://newxlearning.org/ New Cross Learning] community Library in London. The library had to cancel the event after harassment by transgender activists. The organizers of the meeting then decided to meet at Speakers' Corner, before going to the newly chosen meeting place, which was not announced to protect it from harassment.

[[File:Tara-Flik-Wood-threat.webp|thumb|right|300px|Wolf going by the name "Tara Flik Wood" on social media, posting intent to assault prior to the event]]

At Speakers' Corner, the women were met with a group of transgender activists shouting slogans, notably "when TERFs attack, we fight back." Maria MacLachlan, who was filming the protesters with her digital camera, was attacked by someone running out of the trans activist group, who then tried to grab her camera. As the unsuccessful attacker ran back behind his friends, MacLachlan tried to get closer to the group to get his face on camera. Several of the activists started assaulting her in that moment. The whole sequence of events was [https://www.youtube.com/watch?v=9_d3ozhSE-U caught on camera] thanks others at the place also filming what was going on. MacLachlan's camera was broken and the memory card stolen.

One of the attackers, later revealed to be Tara Wolf, was ultimately charged with assault by beating. Prior to the event, he had posted that he wants to "f*ck up some terfs" on social media.

The event could be considered a cornerstone in the escalating hatred transgender activists show against feminists, as no such clearly documented assault in relation to the slur "TERF" existed before, and the event gained widespread attention in the news, being covered by The Guardian,<ref name=guardian/> The New Statesman,<ref name=statesman1/><ref name=statesman2/> The Telegraph,<ref name=telegraph/> The Times,<ref name=times1/><ref name=times2/> The Evening Standard,<ref name=standard/> and The Daily Mail.<ref name=dailymail/> Of course, it was also covered by Feminist Current.<ref name=fc1/><ref name=fc2/>

In the aftermath of the event, many transgender activists online defended or even celebrated the assault, leading Meghan Murphy to publish the piece [https://www.feministcurrent.com/2017/09/21/terf-isnt-slur-hate-speech/ '' 'TERF' isn't just a slur, it's hate speech'']. Some publications in support of transgender activists have tried to claim that the assailant was really acting in self-defense, and tried to prove this claim by uploading carefully edited cuts of the recording showing the assault,<ref name=planettrans/> or by framing the assault as "standing up to bullies" who "provoke" transgender activists (by having opinions they don't like, we have to presume).<ref name=queerness/>

<gallery caption="Images" widths=300px heights=175px>
File:Tara-Flik-Wood-attack.png | A frame from the video documenting the assault
File:Tara-Flik-Wood-supporters.jpg | Some of Wolf's supporters outside court
File:MacLachlan-supporters.jpg | For contrast, MacLachlan's supporters
</gallery>

=== Verbal abuse and other harassment against Prof. Rosa Freedman ===

In December 2018, human rights lawyer Prof. [[Rosa Freedman]] found her office door covered in urine after attending debates surrounding proposed changes to the UK's Gender Recognition Act. She also reported being called a "Nazi" (she is Jewish) who "should be raped" (she is a survivor of sexual violence) and receiving abusive anonymous phone calls.<ref name=freedman/>

=== Trans activist Dana Rivers commits triple homicide ===

In 2018, transgender activist Dana Rivers was on trial for triple murder.<ref name=rivers/> The victims were a lesbian couple and their son. Rivers stabbed and shot the victims, before trying to set their house on fire, in November 2016. It remains unclear whether Rivers was motivated by the hatred against feminists and lesbians promulgated by transgender activists. Rivers was, however, a member of the group [[Wikipedia:Camp Trans|Camp Trans]], which was created to protest the women-only rule of the Michigan Womyn's Music Festival (aka MichFest).<ref name=camptrans/> Autostraddle described Rivers as a "very well-known transgender activist."<ref name=autostraddle/>

=== Trans activist lashes at Julie Bindel after speech ===

[[File:Joe-brennan-twitter.jpg|thumb|right|200px|"Cathy Brennan" encouraging physical violence against women]]

In June 2019, [[Julie Bindel]] was physically attacked by a trans-identifying male person after holding a keynote speech about male violence against women at the Edinburgh University, together with Prof. Rosa Freedman.<ref name=edinburgh/>

Bindel recounted the incidence the following way: "He was shouting and ranting and raving, 'you're a f***** c***, you're a f****** bitch, a f****** Terf" and the rest of it. We were trying to walk to the cab to take us to the airport, and then he just lunged at me and almost punched me in the face, but a security guard pulled him away."

The male person, who calls himself "[[Cathy Brennan]]" (not to be confused with the lesbian and radical feminist lawyer who was born with that name) had previously stood out on social media for encouraging physical violence against feminists at Gay Pride events.

{{clear}}

=== Harassment and vandalism against Vancouver Rape Relief ===

In August 16, 2019, the Facebook account of [[Vancouver Rape Relief and Women's Shelter]] reported that a dead rat had been nailed to their door frame.<ref name=vrr-fb/> On August 26, their Twitter account followed up by reporting their door and windows were vandalized with phrases such as "FUCK TERFS" and "KILL TERFS" as well as "TRANS POWER".<ref name=vrr-twitter/> The repeated harassment and vandalism garnered attention in local and national news.<ref name=vrr-vancouverisawesome/><ref name=vrr-nationalreview/><ref name=vrr-citynews/>

<gallery caption="Images of the vandalism" widths=300px heights=175px>
File:Vrr-vandalism1.jpg | A dead rat nailed to the door frame of VRR
File:Vrr-vandalism2.jpg | The text "KILL TERFS ... TRANS POWER" written on the window
File:Vrr-vandalism3.jpg | The phrases "FUCK TERFS" and "TRANS WOMEN ARE WOMEN" written on the door and window
</gallery>

== Analysis of the term ==

[[File:Theterfs-delusion.png|thumb|right|500px|Some of the extreme hyperbole used to justify hatred against radical feminists]]

The original acronym could be split in two halves: "trans-exclusionary" and "radical feminist." Though many people targeted with the word do not see themselves as radical feminists, their ideals most often tend to align with [[radical feminism]] anyway, making that part somewhat accurate. The "trans-exclusionary" part however is rather ambiguous, and its meaning seems to change at the whim of the person using the term.

When those using the term want to justify it as an objective and accurate description, they will use rather mundane and basic definitions of "exclusion" that applies easily to most people the term is used against. Examples of this might include:

* Wanting transwomen with unfair anatomic advantages to be excluded from women's sports
* Wanting transwomen with an obvious male anatomy (such as intact male genitals) to be excluded from sex segregated spaces of privacy, such as changing rooms
* Not considering transwomen to be ''literally'' women, by pointing at the dictionary definition that is "adult human female"
* Wanting to exclude transwomen from some political groups that want to focus on struggles unique to people born with female anatomy
* Wanting to exclude crimes committed by transwomen from being recorded as female criminality, especially since the crime patterns of transwomen seem more in line with crime patterns of men,<ref name=dhejne/> who commit the vast majority of violent crimes, specifically sexual violence<ref name=wpcrime/>

However, once the term "TERF" is applied to someone on the basis of them holding these opinions (which many people including non-feminists would agree are sensible), the definition of "exclusion" is quickly sharpened to justify expressions of hatred. Sometimes, the "trans-exclusionary" is even hyperbolically turned into "trans-exterminatory" to increase its panic-inducing effect. The website (or should we say whacksite) [http://theterfs.com/ ''The TERFs''] goes as far as claiming that the people labeled "TERF" want to:

* Exclude trans people from housing (make them homeless)
* Exclude trans people from employment (make them unemployed)
* Exclude them from education (keep them uneducated)
* Exclude them from accommodation equality
* Exclude them from local, state, national and United Nations protections(!)

As such, the women labeled "TERF" are represented less as women who simply want to uphold women's sex-based rights, and more like fascist monsters. This is then used to incite hatred and violence against them. It's also noteworthy how exclusion of transwomen (from female-only spaces etc.) turns here into supposed exclusion of all trans people (from whatever). In fact, women targeted as "TERFs" will frequently say explicitly that they welcome transmen in their groups, since transmen also face the sex-based oppression all women face from birth.

The strategy of transgender activists of using simple definitions of "TERF" to make the term look accurate, but then twist the definition to justify hatred, is quite similar to a "troll" strategy that has been noted by philosopher Nicholas Shackel, and dubbed the ''Motte and Bailey Doctrine'' in a paper titled [https://philpapers.org/archive/SHATVO-2.pdf ''The Vacuity of Postmodernist Methodology'']:

[[File:Motte-Bailey.png|thumb|right|500px|An illustration of the "motte and bailey" style of fallacious argumentation.]]

<blockquote>
"Troll’s Truisms are used to insinuate an exciting falsehood, which is a desired doctrine, yet permit retreat to the trivial truth when pressed by an opponent. In so doing they exhibit a property which makes them the simplest possible case of what I shall call a Motte and Bailey Doctrine (since a doctrine can be a single belief or an entire body of beliefs).

A Motte and Bailey castle is a medieval system of defence in which a stone tower on a mound (the Motte) is surrounded by an area of land (the Bailey) which in turn is encompassed by some sort of a barrier such as a ditch. Being dark and dank, the Motte is not a habitation of choice. The only reason for its existence is the desirability of the Bailey, which the combination of the Motte and ditch makes relatively easy to retain despite attack by marauders. When only lightly pressed, the ditch makes small numbers of attackers easy to defeat as they struggle across it: when heavily pressed the ditch is not defensible and so neither is the Bailey. Rather one retreats to the insalubrious but defensible, perhaps impregnable, Motte. Eventually the marauders give up, when one is well placed to reoccupy desirable land.

For my purposes the desirable but only lightly defensible territory of the Motte and Bailey castle, that is to say, the Bailey, represents a philosophical doctrine or position with similar properties: desirable to its proponent but only lightly defensible. The Motte is the defensible but undesired position to which one retreats when hard pressed. I think it is evident that Troll’s Truisms have the Motte and Bailey property, since the exciting falsehoods constitute the desired but indefensible region within the ditch whilst the trivial truth constitutes the defensible but dank Motte to which one may retreat when pressed."
</blockquote>

In our case, the ''Motte'' is an easily defensible statement like: ''"You don't consider transwomen literally women, therefore you are trans-exclusionary, which makes you a TERF."'' Whereas the ''Bailey'' is: ''"You want to exclude trans people from housing and employment, therefore I'm justified in hating you with a passion!"''

It could also be called a "bait-and-switch" argument, where one is "baited" into agreeing with the claim that someone is a "TERF" by using a mundane definition of "trans exclusion," and then the definition is switched into something bad, to justify expressions of hatred.

=== Inspection of the claims on ''The TERFs'' ===

[[File:Theterfs-logo.png|thumb|right|300px|Actual logo of TheTERFs.com: Evil black scribbles]]

It should be noted that ''The TERFs'' offers little more than out-of-context quotes from several decades ago as "evidence" for its hyperbolic claims regarding so-called "TERF" ideology. It also showcases a small number of cherry-picked tweets, half of which are from right-wing sources that also happen to oppose the transgender movement, which trans activists claim proves that feminists are secretly allied with them in a big conspiracy. (This line of thinking is often ridiculed as: "Hitler was a vegetarian, therefore vegetarians are Nazis.")

Regarding supposed evidence of "real-life violence motivated by TERF ideology," the website lists six examples, of which three don't actually present any violence. Those which do, relate to incidences from several decades ago, in which women tried to evict transwomen from female-only groups or spaces, using physical force or face-to-face threats. Since women are expected to be always nice and passive towards males, even when trying to form radical feminist groups and keep males out of them, this is of course seen as unacceptable. Two examples are about verbal threats, which is a run-of-the-mill experience for feminists opposing transgender activists, and the last one is about the deaths of trans people that are ''assumed'' to be related to lack of access to health care, which to be very blunt has fuck-all to do with feminist ideologies.

To summarize: there is not a single documented instance of a feminist group going up to a transgender group to assault or threaten them. Any claims of "violence by TERFs" refer either to women trying to evict transwomen from female-only spaces, extremely rare verbal threats, and lots and lots of fabrication. In comparison, transgender activists have gone to women's libraries to vandalize them,<ref name=vwl/> went to feminist meetings to assault women,<ref name=fc1/><ref name=fc2/> appeared with face masks at feminist conferences to intimidate women,<ref name=jamjar/> and the ubiquitous verbal abuse they send in the direction of women fills up [https://terfisaslur.com/ a whole website] set up solely to archive it.

== 'SWERF' ==

The word ''SWERF'' is a close relative to ''TERF'' and is applied in a similarly dishonest, misrepresenting way. Women (and men who care about women's rights) who are critical of the sex industry for its exploitative nature are accused of being "sex-worker exclusionary" in an attempt to make them seem hateful towards an oppressed group.

In fact, the people slurred as "SWERF" tend to be supporters of the [[Nordic Model]] against prostitution, which sees a high-quality welfare system as a necessary component in tackling prostitution, and in alleviating problems faced by women who would otherwise choose to do prostitution out of economic desperation. Further, many of those slurred "SWERF" tend to be women who worked in prostitution themselves.

Notably, one of the biggest anti-prostitution and anti-pornography feminists [[Andrea Dworkin]] was an ex-prostitute herself, and not ashamed of admitting so. Another notable example is [[Rachel Moran]], who was in prostitution between the ages of 15 and 22, only to become one of the most notable anti-prostitution, pro-Nordic Model activists in recent years.

== Gallery ==

<gallery caption="Some examples of tweets using 'terf'" widths=300px heights=150px>
File:Terf-gallery-1.png | "terf isn't a slur ... terfs don't deserve rights"
File:Terf-gallery-2.jpg | "All terfs will be arrested on sight"
File:Terf-gallery-3.jpg | "It's [[Wikipedia:Pride Month|Pride]] punch a terf"
File:Terf-gallery-4.png | Putting "terf" right aside "nazi"
File:Terf-gallery-5.png | [[Wikipedia:Arthur Chu|Arthur Chu]] supporting SonicFox's hate speech
File:Terf-gallery-6.png | "Being a TERF is a choice. Like being a Nazi."
</gallery>

== See also ==

* [[Transgender ideology]]

== External links ==

* [https://www.feministcurrent.com/2017/09/21/terf-isnt-slur-hate-speech/ TERF isn't just a slur, it's hate speech | Feminist Current]
* [https://www.feministcurrent.com/tag/terf/ TERF Archives | Feminist Current]
* [https://speakupforwomen.nz/dont-call-women-terfs/ Don't Call Women 'Terfs' | Speak Up for Women NZ]
* [https://terfisaslur.com/ terfisaslur.com - Documenting the abuse, harassment and misogyny of transgender identity politics]

== References ==

<references>

<ref name=terf-origin>
{{cite web
|url=https://hoydenabouttown.com/2008/08/17/carnivalia-transgenderism-and-the-gender-binary/
|title=Carnivalia, transgenderism and the gender binary
|author=Viv Smythe (aka tigtog)
|date=August 17, 2008
|website=Hoyden About Town
}}
</ref>

<ref name=guardian-smythe>
{{cite web
|url=https://www.theguardian.com/commentisfree/2018/nov/29/im-credited-with-having-coined-the-acronym-terf-heres-how-it-happened
|title=I'm credited with having coined the word 'Terf'. Here's how it happened
|author=Viv Smythe (aka tigtog)
|date=November 28, 2018
|website=The Guardian
}}
</ref>

<ref name=fc-egbert>
{{cite web
|url=https://www.feministcurrent.com/2014/07/16/defending-the-terf-gender-as-political/
|title=Defending the ‘TERF’: Gender as political
|author=C.K. Egbert
|date=July 16, 2014
|website=Feminist Current
}}
</ref>

<ref name=fc-ditum>
{{cite web
|url=https://www.feministcurrent.com/2014/07/29/how-terf-works/
|title=How ‘TERF’ works
|author=Sarah Ditum
|date=July 29, 2014
|website=Feminist Current
}}
</ref>

<ref name=vice-robbins>
{{cite web
|url=https://www.vice.com/en_uk/article/7bap7y/whats-the-block-blot-martin-robbins-757
|title=I Am Now Officially a Transphobic Twitter Troll
|author=Martin Robbins
|date=August 8, 2014
|website=Vice
}}
</ref>

<ref name=fc-white>
{{cite web
|url=https://www.feministcurrent.com/2015/11/10/why-i-no-longer-hate-terfs/
|title=Why I no longer hate ‘TERFs’
|author=Penny White
|date=November 10, 2015
|website=Feminist Current
}}
</ref>

<ref name=fc-tra-violence>
{{cite web
|url=https://www.feministcurrent.com/2018/05/01/trans-activism-become-centered-justifying-violence-women-time-allies-speak/
|title=Trans activism is excusing & advocating violence against women, and it’s time to speak up
|author=Meghan Murphy
|date=May 1, 2018
|website=Feminist Current
}}
</ref>

<ref name=sonicfox>
{{cite web
|url=https://www.newstatesman.com/politics/feminism/2019/05/welcome-age-ironic-bigotry-where-old-hatreds-are-cloaked-woke-new-language
|title=Welcome to the age of ironic bigotry, where old hatreds are cloaked in woke new language
|author=Helen Lewis
|date=May 1, 2019
|website=The New Statesman
}}
</ref>

<ref name=rousey-fox>
{{cite web
|url=https://bleacherreport.com/articles/1651451-ronda-rousey-vs-fallon-fox-head-to-toe-breakdown
|title=Ronda Rousey vs. Fallon Fox Head-to-Toe Breakdown
|author=Jordy McElroy
|date=May 25, 2013
|website=Bleacher Report
}}
</ref>

<ref name=rousey-marysue>
{{cite web
|url=https://www.themarysue.com/ronda-rousey-and-transmisogyny/
|title=When Role Models Become Problematic: Ronda Rousey and Transmisogyny
|author=Teresa Jusino
|date=August 3, 2015
|website=The Mary Sue
}}
</ref>

<ref name=fallon-fox>
{{cite web
|url=https://archive.is/yZfcs
|title=After Being TKO’d by Fallon Fox, Tamikka Brents Says Transgender Fighters in MMA ‘Just Isn’t Fair’
|website=Cage Potato
}}
</ref>

<ref name=rousey-reddit>
{{cite web
|url=https://www.reddit.com/r/GamerGhazi/comments/ah4120/celebrity_terf_ronda_rousey_to_play_sonya_blade/
|title=Celebrity TERF Ronda Rousey to play Sonya Blade in Mortal Kombat 11
|website=reddit
}}
</ref>

<ref name=pov-terf>
{{cite web
|url=https://web.archive.org/web/20191019184351/https://radfemrebecca.wordpress.com/2019/10/06/terfinmymentions/amp/
|title=“This is what I want to do to you” – Dissecting the ‘POV ur a TERF in my mentions’ trend.
|author=Rebecca
|date=October 6, 2019
|website=radfemrebecca.wordpress.com
}}
</ref>

<ref name=medium-izaguirre>
{{cite web
|url=https://web.archive.org/web/20200527135844/https://medium.com/@laura.izaguirre/resisting-terfs-and-transforming-their-organizations-95cd21714fc8
|title=Resisting TERF's and Transforming Their Organizations (Archived from Medium.com)
|author=Laura Izaguirre
|date=May 26, 2020
|website=Medium.com
}}
</ref>

<ref name=elcomun-calvo>
{{cite web
|url=https://elcomun.es/2021/02/20/violencia-machista-ahorcan-a-una-muneca-con-la-cara-de-carmen-calvo-en-santiago/
|title=Violencia machista: Ahorcan a una muñeca con la cara de Carmen Calvo en Santiago (English: "Sexist violence: They hang a doll with the face of Carmen Calvo in Santiago")
|date=February 20, 2021
|website=El Común
}}

English translation by Google: https://2lovojqocmwrvggaapshhrg6yi-adwhj77lcyoafdy-elcomun-es.translate.goog/2021/02/20/violencia-machista-ahorcan-a-una-muneca-con-la-cara-de-carmen-calvo-en-santiago/
</ref>

<ref name=wp-izaguirre>
{{cite web
|url=https://web.archive.org/web/20200528035805/https://queeratxlaura.wordpress.com/2020/05/27/resisting-terfs-and-transforming-their-organizations/
|title=Resisting TERF's and Transforming Their Organizations (Archived from WordPress.com)
|author=Laura Izaguirre
|date=May 28, 2020
|website=WordPress
}}
</ref>

<ref name=vwl>
{{cite web
|url=https://www.feministcurrent.com/2017/02/07/vancouver-womens-library-opens-amid-anti-feminist-backlash/
|title=Vancouver Women’s Library opens amid anti-feminist backlash
|author=Meghan Murphy
|date=February 7, 2017
|website=Feminist Current
}}
</ref>

<ref name=guardian>
{{cite web
|url=https://www.theguardian.com/uk-news/2017/oct/26/woman-punched-in-brawl-between-transgender-activists-and-radical-feminists
|title=Suspects sought after brawl between transgender activists and radical feminists
|author=Press Association
|date=October 26, 2017
|website=The Guardian
}}
</ref>

<ref name=statesman1>
{{cite web
|url=https://www.newstatesman.com/politics/feminism/2018/04/madness-our-gender-debate-where-feminists-defend-slapping-60-year-old
|title=The madness of our gender debate, where feminists defend slapping a 60-year-old woman
|author=Helen Lewis
|date=April 20, 2018
|website=NewStatesman
}}
</ref>

<ref name=statesman2>
{{cite web
|url=https://www.newstatesman.com/politics/feminism/2017/09/trans-rights-terfs-and-bruised-60-year-old-what-happened-speakers-corner
|title=Trans rights, TERFs, and a bruised 60-year-old: what happened at Speakers’ Corner?
|author=Anoosh Chakelian
|date=September 14, 2017
|website=NewStatesman
}}
</ref>

<ref name=telegraph>
{{cite web
|url=https://www.telegraph.co.uk/news/2018/04/12/radical-feminist-warned-refer-transgender-defendant-assault/
|title=Radical feminist warned to refer to transgender defendant as a 'she' during assault case
|author=Victoria Ward
|date=April 12, 2018
|website=The Telegraph
}}
</ref>

<ref name=times1>
{{cite web
|url=https://www.thetimes.co.uk/article/transgender-activist-denies-thumping-radical-feminist-wvzgq6fx7
|title=Transgender activist Tara Wolf denies thumping radical feminist
|author=Lucy Bannerman
|date=April 12, 2018
|website=The Times
}}
</ref>

<ref name=times2>
{{cite web
|url=https://www.thetimes.co.uk/article/trans-attacker-tara-wolf-is-a-thug-says-feminist-maria-maclachlan-pq0bwvthv
|title=Trans attacker is a thug, says feminist
|author=Lucy Bannerman
|date=April 14, 2018
|website=The Times
}}
</ref>

<ref name=standard>
{{cite web
|url=https://www.standard.co.uk/news/crime/transgender-activist-tara-wolf-fined-150-for-assaulting-exclusionary-radical-feminist-in-hyde-park-a3813856.html
|title=Transgender activist Tara Wolf fined £150 for assaulting 'exclusionary' radical feminist in Hyde Park
|author=Martin Coulter
|date=April 13, 2018
|website=EveningStandard
}}
</ref>

<ref name=dailymail>
{{cite web
|url=https://www.dailymail.co.uk/news/article-5613057/Model-punched-feminist-smashed-120-camera-violent-brawl-walks-free-court.html
|title=Transgender model who punched feminist and smashed her £120 camera in violent brawl at Hyde Park Speakers' Corner protest walks free from court
|author=Bridie Pearson-jones
|date=April 13, 2018
|website=MailOnline
}}
</ref>

<ref name=fc1>
{{cite web
|url=https://www.feministcurrent.com/2017/09/15/historic-speakers-corner-becomes-site-anti-feminist-silencing-violence/
|title=Historic Speaker’s Corner becomes site of anti-feminist silencing and violence
|author=Meghan Murphy
|date=September 15, 2017
|website=Feminist Current
}}
</ref>

<ref name=fc2>
{{cite web
|url=https://www.feministcurrent.com/2018/04/27/trans-identified-male-tara-wolf-charged-assault-hyde-park-attack/
|title=Trans-identified male, Tara Wolf, convicted of assault after Hyde Park attack
|author=Jen Izaakson
|date=April 27, 2018
|website=Feminist Current
}}
</ref>

<ref name=rivers>
{{cite web
|url=https://sanfrancisco.cbslocal.com/2018/03/07/transgender-activist-trial-oakland-triple-murder/
|title=Transgender Activist Ordered To Stand Trial For Oakland Triple Murder
|date=March 7, 2018
|website=CBS SF BayArea
}}
</ref>

<ref name=camptrans>
{{cite web
|url=http://eminism.org/michigan/20000812-camptrans.txt
|title='MICHIGAN EIGHT' EVICTED OVER FESTIVAL'S NEW 'DON'T ASK DON'T TELL' GENDER POLICY
|date=August 12, 2000
}}
</ref>

<ref name=autostraddle>
{{cite web
|url=https://www.autostraddle.com/more-bad-news-oakland-lesbian-couple-and-their-son-brutally-murdered-by-former-lgbt-activist-359026/
|title=Oakland Lesbian Couple and Their Son Murdered By Former LGBT Activist
|author=Riese
|date=November 16, 2016
|website=Autostraddle
}}
</ref>

<ref name=edinburgh>
{{cite web
|url=https://www.scotsman.com/news/crime/feminist-speaker-julie-bindel-attacked-by-transgender-person-at-edinburgh-university-after-talk-1-4942260
|title=Feminist speaker Julie Bindel 'attacked by transgender person' at Edinburgh University after talk
|author=Gina Davidson
|date=June 6, 2019
|website=The Scotsman
}}
</ref>

<ref name=planettrans>
{{cite web
|url=https://planettransgender.com/when-terf-attack-at-hyde-park/
|title=Trans Activists Vilified For Defense against TERF Attack at Hyde Park
|author=Kelli Busey
|date=September 18, 2017
|website=Planet Transgender
}}
</ref>

<ref name=queerness>
{{cite web
|url=https://thequeerness.com/2017/09/29/trial-by-media-over-speakers-corner-fracas/
|title=Trial by media over Speakers’ Corner fracas
|author=Sam Hope
|date=September 29, 2017
|website=The Queerness
}}
</ref>

<ref name=jamjar>
{{cite web
|url=https://twitter.com/magdalenberns/status/988003582250291201
|title=Masked transactivists surround woman on stairwell
|author=Magdalen Berns
|date=April 22, 2018
|website=Twitter
}}
</ref>

<ref name=dhejne>
{{cite web
|url=https://fairplayforwomen.com/criminality/
|title=Study suggests that transwomen exhibit a male pattern of criminality
|date=September 8, 2018
|website=Fair Play For Women
}}
</ref>

<ref name=wpcrime>
{{cite web
|url=https://en.wikipedia.org/wiki/Sex_differences_in_crime#Statistics
|title=Sex differences in crime
|website=Wikipedia
|access-date=May 31, 2019
}}
</ref>

<ref name=freedman>
{{cite web
|url=https://www.bbc.com/news/uk-england-berkshire-46454454
|title=Rosa Freedman: Professor's door 'covered in urine' after gender law debate
|date=December 5, 2018
|website=BBC News
}}
</ref>

<ref name=vrr-fb>
{{cite web
|url=https://www.facebook.com/VancouverRapeRelief/posts/710603379412445
|title=Vancouver Rape Relief & Women's Shelter - Facebook
|date=August 16, 2019
|website=Facebook
}}
</ref>

<ref name=vrr-twitter>
{{cite web
|url=https://twitter.com/VanRapeRelief/status/1166426534321647616
|title=VancouverRapeRelief on Twitter
|date=August 26, 2019
|website=Twitter
}}
</ref>

<ref name=vrr-vancouverisawesome>
{{cite web
|url=https://www.vancouverisawesome.com/2019/08/28/vancouver-rape-relief-centre-targeted-vandalism-threats-transgender-controversy/
|title=Vancouver Rape Relief centre targeted with vandalism, threats over transgender controversy
|author=Jessica Kerr
|date=August 28, 2019
|website=Vancouver is Awesome
}}
</ref>

<ref name=vrr-nationalreview>
{{cite web
|url=https://www.nationalreview.com/2019/08/women-only-rape-relief-shelter-defunded-then-vandalized/
|title=Women-Only Rape-Relief Shelter Defunded, Then Vandalized
|author=Madeleine Kearns
|date=August 28, 2019
|website=National Review
}}
</ref>

<ref name=vrr-citynews>
{{cite web
|url=https://www.citynews1130.com/2019/08/27/vancouver-rape-relief-centre-vandalized-likely-over-restrictions-for-transgender-people/
|title=Vancouver Rape Relief centre vandalized, likely over restrictions for transgender people
|author=Jonathan Szekeres and Lauren Boothby
|date=Aug 27, 2019
|website=City News
}}
</ref>

</references>



[[pt:TERF]]

FeministWiki:Sandbox

2022-01-18T02:19:10Z

Technician :

Test

Test 2

Modèle:Reply to

2022-01-02T20:22:40Z

Technician :

<includeonly>{{#if:{{{1|<noinclude>$</noinclude>}}}
|@[[User:{{{1|Example}}}|{{{1|Example}}}]]{{#if:{{{2|}}}
|, [[User:{{{2}}}|{{{2}}}]]{{#if:{{{3|}}}
|, [[User:{{{3}}}|{{{3}}}]]{{#if:{{{4|}}}
|, [[User:{{{4}}}|{{{4}}}]]{{#if:{{{5|}}}
|, [[User:{{{5}}}|{{{5}}}]]
}}
}}
}}
}}{{{p|:}}}
|{{error|Error in replyto template: Username not given. See [[Template:Reply to]] for usage.}}
}}</includeonly><noinclude>
Example use:

<blockquote>{{C|<nowiki>{{reply to|user1}}</nowiki> Hey mate!}}</blockquote>

Becomes:

<blockquote>{{reply to|user1}} Hey mate!</blockquote>

Example replying to multiple people:

<blockquote>{{C|<nowiki>{{reply to|user1|user2|user3}}</nowiki> Hey mates!}}</blockquote>

Becomes:

<blockquote>{{reply to|user1|user2|user3}} Hey mates!</blockquote>

</noinclude>

Modèle:PageSeo

2021-12-30T23:26:52Z

Technician :

<includeonly>{{#seo:
|title = {{{fulltitle|{{{title|{{PAGENAME}}}}} - {{SITENAME}}}}}
|keywords = {{{fullkeywords|{{{keywords|{{PAGENAME}}}}}, {{SITENAME}}, feminism, wiki, feminist wiki, radical feminism, radfem, gender critical}}}
|description = {{{description}}}
|image = {{{image|FeministWiki_banner.png}}}
}}</includeonly><noinclude>
Use this template to optimize a page for SEO. It uses the <code><nowiki>{{#seo:}}</nowiki></code> tag underneath, which is added by the WikiSEO plugin.

Example usage:
<pre>
{{PageSeo
|title = An alternative title
|keywords = blah, blub, blip
|description = A summary of what this wiki page is about
|image = Some_image.jpg
}}
</pre>

Default values:

* {{c|title}}: <nowiki>{{PAGENAME}}</nowiki> - {{SITENAME}}
* {{c|keywords}}: <nowiki>{{PAGENAME}}</nowiki>, {{SITENAME}}, feminism, wiki, feminist wiki, radical feminism, radfem, gender critical

If you provide the {{c|title}} parameter, it will automatically have {{c| - {{SITENAME}}}} appended to it.

If you provide the {{c|keywords}} parameter, it will automatically have {{c|, {{SITENAME}}, feminism, wiki, feminist wiki, radical feminism, radfem, gender critical}} appended to it.

If you would like to control the full title or keywords tag contents, you can use {{c|fulltitle}} and {{c|fullkeywords}} instead.
</noinclude>

Accueil

2021-12-24T23:42:23Z

Technician :

{{PageSeo
|description = Welcome to the FeministWiki, a digital platform for feminism consisting of a wiki, forum, chat, blog hosting, and more.
}}
''Services: [https://blogs.feministwiki.org FeministBlog] - [https://files.feministwiki.org/ FeministFiles] - [https://mail.feministwiki.org/ FeministMail] - [https://forum.feministwiki.org/ FeministForum] - [https://chat.feministwiki.org/ FeministChat] - [[Help:IRC|FeministIRC]]''

Welcome to '''FeministWiki''', a wiki and integrated digital platform for feminism.

A wiki is a knowledge-base like an encyclopedia, but managed by the public. The FeministWiki specializes on feminism, and is managed by feminists and their supporters. It aims to archive and present information relevant to feminism, and explain feminist perspectives to the reader.

Further, the FeministWiki aims to offer a complete integrated digital platform for feminists, with components such as a blogging platform, a discussion forum, a messaging/chat system, private and shared online storage for large files, and more. You don't need to provide any personally identifying information to become a member, nor do you need to pay; the modest number of users makes the platform cheap to run.

There are a few ways to become a member:
* Use the [https://account.feministwiki.org/register.html Registration Request] form to ask for an account.
* Contact an existing member and have them create an account for you by using the [https://account.feministwiki.org/add-member.html Add Member] form.
* Ask for an account by manually contacting technician@feministwiki.org or by messaging an official social media account of the FeministWiki.

Spinster: [https://spinster.xyz/@FeministWiki @FeministWiki] (preferred) 
Twitter: [https://twitter.com/@FeministWiki @FeministWiki] 
Facebook: [https://www.facebook.com/FeministWiki/ FeministWiki]

Once you are a member, you will be given a username and password with which you can log in to all FeministWiki services. For example, to log in to the wiki itself (which you are reading right now), use the small "Log in" link at the top right of the page.

See the '''[[FW:Welcome|Welcome page]]''' for further information on the FeministWiki, written with new members in mind.

Heads up: '''the FeministWiki needs you'''. All of the technical infrastructure of the FeministWiki is only useful if there's a community making use of it, and content on the wiki doesn't write itself! Be bold, don't shy off of asking for membership, and let the community and the world benefit from your added knowledge. You can become a member even if you have no intention to contribute to the actual wiki; feel free to chat away with other members, discuss matters important to you on the forum, or use the file storage to have a central place to store your favorite information-material on feminism.

== Check out some articles ==

=== Notable feminists ===

* [[Magdalen Berns]]

=== Sex, gender, and trans activism ===

* [[Gender]]
* [[Transgender ideology]]
* [[Cisgender]]
* [[Autogynephilia]]
* [[TERF]]
* [[Transwomen in women's sports]]

=== Sex industry ===

* [[Nordic Model]]

=== Black feminism ===

* [[Black feminism]]

== What is feminism? ==

There are a variety of ideological groupings which call themselves feminism, and some of them are in contradiction with each other. As such, one cannot support all types of feminism at the same time. The FeministWiki is for people who adhere to a relatively straightforward and classical interpretation of feminism: the liberation of female people from male supremacism. This is sometimes called ''radical feminism'' because male supremacism is a radical notion for many people, and its elimination requires radical changes to society.

Male supremacism refers to social and political systems that use stereotypes, myths, discrimination, belittlement, violence, and other means to keep men in power and women oppressed. Women are then exploited for reproductive and domestic labor, men's sexual gratification, and more. Male supremacism also causes collateral damage to some men and boys. For instance, effeminate boys and gay men are often targeted with punishment for failing to uphold the myth that men are inherently masculine.

The FeministWiki promotes second-wave feminist literature:

* [https://radfem.org/ Radical Feminist Archives]

The FeministWiki is in full support of the Women's Human Rights Campaign:

* [https://www.womensdeclaration.com/ Declaration on Women's Sex-Based Rights]

The FeministWiki recommends Spinster as a feminism-friendly social media platform:

* [https://spinster.xyz Spinster.xyz]

Further, the FeministWiki promotes and stands in solidarity with the following groups and organizations:

* [http://lgballiance.org.uk LGB Alliance]: The LGB Alliance
* [http://womensliberationfront.org/ WoLF]: The Women's Liberation Front
* [https://feministcurrent.com/ Feminist Current]: Canadian feminist news, commentary, and podcasts
* [https://www.rapereliefshelter.bc.ca/ Vancouver Rape Relief]: Female-only rape-relief and women's shelter
* [https://nordicmodelnow.org/ Nordic Model Now]: Educational movement for the abolition of prostitution
* [http://www.spaceintl.org/ SPACE International]: Survivors of Prostitution Abuse Calling for Enlightenment
* [https://womansplaceuk.org/ Woman's Place UK]: Women's campaigning group scrutinizing gender self-identification
* [https://speakupforwomen.nz/ Speak Up for Women]: New Zealand group opposing gender self-identification
* [https://feministstruggle.org/ Feminists in Struggle - FIST]: US-based female-only radical feminist network
* [https://pussychurchofmodernwitchcraft.com/ The Pussy Church of Modern Witchcraft]: Lesbian-led Church for Women and Girls

== Who's behind the project? ==

The FeministWiki platform is offered to the public by the German non-profit organization '''FeministWiki gemeinnützige UG (haftungsbeschränkt)'''.

The FeministWiki g. UG is bound by German law to abide by the following mission statement (translated from German):

<blockquote>
1. The company exclusively and directly pursues charitable purposes within the meaning of the section "tax-privileged purposes" of the tax code (AO). The purpose of the company is to promote equality between women and men.

2. The purpose of the statutes is achieved by operating an online platform called feministwiki.org. The platform serves as an information center on the subject of feminism, as well as a communication platform for people who want to stand up for the rights of women.
</blockquote>

The technical infrastructure is managed by [[FW:Technician|the technician]], who also offers user support. For support, please contact technician@feministwiki.org.

For legal inquiries please contact info@feministwiki.org or feministwiki@gmail.com.


[[de:Hauptseite]] [[pt:Página_principal]]

FeministWiki:Server setup

2021-08-31T19:44:01Z

Technician : /* Put config files in place */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Configure reverse DNS ===

In the settings of the VPS host (e.g. Strato AG), you can configure reverse-DNS for the IP address of the server. Set the FQDN for the IP address to {{C|feministwiki.org}}. It's good to do this early since it can take some time to propagate.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts
* Likewise, don't forget {{C|chmod +x}} for <code>/etc/cron.{hourly,daily,weekly,monthly}</code> and {{C|/etc/boot.d}}

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

=== Deactivate again ===

After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

'''Simply repeat the whole section ''Copying over live data''.'''

The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.

So just repeat the steps from that section exactly one more time.

=== Reboot the new server ===

At this point we can reboot the new server again, to make sure all services are properly restarted.

=== Open ports on the new server ===

Now we can open the ports again on the new server:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-31T19:38:06Z

Technician : /* Put config files in place */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Configure reverse DNS ===

In the settings of the VPS host (e.g. Strato AG), you can configure reverse-DNS for the IP address of the server. Set the FQDN for the IP address to {{C|feministwiki.org}}. It's good to do this early since it can take some time to propagate.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts
* Likewise, don't forget {{C|chmod +x}} for {{C|/etc/cron.*}} and {{C|/etc/boot.d}}

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

=== Deactivate again ===

After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

'''Simply repeat the whole section ''Copying over live data''.'''

The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.

So just repeat the steps from that section exactly one more time.

=== Reboot the new server ===

At this point we can reboot the new server again, to make sure all services are properly restarted.

=== Open ports on the new server ===

Now we can open the ports again on the new server:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

TERF

2021-08-31T16:01:11Z

Technician : /* Not even pretending that this isn't a witch hunt? */

{{PageSeo
|description = TERF is a slur used by transgender activists to debase anyone who criticizes their movement on the basis of feminist concerns.
|image = Sonicfox.png
}}
[[File:Terf-slur-02.png|thumb|300px|A typical tweet containing "terf"]]

The word ''TERF'' (or ''terf''; pl ''terfs'' or ''terves'') is a slur that is used predominantly by transgender activists and their allies against people who criticize the transgender movement on the basis of feminist concerns. Since the slur is used for people with feminist concerns, the main target tend to be women. As such, it's usually understood to be an anti-feminist, sexist and misogynist slur.

The word was invented as an acronym for ''Trans-Exclusionary Radical Feminist'', where the "trans-exclusionary" part referred to those holding roughly the position that transwomen should not be included under a feminist definition of womanhood, and the "radical feminist" part was meant neutrally, i.e. for people who would indeed describe themselves as [[Radical feminism|radical feminists]] in the true sense.

Over time, the acronym pretty much became a [[Wikipedia:Four-letter word|four-letter word]]. Nowadays the capitalization is frequently omitted, and the already ambiguous original meaning ignored entirely. Still, users of the term tend to claim that it's a neutral description. The "trans-exclusionary" part may now refer to anyone who thinks transwomen should not have unfettered access to all female-only spaces (e.g. changing rooms), should not partake in women's sports where they have [[Transwomen in women's sports|unfair advantages]], should not be considered a natural part of the lesbian dating pool, etc. Although most members of the public would see these as rather sensible positions (especially when considering that a "transwoman" may have intact male anatomy), transgender activists nevertheless see all of these types of "exclusion" as unacceptable.

A closely associated term is ''SWERF'', which is supposed to stand for ''Sex-Worker-Exclusionary Radical Feminist'' and is used for those who see the sex industry (prostitution, pornography, etc.) as highly exploitative and sexist. Like ''TERF'', the term is almost always applied as a slur, and to misrepresent the political position of the person it's used against. Ironically, some of those who have to face the term most commonly are women who worked in prostitution and became anti-prostitution activists as a result of their own experiences as so-called sex workers.

== History and evolution towards hate speech ==

=== Origin ===

The oldest known use of the term is by Viv Smythe aka "tigtog" in a blog post from 2008.<ref name=terf-origin/> She defended the term as late as 2018, in an article for The Guardian.<ref name=guardian-smythe/> Transgender activists frequently try to defend the term on the grounds that Viv Smythe is a woman who herself claims to be a radical feminist, and seems to have first used the term in a way that is not derogatory. Of course, the benign origins of a term does not mean that it cannot evolve into a slur.

=== Early years and first principled critique ===

The evolution of the term from 2008 into the early to mid 2010s is not well documented. Mostly, feminists had to face the term on social media, where it began to be used regularly to debase their position. In July 2014, [https://www.feministcurrent.com/ Feminist Current] published two articles referencing the term. The first, written by C. K. Egbert and titled ''Defending the 'TERF': Gender as political'', explains and defends in length the political theory underlining the ideas supported by feminists who are slurred as "terf."<ref name=fc-egbert/> The second, written by [[Sarah Ditum]] and titled ''How 'TERF' works'', shortly analyses a situation in which a woman is pressured to retract a statement opposing violence against women, on the grounds that the statement originally stems from a feminist who is considered a "terf".<ref name=fc-ditum/> Since Feminist Current is highly acclaimed among radical-leaning feminists, its decision to support the women slurred with "terf" could be seen as a turning point.

In August 2014, Vice published an article titled ''I Am Now Officially a Transphobic Twitter Troll'' (subtitle: ''At least according to the 'Block Bot' I am'') by author Martin Robbins.<ref name=vice-robbins/> In the article, Robbins talks about how the "Block Bot" project on Twitter, which is supposed to help people avoid abusive trolls, has included feminist authors and journalists such as [[Caroline Criado-Perez]] and [[Helen Lewis]] among the people who should be blocked. Ironically, Lewis seems to have made it to the list for complaining about abusive trolls, as the "evidence" for the reason to ban her includes objections to tweets such as "kill TERFS", "burn TERFS", or hateful jokes such as "what's better than 1 dead terf? 2 dead terfs."

Another Feminist Current article defending those targeted with the slur was published in November 2015, written by [[Penny White]] and titled ''Why I no longer hate ‘TERFs’''.<ref name=fc-white/> In the article, White explains how she herself used to be convinced that so-called "TERFs" are worthy of contempt, but changed her mind after starting to look closer into the issue. This experience seems to resonate with many women and some socially liberal men to this day, who start out being supportive of the transgender movement, only to start becoming skeptical after negative experiences and observations, ultimately leading them to be also labeled "terf" and shunned by transgender activists and their allies. After that, Feminist Current started publishing articles critical of the transgender movement with some frequency, much to the enragement of transgender activists.

=== The mainstreaming of hatred ===

[[File:Mya-byrne-punches-terfs.jpg|thumb|left|150px|Totally not a slur!]]

In June 2017, transgender activist [[Wikipedia:Mya Byrne|Mya Byrne]] came to the San Francisco Pride Parade with a t-shirt reading "I PUNCH TERFS", decorated with a large fake blood-stain. Byrne uploaded a selfie of him wearing the t-shirt at the parade, captioned "This is what gay liberation looks like #pride #yesallterfs" which sparked many negative reactions.<ref name=fc-tra-violence/> The t-shirt would later be displayed at an "art exhibit" at the San Francisco Public Library, set up by the trans activist group ''The Degenderettes''. After complaints, the library removed the t-shirt from the exhibition, though similar items showcasing a violent mentality remained, such as baseball bats wrapped in barbed wire and painted in the colors of the transgender pride flag.<ref name=fc-tra-violence/>

[[File:Sonicfox.png|thumb|right|250px|Dominique McLean's tweet glorifying violence against women.]]

In April 2019, professional video gamer [[Wikipedia:SonicFox|Dominique McLean]] aka "SonicFox" uploaded a video on Twitter, captioned "what I do to terfs," in which a male video game character strikes a female character's neck so hard that her skin comes off, while the camera shows her agony-filled face. McLean can be heard yelling "terf!" into his microphone with each blow dealt to the female character, reminiscent of a lynching.<ref name=sonicfox/> The tweet garnered hundreds of thousands of views, and many thousands of retweets and likes.

McLean's tweet is made even worse by the fact that the hatred is not just directed at an imagined "terf" character, but at the voice actress behind the character, [[Wikipedia:Ronda Rousey|Ronda Rousey]]. Rousey, who is an MMA (mixed martial arts) fighter, is deemed a "TERF" for having stated, with blunt wording, that it's unfair for male MMA fighter [[Transwomen in women's sports#Fallon_Fox|Fallon Fox]] to compete against women.<ref name=rousey-fox/><ref name=rousey-marysue/><ref name=rousey-reddit/> A year after Rousey's remarks, female MMA fighter Tamikka Brents had suffered a concussion and an orbital bone fracture during a fight with Fallon Fox,<ref name=fallon-fox/> but this does not seem to have changed the opinion of trans activists.

After McLean's tweet was reported for hate speech, Twitter initially decided that it [[:File:Sonicfox2.png|did not break their rules]]. Only after continued protest and many more reports did Twitter react, by removing the tweet and issuing SonicFox a mere 24-hour ban.

{{clear}}

<gallery caption="Aftermath of McLean's tweet" widths=300px heights=150px>
File:Sonicfox3.jpg | McLean defending his tweet on the grounds that Ronda Rousey is a "TERF" and as such deserving of the violent contempt
File:Sonicfox-copycat1.png | A copycat tweet showing another video game character dying brutally, with the caption "FUCK TERFS"
File:Sonicfox-copycat2.png | Another copycat tweet. The GIF is of [[Wikipedia:Mortal Kombat 11|Mortal Kombat 11]] character [[Wikipedia:Scorpion (Mortal Kombat)|Scorpion]] burning his opponent alive.
</gallery>

=== New trend: Threaten women with your face attached ===

In late 2019, a social media trend dubbed "POV you're a TERF in my mentions" started, in which trans activists would pose with a weapon (baseball bat, sword, machete, or the like), sometimes preparing to strike or showing them mid-strike, taken as a [[Wikipedia:Point-of-view shot|point-of-view (POV) shot]] and captioned with the phrase "POV you're a TERF in my mentions" or variations thereof.<ref name=pov-terf/> The pictures are supposed to represent the point-of-view (POV) of a "TERF" being assaulted by the depicted person. In other words, women called "TERF" who look at the pictures are supposed to imagine themselves being violently assaulted.

<gallery caption="Some examples of the trend" widths=300px heights=150px>
File:Pov-terf-1.webp
File:Pov-terf-2.webp
File:Pov-terf-3.webp
File:Pov-terf-4.webp
File:Pov-terf-5.webp
File:Pov-terf-6.webp
</gallery>

=== Political justification of physical violence ===

In May 2020, a blog post was published on Medium.com equating so-called TERFs with Nazis in a credible-sounding tone and explaining how various methods such as infiltration, censorship, property damage, and physical violence should be used against them.<ref name=medium-izaguirre/> The account posting the article was suspended from Medium.com on the same day and the content re-uploaded on WordPress.com soon after.<ref name=wp-izaguirre/> (Still online as of February 20th, 2021)

=== Not even pretending that this isn't a witch hunt? ===

[[File:Spain-witch-puppet.jpg|thumb|right|150px|Effigy of [[Wikipedia:Carmen Calvo|Carmen Calvo]] hanging from a tree.]]

In February 2021, Spanish news reported about the appearance of a puppet hanging from a tree, with its face made to resemble female politician, author, and vice president [[Wikipedia:Carmen Calvo|Carmen Calvo]] of the Socialist Workers' Party.<ref name=elcomun-calvo/> Calvo had come under fire from trans activists for her opposition to a proposed [[gender self-identification]] law. The puppet was hanging from a tree in Plaza 8 de Marzo in Santiago de Compostela, with a sign hanging from its neck saying "Me he perdido…por dónde queda el patriarcado" which could be translated as "I'm lost... where's the patriarchy again?" referring clearly to trans activist claim that it's not them who uphold patriarchal norms, but rather the feminists who oppose them.

{{clear}}

== Real-life assault, vandalism, and harassment ==

This section lists known incidents of real-life (offline) assault, vandalism, verbal abuse and other harassment against feminist organizations or women, by trans activists who are radicalized by the notion that so-called "TERFs" are after their human rights. (The only case in which the motivation is not entirely clear is the triple homicide of a lesbian couple and their son, committed by formerly prominent trans activist Dana Rivers.)

=== Vandalism of Vancouver Women's Library ===

In February 2017, the newly opened Vancouver Women's Library was met with a small group of vandals, including a very obviously male person who claimed to be a female sex worker. They ripped off a poster, poured wine on a book, and harassed those trying to partake in the opening celebration. Their stated reason was that the library included books which they deemed to support so-called "TERF" and "SWERF" ideology.<ref name=vwl/>

<gallery caption="Images of the vandalism" widths=210px heights=175px>
File:VWL-Vandalized-1.jpg | Crude graffiti sprayed at the entrance
File:VWL-Vandalized-2.jpg | Close-up of some of the sprayed text
File:VWL-Vandalized-3.jpg | Wine poured on one of the books
File:VWL-Defamation.jpg | The supposed reason for the vandalism
</gallery>

=== Physical assault at Speakers' Corner, London ===

In September 2017, Maria MacLachlan was physically assaulted by a transgender activist.

Initially, a group of feminists wanted to hold a meeting to discuss proposed changes to the [https://www.feministcurrent.com/2018/09/14/never-mind-reforming-gender-recognition-act-theres-no-need-gender-recognition-certificates/ Gender Recognition Act] (GRA) in the [https://newxlearning.org/ New Cross Learning] community Library in London. The library had to cancel the event after harassment by transgender activists. The organizers of the meeting then decided to meet at Speakers' Corner, before going to the newly chosen meeting place, which was not announced to protect it from harassment.

[[File:Tara-Flik-Wood-threat.webp|thumb|right|300px|Wolf going by the name "Tara Flik Wood" on social media, posting intent to assault prior to the event]]

At Speakers' Corner, the women were met with a group of transgender activists shouting slogans, notably "when TERFs attack, we fight back." Maria MacLachlan, who was filming the protesters with her digital camera, was attacked by someone running out of the trans activist group, who then tried to grab her camera. As the unsuccessful attacker ran back behind his friends, MacLachlan tried to get closer to the group to get his face on camera. Several of the activists started assaulting her in that moment. The whole sequence of events was [https://www.youtube.com/watch?v=9_d3ozhSE-U caught on camera] thanks others at the place also filming what was going on. MacLachlan's camera was broken and the memory card stolen.

One of the attackers, later revealed to be Tara Wolf, was ultimately charged with assault by beating. Prior to the event, he had posted that he wants to "f*ck up some terfs" on social media.

The event could be considered a cornerstone in the escalating hatred transgender activists show against feminists, as no such clearly documented assault in relation to the slur "TERF" existed before, and the event gained widespread attention in the news, being covered by The Guardian,<ref name=guardian/> The New Statesman,<ref name=statesman1/><ref name=statesman2/> The Telegraph,<ref name=telegraph/> The Times,<ref name=times1/><ref name=times2/> The Evening Standard,<ref name=standard/> and The Daily Mail.<ref name=dailymail/> Of course, it was also covered by Feminist Current.<ref name=fc1/><ref name=fc2/>

In the aftermath of the event, many transgender activists online defended or even celebrated the assault, leading Meghan Murphy to publish the piece [https://www.feministcurrent.com/2017/09/21/terf-isnt-slur-hate-speech/ '' 'TERF' isn't just a slur, it's hate speech'']. Some publications in support of transgender activists have tried to claim that the assailant was really acting in self-defense, and tried to prove this claim by uploading carefully edited cuts of the recording showing the assault,<ref name=planettrans/> or by framing the assault as "standing up to bullies" who "provoke" transgender activists (by having opinions they don't like, we have to presume).<ref name=queerness/>

<gallery caption="Images" widths=300px heights=175px>
File:Tara-Flik-Wood-attack.png | A frame from the video documenting the assault
File:Tara-Flik-Wood-supporters.jpg | Some of Wolf's supporters outside court
File:MacLachlan-supporters.jpg | For contrast, MacLachlan's supporters
</gallery>

=== Verbal abuse and other harassment against Prof. Rosa Freedman ===

In December 2018, human rights lawyer Prof. [[Rosa Freedman]] found her office door covered in urine after attending debates surrounding proposed changes to the UK's Gender Recognition Act. She also reported being called a "Nazi" (she is Jewish) who "should be raped" (she is a survivor of sexual violence) and receiving abusive anonymous phone calls.<ref name=freedman/>

=== Trans activist Dana Rivers commits triple homicide ===

In 2018, transgender activist Dana Rivers was on trial for triple murder.<ref name=rivers/> The victims were a lesbian couple and their adoptive son. Rivers stabbed and shot the victims, before trying to set their house on fire, in November 2016. It remains unclear whether Rivers was motivated by the hatred against feminists and lesbians promulgated by transgender activists. Rivers was, however, a member of the group [[Wikipedia:Camp Trans|Camp Trans]], which was created to protest the women-only rule of the Michigan Womyn's Music Festival (aka MichFest).<ref name=camptrans/> Autostraddle described Rivers as a "very well-known transgender activist."<ref name=autostraddle/>

=== Trans activist lashes at Julie Bindel after speech ===

[[File:Joe-brennan-twitter.jpg|thumb|right|200px|"Cathy Brennan" encouraging physical violence against women]]

In June 2019, [[Julie Bindel]] was physically attacked by a trans-identifying male person after holding a keynote speech about male violence against women at the Edinburgh University, together with Prof. Rosa Freedman.<ref name=edinburgh/>

Bindel recounted the incidence the following way: "He was shouting and ranting and raving, 'you're a f***** c***, you're a f****** bitch, a f****** Terf" and the rest of it. We were trying to walk to the cab to take us to the airport, and then he just lunged at me and almost punched me in the face, but a security guard pulled him away."

The male person, who calls himself "[[Cathy Brennan]]" (not to be confused with the lesbian and radical feminist lawyer who was born with that name) had previously stood out on social media for encouraging physical violence against feminists at Gay Pride events.

{{clear}}

=== Harassment and vandalism against Vancouver Rape Relief ===

In August 16, 2019, the Facebook account of [[Vancouver Rape Relief and Women's Shelter]] reported that a dead rat had been nailed to their door frame.<ref name=vrr-fb/> On August 26, their Twitter account followed up by reporting their door and windows were vandalized with phrases such as "FUCK TERFS" and "KILL TERFS" as well as "TRANS POWER".<ref name=vrr-twitter/> The repeated harassment and vandalism garnered attention in local and national news.<ref name=vrr-vancouverisawesome/><ref name=vrr-nationalreview/><ref name=vrr-citynews/>

<gallery caption="Images of the vandalism" widths=300px heights=175px>
File:Vrr-vandalism1.jpg | A dead rat nailed to the door frame of VRR
File:Vrr-vandalism2.jpg | The text "KILL TERFS ... TRANS POWER" written on the window
File:Vrr-vandalism3.jpg | The phrases "FUCK TERFS" and "TRANS WOMEN ARE WOMEN" written on the door and window
</gallery>

== Analysis of the term ==

[[File:Theterfs-delusion.png|thumb|right|500px|Some of the extreme hyperbole used to justify hatred against radical feminists]]

The original acronym could be split in two halves: "trans-exclusionary" and "radical feminist." Though many people targeted with the word do not see themselves as radical feminists, their ideals most often tend to align with [[radical feminism]] anyway, making that part somewhat accurate. The "trans-exclusionary" part however is rather ambiguous, and its meaning seems to change at the whim of the person using the term.

When those using the term want to justify it as an objective and accurate description, they will use rather mundane and basic definitions of "exclusion" that applies easily to most people the term is used against. Examples of this might include:

* Wanting transwomen with unfair anatomic advantages to be excluded from women's sports
* Wanting transwomen with an obvious male anatomy (such as intact male genitals) to be excluded from sex segregated spaces of privacy, such as changing rooms
* Not considering transwomen to be ''literally'' women, by pointing at the dictionary definition that is "adult human female"
* Wanting to exclude transwomen from some political groups that want to focus on struggles unique to people born with female anatomy
* Wanting to exclude crimes committed by transwomen from being recorded as female criminality, especially since the crime patterns of transwomen seem more in line with crime patterns of men,<ref name=dhejne/> who commit the vast majority of violent crimes, specifically sexual violence<ref name=wpcrime/>

However, once the term "TERF" is applied to someone on the basis of them holding these opinions (which many people including non-feminists would agree are sensible), the definition of "exclusion" is quickly sharpened to justify expressions of hatred. Sometimes, the "trans-exclusionary" is even hyperbolically turned into "trans-exterminatory" to increase its panic-inducing effect. The website (or should we say whacksite) [http://theterfs.com/ ''The TERFs''] goes as far as claiming that the people labeled "TERF" want to:

* Exclude trans people from housing (make them homeless)
* Exclude trans people from employment (make them unemployed)
* Exclude them from education (keep them uneducated)
* Exclude them from accommodation equality
* Exclude them from local, state, national and United Nations protections(!)

As such, the women labeled "TERF" are represented less as women who simply want to uphold women's sex-based rights, and more like fascist monsters. This is then used to incite hatred and violence against them. It's also noteworthy how exclusion of transwomen (from female-only spaces etc.) turns here into supposed exclusion of all trans people (from whatever). In fact, women targeted as "TERFs" will frequently say explicitly that they welcome transmen in their groups, since transmen also face the sex-based oppression all women face from birth.

The strategy of transgender activists of using simple definitions of "TERF" to make the term look accurate, but then twist the definition to justify hatred, is quite similar to a "troll" strategy that has been noted by philosopher Nicholas Shackel, and dubbed the ''Motte and Bailey Doctrine'' in a paper titled [https://philpapers.org/archive/SHATVO-2.pdf ''The Vacuity of Postmodernist Methodology'']:

[[File:Motte-Bailey.png|thumb|right|500px|An illustration of the "motte and bailey" style of fallacious argumentation.]]

<blockquote>
"Troll’s Truisms are used to insinuate an exciting falsehood, which is a desired doctrine, yet permit retreat to the trivial truth when pressed by an opponent. In so doing they exhibit a property which makes them the simplest possible case of what I shall call a Motte and Bailey Doctrine (since a doctrine can be a single belief or an entire body of beliefs).

A Motte and Bailey castle is a medieval system of defence in which a stone tower on a mound (the Motte) is surrounded by an area of land (the Bailey) which in turn is encompassed by some sort of a barrier such as a ditch. Being dark and dank, the Motte is not a habitation of choice. The only reason for its existence is the desirability of the Bailey, which the combination of the Motte and ditch makes relatively easy to retain despite attack by marauders. When only lightly pressed, the ditch makes small numbers of attackers easy to defeat as they struggle across it: when heavily pressed the ditch is not defensible and so neither is the Bailey. Rather one retreats to the insalubrious but defensible, perhaps impregnable, Motte. Eventually the marauders give up, when one is well placed to reoccupy desirable land.

For my purposes the desirable but only lightly defensible territory of the Motte and Bailey castle, that is to say, the Bailey, represents a philosophical doctrine or position with similar properties: desirable to its proponent but only lightly defensible. The Motte is the defensible but undesired position to which one retreats when hard pressed. I think it is evident that Troll’s Truisms have the Motte and Bailey property, since the exciting falsehoods constitute the desired but indefensible region within the ditch whilst the trivial truth constitutes the defensible but dank Motte to which one may retreat when pressed."
</blockquote>

In our case, the ''Motte'' is an easily defensible statement like: ''"You don't consider transwomen literally women, therefore you are trans-exclusionary, which makes you a TERF."'' Whereas the ''Bailey'' is: ''"You want to exclude trans people from housing and employment, therefore I'm justified in hating you with a passion!"''

It could also be called a "bait-and-switch" argument, where one is "baited" into agreeing with the claim that someone is a "TERF" by using a mundane definition of "trans exclusion," and then the definition is switched into something bad, to justify expressions of hatred.

=== Inspection of the claims on ''The TERFs'' ===

[[File:Theterfs-logo.png|thumb|right|300px|Actual logo of TheTERFs.com: Evil black scribbles]]

It should be noted that ''The TERFs'' offers little more than out-of-context quotes from several decades ago as "evidence" for its hyperbolic claims regarding so-called "TERF" ideology. It also showcases a small number of cherry-picked tweets, half of which are from right-wing sources that also happen to oppose the transgender movement, which trans activists claim proves that feminists are secretly allied with them in a big conspiracy. (This line of thinking is often ridiculed as: "Hitler was a vegetarian, therefore vegetarians are Nazis.")

Regarding supposed evidence of "real-life violence motivated by TERF ideology," the website lists six examples, of which three don't actually present any violence. Those which do, relate to incidences from several decades ago, in which women tried to evict transwomen from female-only groups or spaces, using physical force or face-to-face threats. Since women are expected to be always nice and passive towards males, even when trying to form radical feminist groups and keep males out of them, this is of course seen as unacceptable. Two examples are about verbal threats, which is a run-of-the-mill experience for feminists opposing transgender activists, and the last one is about the deaths of trans people that are ''assumed'' to be related to lack of access to health care, which to be very blunt has fuck-all to do with feminist ideologies.

To summarize: there is not a single documented instance of a feminist group going up to a transgender group to assault or threaten them. Any claims of "violence by TERFs" refer either to women trying to evict transwomen from female-only spaces, extremely rare verbal threats, and lots and lots of fabrication. In comparison, transgender activists have gone to women's libraries to vandalize them,<ref name=vwl/> went to feminist meetings to assault women,<ref name=fc1/><ref name=fc2/> appeared with face masks at feminist conferences to intimidate women,<ref name=jamjar/> and the ubiquitous verbal abuse they send in the direction of women fills up [https://terfisaslur.com/ a whole website] set up solely to archive it.

== 'SWERF' ==

The word ''SWERF'' is a close relative to ''TERF'' and is applied in a similarly dishonest, misrepresenting way. Women (and men who care about women's rights) who are critical of the sex industry for its exploitative nature are accused of being "sex-worker exclusionary" in an attempt to make them seem hateful towards an oppressed group.

In fact, the people slurred as "SWERF" tend to be supporters of the [[Nordic Model]] against prostitution, which sees a high-quality welfare system as a necessary component in tackling prostitution, and in alleviating problems faced by women who would otherwise choose to do prostitution out of economic desperation. Further, many of those slurred "SWERF" tend to be women who worked in prostitution themselves.

Notably, one of the biggest anti-prostitution and anti-pornography feminists [[Andrea Dworkin]] was an ex-prostitute herself, and not ashamed of admitting so. Another notable example is [[Rachel Moran]], who was in prostitution between the ages of 15 and 22, only to become one of the most notable anti-prostitution, pro-Nordic Model activists in recent years.

== Gallery ==

<gallery caption="Some examples of tweets using 'terf'" widths=300px heights=150px>
File:Terf-gallery-1.png | "terf isn't a slur ... terfs don't deserve rights"
File:Terf-gallery-2.jpg | "All terfs will be arrested on sight"
File:Terf-gallery-3.jpg | "It's [[Wikipedia:Pride Month|Pride]] punch a terf"
File:Terf-gallery-4.png | Putting "terf" right aside "nazi"
File:Terf-gallery-5.png | [[Wikipedia:Arthur Chu|Arthur Chu]] supporting SonicFox's hate speech
File:Terf-gallery-6.png | "Being a TERF is a choice. Like being a Nazi."
</gallery>

== See also ==

* [[Transgender ideology]]

== External links ==

* [https://www.feministcurrent.com/2017/09/21/terf-isnt-slur-hate-speech/ TERF isn't just a slur, it's hate speech | Feminist Current]
* [https://www.feministcurrent.com/tag/terf/ TERF Archives | Feminist Current]
* [https://speakupforwomen.nz/dont-call-women-terfs/ Don't Call Women 'Terfs' | Speak Up for Women NZ]
* [https://terfisaslur.com/ terfisaslur.com - Documenting the abuse, harassment and misogyny of transgender identity politics]

== References ==

<references>

<ref name=terf-origin>
{{cite web
|url=https://hoydenabouttown.com/2008/08/17/carnivalia-transgenderism-and-the-gender-binary/
|title=Carnivalia, transgenderism and the gender binary
|author=Viv Smythe (aka tigtog)
|date=August 17, 2008
|website=Hoyden About Town
}}
</ref>

<ref name=guardian-smythe>
{{cite web
|url=https://www.theguardian.com/commentisfree/2018/nov/29/im-credited-with-having-coined-the-acronym-terf-heres-how-it-happened
|title=I'm credited with having coined the word 'Terf'. Here's how it happened
|author=Viv Smythe (aka tigtog)
|date=November 28, 2018
|website=The Guardian
}}
</ref>

<ref name=fc-egbert>
{{cite web
|url=https://www.feministcurrent.com/2014/07/16/defending-the-terf-gender-as-political/
|title=Defending the ‘TERF’: Gender as political
|author=C.K. Egbert
|date=July 16, 2014
|website=Feminist Current
}}
</ref>

<ref name=fc-ditum>
{{cite web
|url=https://www.feministcurrent.com/2014/07/29/how-terf-works/
|title=How ‘TERF’ works
|author=Sarah Ditum
|date=July 29, 2014
|website=Feminist Current
}}
</ref>

<ref name=vice-robbins>
{{cite web
|url=https://www.vice.com/en_uk/article/7bap7y/whats-the-block-blot-martin-robbins-757
|title=I Am Now Officially a Transphobic Twitter Troll
|author=Martin Robbins
|date=August 8, 2014
|website=Vice
}}
</ref>

<ref name=fc-white>
{{cite web
|url=https://www.feministcurrent.com/2015/11/10/why-i-no-longer-hate-terfs/
|title=Why I no longer hate ‘TERFs’
|author=Penny White
|date=November 10, 2015
|website=Feminist Current
}}
</ref>

<ref name=fc-tra-violence>
{{cite web
|url=https://www.feministcurrent.com/2018/05/01/trans-activism-become-centered-justifying-violence-women-time-allies-speak/
|title=Trans activism is excusing & advocating violence against women, and it’s time to speak up
|author=Meghan Murphy
|date=May 1, 2018
|website=Feminist Current
}}
</ref>

<ref name=sonicfox>
{{cite web
|url=https://www.newstatesman.com/politics/feminism/2019/05/welcome-age-ironic-bigotry-where-old-hatreds-are-cloaked-woke-new-language
|title=Welcome to the age of ironic bigotry, where old hatreds are cloaked in woke new language
|author=Helen Lewis
|date=May 1, 2019
|website=The New Statesman
}}
</ref>

<ref name=rousey-fox>
{{cite web
|url=https://bleacherreport.com/articles/1651451-ronda-rousey-vs-fallon-fox-head-to-toe-breakdown
|title=Ronda Rousey vs. Fallon Fox Head-to-Toe Breakdown
|author=Jordy McElroy
|date=May 25, 2013
|website=Bleacher Report
}}
</ref>

<ref name=rousey-marysue>
{{cite web
|url=https://www.themarysue.com/ronda-rousey-and-transmisogyny/
|title=When Role Models Become Problematic: Ronda Rousey and Transmisogyny
|author=Teresa Jusino
|date=August 3, 2015
|website=The Mary Sue
}}
</ref>

<ref name=fallon-fox>
{{cite web
|url=https://archive.is/yZfcs
|title=After Being TKO’d by Fallon Fox, Tamikka Brents Says Transgender Fighters in MMA ‘Just Isn’t Fair’
|website=Cage Potato
}}
</ref>

<ref name=rousey-reddit>
{{cite web
|url=https://www.reddit.com/r/GamerGhazi/comments/ah4120/celebrity_terf_ronda_rousey_to_play_sonya_blade/
|title=Celebrity TERF Ronda Rousey to play Sonya Blade in Mortal Kombat 11
|website=reddit
}}
</ref>

<ref name=pov-terf>
{{cite web
|url=https://web.archive.org/web/20191019184351/https://radfemrebecca.wordpress.com/2019/10/06/terfinmymentions/amp/
|title=“This is what I want to do to you” – Dissecting the ‘POV ur a TERF in my mentions’ trend.
|author=Rebecca
|date=October 6, 2019
|website=radfemrebecca.wordpress.com
}}
</ref>

<ref name=medium-izaguirre>
{{cite web
|url=https://web.archive.org/web/20200527135844/https://medium.com/@laura.izaguirre/resisting-terfs-and-transforming-their-organizations-95cd21714fc8
|title=Resisting TERF's and Transforming Their Organizations (Archived from Medium.com)
|author=Laura Izaguirre
|date=May 26, 2020
|website=Medium.com
}}
</ref>

<ref name=elcomun-calvo>
{{cite web
|url=https://elcomun.es/2021/02/20/violencia-machista-ahorcan-a-una-muneca-con-la-cara-de-carmen-calvo-en-santiago/
|title=Violencia machista: Ahorcan a una muñeca con la cara de Carmen Calvo en Santiago (English: "Sexist violence: They hang a doll with the face of Carmen Calvo in Santiago")
|date=February 20, 2021
|website=El Común
}}

English translation by Google: https://2lovojqocmwrvggaapshhrg6yi-adwhj77lcyoafdy-elcomun-es.translate.goog/2021/02/20/violencia-machista-ahorcan-a-una-muneca-con-la-cara-de-carmen-calvo-en-santiago/
</ref>

<ref name=wp-izaguirre>
{{cite web
|url=https://web.archive.org/web/20200528035805/https://queeratxlaura.wordpress.com/2020/05/27/resisting-terfs-and-transforming-their-organizations/
|title=Resisting TERF's and Transforming Their Organizations (Archived from WordPress.com)
|author=Laura Izaguirre
|date=May 28, 2020
|website=WordPress
}}
</ref>

<ref name=vwl>
{{cite web
|url=https://www.feministcurrent.com/2017/02/07/vancouver-womens-library-opens-amid-anti-feminist-backlash/
|title=Vancouver Women’s Library opens amid anti-feminist backlash
|author=Meghan Murphy
|date=February 7, 2017
|website=Feminist Current
}}
</ref>

<ref name=guardian>
{{cite web
|url=https://www.theguardian.com/uk-news/2017/oct/26/woman-punched-in-brawl-between-transgender-activists-and-radical-feminists
|title=Suspects sought after brawl between transgender activists and radical feminists
|author=Press Association
|date=October 26, 2017
|website=The Guardian
}}
</ref>

<ref name=statesman1>
{{cite web
|url=https://www.newstatesman.com/politics/feminism/2018/04/madness-our-gender-debate-where-feminists-defend-slapping-60-year-old
|title=The madness of our gender debate, where feminists defend slapping a 60-year-old woman
|author=Helen Lewis
|date=April 20, 2018
|website=NewStatesman
}}
</ref>

<ref name=statesman2>
{{cite web
|url=https://www.newstatesman.com/politics/feminism/2017/09/trans-rights-terfs-and-bruised-60-year-old-what-happened-speakers-corner
|title=Trans rights, TERFs, and a bruised 60-year-old: what happened at Speakers’ Corner?
|author=Anoosh Chakelian
|date=September 14, 2017
|website=NewStatesman
}}
</ref>

<ref name=telegraph>
{{cite web
|url=https://www.telegraph.co.uk/news/2018/04/12/radical-feminist-warned-refer-transgender-defendant-assault/
|title=Radical feminist warned to refer to transgender defendant as a 'she' during assault case
|author=Victoria Ward
|date=April 12, 2018
|website=The Telegraph
}}
</ref>

<ref name=times1>
{{cite web
|url=https://www.thetimes.co.uk/article/transgender-activist-denies-thumping-radical-feminist-wvzgq6fx7
|title=Transgender activist Tara Wolf denies thumping radical feminist
|author=Lucy Bannerman
|date=April 12, 2018
|website=The Times
}}
</ref>

<ref name=times2>
{{cite web
|url=https://www.thetimes.co.uk/article/trans-attacker-tara-wolf-is-a-thug-says-feminist-maria-maclachlan-pq0bwvthv
|title=Trans attacker is a thug, says feminist
|author=Lucy Bannerman
|date=April 14, 2018
|website=The Times
}}
</ref>

<ref name=standard>
{{cite web
|url=https://www.standard.co.uk/news/crime/transgender-activist-tara-wolf-fined-150-for-assaulting-exclusionary-radical-feminist-in-hyde-park-a3813856.html
|title=Transgender activist Tara Wolf fined £150 for assaulting 'exclusionary' radical feminist in Hyde Park
|author=Martin Coulter
|date=April 13, 2018
|website=EveningStandard
}}
</ref>

<ref name=dailymail>
{{cite web
|url=https://www.dailymail.co.uk/news/article-5613057/Model-punched-feminist-smashed-120-camera-violent-brawl-walks-free-court.html
|title=Transgender model who punched feminist and smashed her £120 camera in violent brawl at Hyde Park Speakers' Corner protest walks free from court
|author=Bridie Pearson-jones
|date=April 13, 2018
|website=MailOnline
}}
</ref>

<ref name=fc1>
{{cite web
|url=https://www.feministcurrent.com/2017/09/15/historic-speakers-corner-becomes-site-anti-feminist-silencing-violence/
|title=Historic Speaker’s Corner becomes site of anti-feminist silencing and violence
|author=Meghan Murphy
|date=September 15, 2017
|website=Feminist Current
}}
</ref>

<ref name=fc2>
{{cite web
|url=https://www.feministcurrent.com/2018/04/27/trans-identified-male-tara-wolf-charged-assault-hyde-park-attack/
|title=Trans-identified male, Tara Wolf, convicted of assault after Hyde Park attack
|author=Jen Izaakson
|date=April 27, 2018
|website=Feminist Current
}}
</ref>

<ref name=rivers>
{{cite web
|url=https://sanfrancisco.cbslocal.com/2018/03/07/transgender-activist-trial-oakland-triple-murder/
|title=Transgender Activist Ordered To Stand Trial For Oakland Triple Murder
|date=March 7, 2018
|website=CBS SF BayArea
}}
</ref>

<ref name=camptrans>
{{cite web
|url=http://eminism.org/michigan/20000812-camptrans.txt
|title='MICHIGAN EIGHT' EVICTED OVER FESTIVAL'S NEW 'DON'T ASK DON'T TELL' GENDER POLICY
|date=August 12, 2000
}}
</ref>

<ref name=autostraddle>
{{cite web
|url=https://www.autostraddle.com/more-bad-news-oakland-lesbian-couple-and-their-son-brutally-murdered-by-former-lgbt-activist-359026/
|title=Oakland Lesbian Couple and Their Son Murdered By Former LGBT Activist
|author=Riese
|date=November 16, 2016
|website=Autostraddle
}}
</ref>

<ref name=edinburgh>
{{cite web
|url=https://www.scotsman.com/news/crime/feminist-speaker-julie-bindel-attacked-by-transgender-person-at-edinburgh-university-after-talk-1-4942260
|title=Feminist speaker Julie Bindel 'attacked by transgender person' at Edinburgh University after talk
|author=Gina Davidson
|date=June 6, 2019
|website=The Scotsman
}}
</ref>

<ref name=planettrans>
{{cite web
|url=https://planettransgender.com/when-terf-attack-at-hyde-park/
|title=Trans Activists Vilified For Defense against TERF Attack at Hyde Park
|author=Kelli Busey
|date=September 18, 2017
|website=Planet Transgender
}}
</ref>

<ref name=queerness>
{{cite web
|url=https://thequeerness.com/2017/09/29/trial-by-media-over-speakers-corner-fracas/
|title=Trial by media over Speakers’ Corner fracas
|author=Sam Hope
|date=September 29, 2017
|website=The Queerness
}}
</ref>

<ref name=jamjar>
{{cite web
|url=https://twitter.com/magdalenberns/status/988003582250291201
|title=Masked transactivists surround woman on stairwell
|author=Magdalen Berns
|date=April 22, 2018
|website=Twitter
}}
</ref>

<ref name=dhejne>
{{cite web
|url=https://fairplayforwomen.com/criminality/
|title=Study suggests that transwomen exhibit a male pattern of criminality
|date=September 8, 2018
|website=Fair Play For Women
}}
</ref>

<ref name=wpcrime>
{{cite web
|url=https://en.wikipedia.org/wiki/Sex_differences_in_crime#Statistics
|title=Sex differences in crime
|website=Wikipedia
|access-date=May 31, 2019
}}
</ref>

<ref name=freedman>
{{cite web
|url=https://www.bbc.com/news/uk-england-berkshire-46454454
|title=Rosa Freedman: Professor's door 'covered in urine' after gender law debate
|date=December 5, 2018
|website=BBC News
}}
</ref>

<ref name=vrr-fb>
{{cite web
|url=https://www.facebook.com/VancouverRapeRelief/posts/710603379412445
|title=Vancouver Rape Relief & Women's Shelter - Facebook
|date=August 16, 2019
|website=Facebook
}}
</ref>

<ref name=vrr-twitter>
{{cite web
|url=https://twitter.com/VanRapeRelief/status/1166426534321647616
|title=VancouverRapeRelief on Twitter
|date=August 26, 2019
|website=Twitter
}}
</ref>

<ref name=vrr-vancouverisawesome>
{{cite web
|url=https://www.vancouverisawesome.com/2019/08/28/vancouver-rape-relief-centre-targeted-vandalism-threats-transgender-controversy/
|title=Vancouver Rape Relief centre targeted with vandalism, threats over transgender controversy
|author=Jessica Kerr
|date=August 28, 2019
|website=Vancouver is Awesome
}}
</ref>

<ref name=vrr-nationalreview>
{{cite web
|url=https://www.nationalreview.com/2019/08/women-only-rape-relief-shelter-defunded-then-vandalized/
|title=Women-Only Rape-Relief Shelter Defunded, Then Vandalized
|author=Madeleine Kearns
|date=August 28, 2019
|website=National Review
}}
</ref>

<ref name=vrr-citynews>
{{cite web
|url=https://www.citynews1130.com/2019/08/27/vancouver-rape-relief-centre-vandalized-likely-over-restrictions-for-transgender-people/
|title=Vancouver Rape Relief centre vandalized, likely over restrictions for transgender people
|author=Jonathan Szekeres and Lauren Boothby
|date=Aug 27, 2019
|website=City News
}}
</ref>

</references>



[[pt:TERF]]

FeministWiki:Server setup

2021-08-26T19:05:43Z

Technician : /* Initial setup of the new server */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Configure reverse DNS ===

In the settings of the VPS host (e.g. Strato AG), you can configure reverse-DNS for the IP address of the server. Set the FQDN for the IP address to {{C|feministwiki.org}}. It's good to do this early since it can take some time to propagate.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

=== Deactivate again ===

After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

'''Simply repeat the whole section ''Copying over live data''.'''

The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.

So just repeat the steps from that section exactly one more time.

=== Reboot the new server ===

At this point we can reboot the new server again, to make sure all services are properly restarted.

=== Open ports on the new server ===

Now we can open the ports again on the new server:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-26T19:05:02Z

Technician :

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

=== Configure reverse DNS ===

In the settings of the VPS host (e.g. Strato AG), you can configure reverse-DNS for the IP address of the server. Set the FQDN for the IP address to {{C|feministwiki.org}}.

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

=== Deactivate again ===

After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

'''Simply repeat the whole section ''Copying over live data''.'''

The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.

So just repeat the steps from that section exactly one more time.

=== Reboot the new server ===

At this point we can reboot the new server again, to make sure all services are properly restarted.

=== Open ports on the new server ===

Now we can open the ports again on the new server:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-26T19:02:56Z

Technician : /* Finishing up */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

=== Deactivate again ===

After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

'''Simply repeat the whole section ''Copying over live data''.'''

The techniques and commands described above in the section ''Copying over live data'' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.

So just repeat the steps from that section exactly one more time.

=== Reboot the new server ===

At this point we can reboot the new server again, to make sure all services are properly restarted.

=== Open ports on the new server ===

Now we can open the ports again on the new server:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-26T18:59:11Z

Technician : /* Deactivate again */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

=== Deactivate again ===

After we're done testing, we can "deactivate" the new server again to prepare it for the final switch-over:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-26T18:58:33Z

Technician : /* Test */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

=== Deactivate again ===

After we're done testing, we can "deactivate" the server again to prepare it for the final switch-over:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-26T18:53:33Z

Technician : /* Mailman data */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
rsync -az --delete archives data lists root@feministwiki.dev:/var/lib/mailman

And then this on the new server:

check_perms -f

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing file ownership and permissions.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-25T18:05:32Z

Technician : /* Recreate SQL users */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;
grant all on feministwiki_de.* to feministwiki@localhost;
grant all on feministwiki_es.* to feministwiki@localhost;
grant all on feministwiki_it.* to feministwiki@localhost;
grant all on feministwiki_pt.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Welcome

2021-08-23T16:28:23Z

Technician : /* Can I trust you with my data? */

Welcome to the FeministWiki! This page will guide you through everything you need to know as a member.

== What is the FeministWiki? ==

The FeministWiki is a website with different components, that aims to offer a rich digital platform for feminist information and activism. The components of the FeministWiki are:

* A wiki, where educational and informational articles on feminist topics can be curated by the community. Like Wikipedia, but for feminism. You're reading a page of the wiki right now.
* A [https://blogs.feministwiki.org/ blog platform] where members who wish to publish articles can become authors on the shared main blog, or get their own personalized blog which they have full control over, such as: [https://blogs.feministwiki.org/socjuswiz/ SocialJusticeWizardry]
* A [https://files.feministwiki.org file-storage platform] (similar to DropBox) where you can upload files you want to save, and optionally share them with others. For instance, you could upload PDF documents, information charts, recordings of seminars, or even just feminist memes, so you can access them from any computer by logging in to your FeministFiles account.
* A [https://forum.feministwiki.org/ traditional web forum] where members can hold discussions about all sorts of topics. If you're familiar with the British website Mumsnet, this one's a bit like that.
* A [https://chat.feministwiki.org chat messaging system] that can be used through the website or via smartphone apps. Like WhatsApp, but accessible for FeministWiki members only, and it doesn't need your mobile number.

Members of the FeministWiki can use all of these services by logging in with the same username and password in each of them.

Further, every member is given an e-mail address like ''janedoe@feministwiki.org'' which they can use to send and receive e-mails. This might be useful, for example, if you don't want to use your personal e-mail address for political purposes.

=== What sort of feminism is it for? ===

As explained on the [[Main Page]], the FeministWiki is aimed at classical/radical feminism.

This includes, for instance, anti-prostitution and anti-pornography activism, female reproductive rights, opposition to gender stereotypes, support for female-only spaces, alliance with lesbian feminists and generally support for lesbian rights, and so on.

Genuine intersectional approaches are definitely valued, such as alliance with black feminists, support of women in poverty, etc., whereas faux-intersectionality that denies sex-based oppression and centers male interests is frowned upon.

=== How are new members added? ===

All members of the FeministWiki have a right to [https://add-member.feministwiki.org add further members] as they like.

Please be careful in who you add, as communities like this are juicy targets for troll infiltration. The system internally keeps track of who was added by who, so in the absolute worst case the technician is able to find the source of a troll infiltration and issue a sweeping ban to bring back peace, but it would of course be ideal if something like this didn't happen in the first place.

That said, please bring in as many of your trusted friends as you can! The FeministWiki only has a purpose so long as there's a community making use of it.

=== Can I trust you with my data? ===

'''Please never use any FeministWiki service to store or transmit sensitive private information, period'''. The [[FW:Technician|technician(s)]] who have administrative access to the server can see your chat messages, emails, files uploaded to file storage, and so on, unless you use encryption on your computer before uploading the data. There is also always the possibility of security holes in the server leading to data leaks, even if the technicians are trustworthy and follow state-of-the-art security best practices.

That said, the FeministWiki promises to show the utmost responsibility with regard to data privacy and security. It doesn't expect you to provide any sort of personally identifying information in your profile, and even if you choose to do so, the FeministWiki will never give that information out, '''unless required by German law'''.

Being hosted in Germany and belonging to a German non-profit organization, the FeministWiki is also bound by European and German law with regard to respecting user data and privacy.

For more in-depth information, you may read the [[FW:Privacy policy|FeministWiki Privacy Policy]], which contains a legally binding policy text and some non-binding but detailed explanations in lay terms.

=== Who runs the site? ===

The platform is offered by the German non-profit company '''FeministWiki gemeinnützige UG (haftungsbeschränkt)''' or FeministWiki gUG for short. The founder of the non-profit and administrator of the website is male computer programmer [https://twitter.com/KammerTaylan Taylan Kammer], who also goes by [https://spinster.xyz/@socjuswiz Social Justice Wizard] on Spinster.

== Help topics ==

=== What happens if I lose my password? ===

''Main help page: [[Help:Password]]''

If you want to have safety against lost passwords, you can set a recovery e-mail address via the [https://settings.feministwiki.org/settings.html FeministWiki Account Settings] page. This e-mail address should '''not''' be your FeministWiki e-mail address (like ''janedoe@feministwiki.org''), because you need your FeministWiki password to access that one in the first place. (Chicken and egg problem.) The recovery e-mail address will be invisible to everyone except the FeministWiki technician.

If it's very important for you to keep your identity private, and if you don't trust the technician or fear data leaks, then you can use an e-mail address that isn't tied to your real identity. Just make sure that you can always access the e-mail that you use for this purpose, as otherwise you will not be able to reset your password.

'''Alternatively,''' you can contact the technician by sending an email to technician@feministwiki.org and ask for a manual password reset.

=== How does creating or editing wiki pages work? ===

''Main help page: [[Help:Wiki]]''

Getting the hang of wiki editing may take some time, but the community will surely be delighted by your contributions!

See the help page linked above to get started, or dive right into the official and comprehensive [https://www.mediawiki.org/wiki/Help:Contents MediaWiki help page] if you're already somewhat skilled with software or feel courageous.

=== How do I use the forum? ===

''Main help page: [[Help:Forum]]''

''Forum front-page: https://forum.feministwiki.org/''

An internet forum or web forum is a website that allows members to create "topics" (also called "threads") to discuss a certain matter. Once a topic is created, other members can reply (or "post") to the topic to add their insights. There is no limit to what these topics may be about, so the forum usually offers a number of categories (or "sub forums") under which the topics are grouped. A well-known example of a web forum is the British website [https://www.mumsnet.com/Talk/active-conversations Mumsnet].

For detailed instructions on how to use the FeministWiki forum, visit the forum help page linked above.

=== How do I use the chat system? ===

''Main help page: [[Help:Chat]]''

''Chat web-interface: https://chat.feministwiki.org/''

The simplest way to use the chat is by opening the web interface linked above, and logging in there with your FeministWiki username and password.

You can also access the chat from dedicated chat programs like [https://gajim.org/ Gajim] or smartphone apps like [https://www.xabber.com/android/ Xabber for Android] or [https://chatsecure.org/ ChatSecure for iOS].

For detailed instructions on how to set up some of these chat programs/apps, see the help page linked above.

=== How do I publish on the blog? ===

''Blog front-page: https://blogs.feministwiki.org/''

If you want to publish articles on the FeministWiki blog, ask the technician by sending an e-mail to technician@feministwiki.org, and your FeministWiki account will be granted the ability to publish on the blog. If you want, you can also get a personalized blog that you have full control over, under a name like "blogs.feministwiki.org/JaneDoe".

The blog uses a self-hosted installation of the well-known blogging software [https://wordpress.com/features/ WordPress]. While the WordPress organization controls blogs that are hosted on their own servers, they also release the software behind their blogging system under a [https://www.fsf.org/about/what-is-free-software free software] license, so anyone can install it on their own servers. The FeministWiki has such a local installation of that software, meaning that the WordPress organization has no control over what's published on the FeministWiki blog. As such, you don't need to fear censorship.

For further information on the FeministWiki blog, visit the blog front-page linked above.

=== How do I use the file storage? ===

''Main help page: [[Help:Files]]''

''Files web-interface: https://files.feministwiki.org/''

The FeministWiki file storage lets you upload potentially very large files and save them on the FeministWiki servers. You can then access the files from anywhere, and optionally share some files with others via a link you send them.

To prevent accidental overloading of the server, every member is granted a quota of 1 GB storage by default. If you would like to store more data, just ask the technician to increase your quota.

For detailed information on how to use the file storage, see the help page linked above.

=== How do I use my FeministWiki e-mail address? ===

''Main help page: [[Help:Mail]]''

''Mail web-interface: https://mail.feministwiki.org/''

The easiest way to use your FeministWiki e-mail is by visiting the web interface linked above, and logging in with your FeministWiki username and password.

You can also set up any e-mail program/app on your computer or smartphone to use your FeministWiki e-mail address.

For further details, see the help page linked above.

=== Wait what? You have an IRC server? ===

''Main help page: [[Help:IRC]]''

The FeministWiki offers an ''Internet Relay Chat'' server for those who have been using computers for a long time and feel especially nostalgic, or those among the younger generations who have re-discovered IRC.

The server is only open to members. It rejects connections from those who can't authenticate with a valid FeministWiki username and password. The hostname is '''irc.feministwiki.org''' and only encrypted connections are accepted, on port 6697. To establish a connection, configure your IRC client so that your IRC nick is your FeministWiki username, and make your client use the rudimentary <code>PASS</code> authentication method with your FeministWiki password. (In most IRC clients this will simply correspond to a "password" text field that you fill out while configuring the connection.)

=== I have a friend who wants to become a member! ===

''Interface: [https://account.feministwiki.org/add-member.html Add a member]''

Every member of the FeministWiki can add further members. Currently the above-linked basic web interface is the way to do so.

Simply fill out your own FeministWiki username and password, and then enter the desired username for the member you want to add. After you click the "Add member" button, the page will show some text saying that the operation was successful and show you an automatically generated password. Send the username and the generated password to the new member and inform them that they can change their password after logging in.

FeministWiki:Server setup

2021-08-23T00:04:26Z

Technician : /* Install PHP and modules */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version} \
php${php_version}-apcu \
php${php_version}-bcmath \
php${php_version}-cli \
php${php_version}-ctype \
php${php_version}-curl \
php${php_version}-gd \
php${php_version}-gmp \
php${php_version}-iconv \
php${php_version}-imagick \
php${php_version}-intl \
php${php_version}-json \
php${php_version}-ldap \
php${php_version}-mbstring \
php${php_version}-mysql \
php${php_version}-opcache \
php${php_version}-readline \
php${php_version}-xml \
php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-23T00:04:02Z

Technician : /* Install server components */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2 \
dovecot-core \
dovecot-imapd \
dovecot-ldap \
dovecot-pop3d \
ejabberd \
fail2ban \
inspircd \
mailman \
mariadb-server \
opendkim \
postfix \
postfix-ldap \
slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-23T00:03:00Z

Technician : /* Install miscellaneous tools */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup \
certbot \
dnsutils \
emacs \
git \
mg \
moreutils \
net-tools \
nmap \
rsync \
software-properties-common \
tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-23T00:01:59Z

Technician : /* Install miscellaneous tools */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install -y automysqlbackup
apt-get install -y certbot
apt-get install -y dnsutils
apt-get install -y emacs
apt-get install -y git
apt-get install -y mg
apt-get install -y moreutils
apt-get install -y net-tools
apt-get install -y nmap
apt-get install -y rsync
apt-get install -y software-properties-common
apt-get install -y tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:35:34Z

Technician : Change all uses of the code HTML tag to the "C" template.

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the {{C|A}} entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the {{C|.ssh/id_rsa}} from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in {{C|/root/pwd/meta}} on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in {{C|/root/pwd}}.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our {{C|letsencrypt-refresh}} script makes sure that the cert files are found in {{C|/etc/fw-certs}} and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka {{C|slapd}}) versions which require manual editing of the {{C|slapcat}} output before it's read in on the new server with {{C|slapadd}}.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open {{C|/etc/ldap/schema/ppolicy.ldif}} and search for {{C|pwdMaxRecordedFailure}}. You will note that there is a {{C|olcAttributeTypes: ...}} entry that defines it, and also it's listed in the {{C|MAY}} attributes block of the {{C|olcObjectClasses: ...}} entry that defines the {{C|pwdPolicy}} object class.
# On the old server, save the output of {{C|slapcat -n 0}} to a file, open the file, and search for the block where the {{C|ppolicy}} schema is defined. It should start with the line {{C|dn: cn={4}ppolicy,cn=schema,cn=config}} (the {{C|{4}}} part might contain a different integer, that's OK). There, note that the {{C|olcAttributeTypes: ...}} entry for {{C|pwdMaxRecordedFailure}} is missing, and also it's not listed in the {{C|MAY}} list of the {{C|pwdPolicy}} object class definition. Copy over the attribute type definition from the {{C|ppolicy.ldif}} file on the new server, and amend the {{C|MAY}} list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in {{C|/var/www/}} is important; if not provided, it will copy the directory to {{C|/var/www/www}} on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the {{C|show databases;}} command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the {{C|--all-databases}} option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the {{C|--delete}} argument to the {{C|rsync}} command and the {{C|--add-drop-database}} argument to the {{C|mysqldump}} command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main {{C|A}} entry for {{C|@}} (self-reference i.e. feministwiki.org)
* The {{C|A}} entry for {{C|smtp}} since this is not allowed to be a CNAME
* The {{C|A}} entry for {{C|xmpp}} since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main {{C|A}} entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the {{C|letsencrypt-refresh}} script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:34:16Z

Technician : /* Contents of /var/www */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -az --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:34:07Z

Technician : /* Emails */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -az --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:33:56Z

Technician : /* SQL databases */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| gzip | ssh root@feministwiki.dev 'gunzip | /root/bin/sql'

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:32:29Z

Technician : /* Breaking changes in OpenLDAP */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:32:08Z

Technician : /* LDAP databases */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

==== Breaking changes in OpenLDAP ====

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:31:24Z

Technician : /* Copying over live data */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes. Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T12:29:59Z

Technician : /* Enable Apache modules, config, and sites */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite account
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail
a2ensite xmpp

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T07:59:10Z

Technician : /* Install server components */

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install postfix-ldap
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T07:47:01Z

Technician :

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T03:09:42Z

Technician : /* Emails */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/home/vmail/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T03:09:29Z

Technician : /* Copying over live data */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

Note that the trailing slash in {{C|/var/www/}} is important.

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T03:00:12Z

Technician : /* Initial setup of the new server */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T02:58:20Z

Technician : /* Enable Apache modules */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Enable Apache modules, config, and sites ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

a2enconf 99-feministwiki

a2ensite 000-wiki
a2ensite blogs
a2ensite chat
a2ensite files
a2ensite forum
a2ensite mail

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T02:09:50Z

Technician : /* Install PHP and modules */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-apcu
apt-get install php${php_version}-bcmath
apt-get install php${php_version}-cli
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-gmp
apt-get install php${php_version}-iconv
apt-get install php${php_version}-imagick
apt-get install php${php_version}-intl
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-opcache
apt-get install php${php_version}-readline
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T02:05:44Z

Technician : /* Install server components */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Install PHP and modules ===

This should really be part of the last section, but due to the sheer number of PHP modules we want to install, it's in its own section:

php_version=7.4 # or whatever version we're on

apt-get install php${php_version}
apt-get install php${php_version}-ctype
apt-get install php${php_version}-curl
apt-get install php${php_version}-gd
apt-get install php${php_version}-iconv
apt-get install php${php_version}-json
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install php${php_version}-zip

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T01:47:52Z

Technician : /* Install server components */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-ldap
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T01:45:21Z

Technician : /* Copying over live data */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

Note that although some of the steps described in this section take a long time to finish, they can be done in parallel.

=== LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Contents of /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

=== SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

=== Emails ===

This is a simple one. Run this command from the old server:

rsync -a --delete /home/vmail/ root@feministwiki.dev:/home/vmail

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T01:39:33Z

Technician : /* Install server components */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-ldap
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T00:53:25Z

Technician : /* Stop services on the old server */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T00:37:54Z

Technician : /* Recreate SQL users */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

/root/bin/sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop mariadb
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T00:37:40Z

Technician : /* Recreate SQL users */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch. To do so, run this on the new server:

sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop mariadb
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T00:37:21Z

Technician : /* Copy over SQL databases */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev /root/bin/sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch:

sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop mariadb
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T00:36:36Z

Technician : /* Copy over SQL databases */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministfiles \
feministforum \
feministmail \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev sql

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch:

sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop mariadb
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T00:35:39Z

Technician : /* Copying over live data */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministblog \
feministfiles \
feministforum \
feministmail \
feministsocial \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev 'mysql -u root -p"$(cat /root/pwd/mysql)"'

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Recreate SQL users ==

If the versions of MariaDB on the old and new server are compatible enough, you might be able to dump the {{C|mysql.user}} table and import it on the new server, but it's safer to recreate the users from scratch:

sql << EOF
create user blogs@localhost identified by '$(cat ~/pwd/mysql-blogs)';
grant all on blogs.* to blogs@localhost;

create user feministfiles@localhost identified by '$(cat ~/pwd/mysql-files)';
grant all on feministfiles.* to feministfiles@localhost;

create user feministforum@localhost identified by '$(cat ~/pwd/mysql-forum)';
grant all on feministforum.* to feministforum@localhost;

create user feministmail@localhost identified by '$(cat ~/pwd/mysql-mail)';
grant all on feministmail.* to feministmail@localhost;

create user feministwiki@localhost identified by '$(cat ~/pwd/mysql-wiki)';
grant all on feministwiki.* to feministwiki@localhost;

create user fff@localhost identified by '$(cat ~/pwd/mysql-fff)';
grant all on fff.* to fff@localhost;
EOF

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop mariadb
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!

FeministWiki:Server setup

2021-08-20T00:12:30Z

Technician : /* Install server components */

'''!!! WORK IN PROGRESS !!!'''

These are the steps required to set up a new FeministWiki Debian server.

== Initial setup of the new server ==

This section describes various initialization tasks for the new server that are independent of the old server.

=== Make feministwiki.dev point to the new server ===

During setup and testing of the new server, we want to make it accessible under the '''feministwiki.dev''' domain. So change the <code>A</code> entry of the feministwiki.dev DNS settings to point to the IP address of the new server.

=== Update & upgrade ===

First of all, let's make sure the system is up to date.

apt-get update
apt-get upgrade
apt-get dist-upgrade

=== Install miscellaneous tools ===

Some of these are needed further down, some are just good to have.

apt-get install automysqlbackup
apt-get install certbot
apt-get install dnsutils
apt-get install emacs
apt-get install git
apt-get install mg
apt-get install moreutils
apt-get install net-tools
apt-get install nmap
apt-get install rsync
apt-get install software-properties-common
apt-get install tree

=== Fetch scripts & config repo ===

Set up GitHub ssh access by copying the <code>.ssh/id_rsa</code> from the old server. After that:

cd ~
git clone git@github.com:FeministWiki/FeministWiki.git repo
cp -a repo/root/* repo/root/.??* .
sh repo/decrypt-pwd.sh

The decryption script will prompt you for a password the first time it's used. Enter the password stored in <code>/root/pwd/meta</code> on the old server.

=== Set up firewall ===

For now, block everything but SSH.

apt-get install ufw
ufw allow proto tcp to 0.0.0.0/0 port 22
ufw enable

=== Enable extra repositories ===

We might want to add some additional package repositories so we can use the latest version of some of the used software.

Backports is always OK to add since the packages don't get priority over the stable ones:

echo deb http://deb.debian.org/debian $(lsb_release -sc)-backports main > /etc/apt/sources.list.d/backports.list

PHP repo '''only''' if a very new version is needed:

wget -O /etc/apt/trusted.gpg.d/sury-php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/sury-php.list

MariaDB repo '''only''' if a very new version is needed:

wget https://mariadb.org/mariadb_release_signing_key.asc
apt-key add mariadb_release_signing_key.asc
rm mariadb_release_signing_key.asc
echo "deb http://mirror.23media.de/mariadb/repo/10.4/debian $(lsb_release -sc) main" > /etc/apt/sources.list.d/mariadb.list

=== Install server components ===

Now we can install all the software used for the various FeministWiki services:

php_version=7.4 # or whatever version we're on

apt-get install apache2
apt-get install dovecot-core
apt-get install dovecot-imapd
apt-get install dovecot-pop3d
apt-get install ejabberd # good candidate for backports
apt-get install fail2ban
apt-get install inspircd
apt-get install mailman
apt-get install mariadb-server
apt-get install opendkim
apt-get install php${php_version}
apt-get install php${php_version}-mbstring
apt-get install php${php_version}-mysql
apt-get install php${php_version}-xml
apt-get install postfix
apt-get install slapd

If any installation asks you for a password, remember that most passwords are found in <code>/root/pwd</code>.

Example for installing ejabberd from backports instead:

apt-get install ejabberd/$(lsb_release -sc)-backports # e.g. ejabberd/buster-backports

=== Enable Apache modules ===

We need a number of Apache modules to be enabled which might not be enabled by default:

a2enmod expires
a2enmod headers
a2enmod macro
a2enmod rewrite
a2enmod ssl

=== Create vmail user ===

groupadd -g 5000 vmail
useradd -u 5000 -g vmail -s /usr/sbin/nologin -d /home/vmail -m vmail

=== Put config files in place ===

The principle is simple: take all the config files from {{C|/root/repo/etc}} and put them where they belong in {{C|/etc}}. However, since a new server might mean much newer software, it's possible that some config files aren't compatible anymore, or that some new sensible defaults might be overwritten by the old config. Sadly figuring out these incompatibilities is a manual process: compare the new default config with the old default config and to our current config, to figure out what our new config should look like.

There's a number of things important to remember however:
* After copying in the new {{C|/etc/aliases}} file, run {{C|newaliases}} for the changes to take effect
* After populating {{C|/etc/letsencrypt/renewal-hooks}}, remember to {{C|chmod +x}} all the scripts

=== Initialize LetsEncrypt ===

First, initialize the certbot configuration:

certbot register -n --agree-tos -m technician@feministwiki.org

Since various DNS entries still point to the old server, we can't get a cert for the real domains yet. For now, just get one for feministwiki.dev:

ufw allow 80
letsencrypt-refresh --dev-only
ufw delete allow 80

Our <code>letsencrypt-refresh</code> script makes sure that the cert files are found in <code>/etc/fw-certs</code> and that the private key and cert-and-key bundle are owned by the "ssl-cert" group and are readable by group members. A number of users have to be added to this group so they can read said files:

adduser ejabberd ssl-cert
adduser irc ssl-cert

== Copying over live data ==

We want to make a first run of this copy process purely for testing purposes, before shutting down the services on the old server and repeating it to ensure integrity of live data.

=== Copy over /var/www ===

This is very simple but takes a lot of time to finish. '''Run it from the old server:'''

rsync -a --delete /var/www/ root@feministwiki.dev:/var/www

Note that the trailing slash in <code>/var/www/</code> is important; if not provided, it will copy the directory to <code>/var/www/www</code> on the new server.

While this command is running, you can do the next two steps in parallel.

=== Copy over SQL databases ===

Run the following command from the old server:

mysqldump -u root -p"$(cat /root/pwd/mysql)" \
--add-drop-database \
--databases blogs \
feministblog \
feministfiles \
feministforum \
feministmail \
feministsocial \
feministwiki \
feministwiki_de \
feministwiki_es \
feministwiki_it \
feministwiki_pt \
fff \
| ssh root@feministwiki.dev 'mysql -u root -p"$(cat /root/pwd/mysql)"'

You can use the <code>show databases;</code> command in the SQL console to make sure that the list of databases is complete. Unfortunately they have to be listed manually, because using the <code>--all-databases</code> option includes system databases that we don't want to copy.

While the above command is still running, you can do the next step in parallel.

=== Copy over LDAP databases ===

Stop the LDAP server and delete the existing configuration and data '''on the new server (careful!)''':

# Commands to run on the NEW (fresh) server:
systemctl stop slapd
rm -r /etc/ldap/slapd.d/*
rm /var/lib/ldap/data.mdb

Then copy over the config and data by running these commands from the old server:

slapcat -n 0 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 0 -F /etc/ldap/slapd.d'
slapcat -n 1 | ssh feministwiki.dev 'sudo -u openldap slapadd -n 1'

Note:

'''There might be incompatible changes between OpenLDAP (aka <code>slapd</code>) versions which require manual editing of the <code>slapcat</code> output before it's read in on the new server with <code>slapadd</code>.'''

Here's one example that occurs when updating from OpenLDAP 2.4.42 or earlier to 2.4.43 or later: the ppolicy overlay has a new attribute in the newer version, so if you simply run the commands above, the first one (the one that copies the config database) will produce the following error message:

User Schema load failed for attribute "pwdMaxRecordedFailure". Error code 17: attribute type undefined

The solution is as follows:

# On the new server, open <code>/etc/ldap/schema/ppolicy.ldif</code> and search for <code>pwdMaxRecordedFailure</code>. You will note that there is a <code>olcAttributeTypes: ...</code> entry that defines it, and also it's listed in the <code>MAY</code> attributes block of the <code>olcObjectClasses: ...</code> entry that defines the <code>pwdPolicy</code> object class.
# On the old server, save the output of <code>slapcat -n 0</code> to a file, open the file, and search for the block where the <code>ppolicy</code> schema is defined. It should start with the line <code>dn: cn={4}ppolicy,cn=schema,cn=config</code> (the <code>{4}</code> part might contain a different integer, that's OK). There, note that the <code>olcAttributeTypes: ...</code> entry for <code>pwdMaxRecordedFailure</code> is missing, and also it's not listed in the <code>MAY</code> list of the <code>pwdPolicy</code> object class definition. Copy over the attribute type definition from the <code>ppolicy.ldif</code> file on the new server, and amend the <code>MAY</code> list to include it.

The above is explained only for instructive purposes, since this particular fix will already have been applied by the time someone reads this guide. It's meant to give you an idea as to how backwards incompatible changes in OpenLDAP schema files can be amended when migrating to a newer version. (Also, no such clear explanation of the fix seems to be found anywhere on the web, so maybe someone who searches the error message above will come upon this guide and be happy!)

=== Mailman data ===

GNU Mailman uses a filesystem-based "database" so we can transfer over its data as follows; run this from the old server:

cd /var/lib/mailman
tar -czf - archives data lists | ssh root@feministwiki.dev 'cd /var/lib/mailman && tar -xzf - && check_perms -f'

The {{C|check_perms}} command, which is part of GNU Mailman, will take care of fixing the group ownership of the extracted files.

== Test ==

It's important to test the new server to make sure everything works well!

=== Reboot ===

We could restart a lot of services manually to ensure they've read their new config, but it's easiest to just reboot. (The new server, obviously.)

=== Open ports ===

We need to open all the ports used by the various FeministWiki services:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw allow proto tcp to 0.0.0.0/0 port $port
done

=== Test! ===

At this point you should test everything using the feministwiki.dev domain name.

Some things may not work because they're hard-coded to work as "feministwiki.org" and not under the "feministwiki.dev" name. This is a point of future improvement: all the services should be configured, if at all possible, in a way that they will work when invoked as feministwiki.dev just as well.

== Finishing up ==

'''Now, all services on the old server should be stopped, because we will begin the final transfer of live data.'''

=== Stop services on the old server ===

Stop all the services that interface with users and/or are responsible for modifying live data:

systemctl stop apache2
systemctl stop dovecot
systemctl stop ejabberd
systemctl stop inspircd
systemctl stop mailman
systemctl stop mariadb
systemctl stop postfix
systemctl stop slapd

Close all the relevant ports just to be double-sure:

for port in 25 80 443 465 587 993 995 5222 5223 5269 5270 5443 6697 7777
do ufw delete allow proto tcp to 0.0.0.0/0 port $port
done

=== Copy over the live data one more time ===

The techniques and commands described above in the section '''Copying over live data''' are ''idempotent'', meaning you can simply repeat them and they will make sure that the new copy of the live data is fresh and doesn't leave any outdated data on the new server. (For instance, the <code>--delete</code> argument to the <code>rsync</code> command and the <code>--add-drop-database</code> argument to the <code>mysqldump</code> command help to make sure of this.)

So in short, just repeat the steps from that section exactly one more time.

=== Update DNS entries ===

You have to change the configuration of the following domains:

* feministwiki.org
* feministwiki.net
* feministwiki.de
* fem.wiki
* fffrauen.de

==== feministwiki.org ====

You only have to change three DNS entries, since most of the subdomains work via CNAME entries:

* The main <code>A</code> entry for <code>@</code> (self-reference i.e. feministwiki.org)
* The <code>A</code> entry for <code>smtp</code> since this is not allowed to be a CNAME
* The <code>A</code> entry for <code>xmpp</code> since this is not allowed to be a CNAME

==== feministwiki.net, feministwiki.de, fem.wiki, fffrauen.de ====

For these, you only have to change the main <code>A</code> entry, since they don't use SMTP or XMPP.

=== Update the certificate ===

Run the <code>letsencrypt-refresh</code> script to get a new certificate which includes all our domain names, since we had started out with just feministwiki.dev.

After this, everything should be functional. If not, it's time for some debugging!