Releasing new versions

In order to release new version you need to be owner of all the gems at RubyGems, ask one of the owners to add you before releasing. Try gem owner decidim to find out the owners of the gem. It is worth making sure you are owner of all gems.

Remember to follow the Gitflow branching workflow.

Create the stable branch for the release

  1. Merge all the Crowdin pull requests created by the user decidim-bot, specially the one that is going to be marged against develop that should be returned by the provided example search.

  2. Go to develop with git checkout develop

  3. Create the release branch git checkout -b release/x.y-stable && git push origin release/x.y-stable.

  4. Add the release branch to Crowdin so that any pending translations will generate a PR to this branch. This needs to be done with the generic user decidim in the GitHub’s Crowdin integration, as this is the user that initially configured the installation and other admins in the Crowdin organization cannot change it. See instructions for this on the "Create the stable branch in Crowdin". Note that after the branch is configured at Crowdin, it takes some time for the synchronization to happen, so be prepared to wait a couple of hours.

Mark develop as the reference to the next release:

  1. Go back to develop with git checkout develop

  2. Turn develop into the dev branch for the next release:

    1. Update .decidim-version to the new dev development version: x.y.z.dev, where x.y.z is the new semver number for the next version

    2. Run bin/rake update_versions, this will update all references to the new version.

    3. Run bin/rake patch_generators, this will update the Gemfile for the generators to the new version.

    4. Run bin/rake bundle, this will update all the Gemfile.lock files

    5. Run bin/rake webpack, this will update the JavaScript bundle.

  3. Update SECURITY.md and change the supported version to the new version.

  4. Update the CHANGELOG.md. At the top you should have an Unreleased header and under that the content Nothing.. The CHANGELOG is automatically generated for the release branches so do not worry about its actual contents at the develop branch. After that, the header with the current version and link with the same beforementioned sections and a Previous versions header at the bottom that links to the previous minor release stable branch.

# Changelog

## [Unreleased](https://github.com/decidim/decidim/tree/HEAD)

Nothing.

...

## Previous versions

Please check [0.XX-stable](https://github.com/decidim/decidim/blob/release/0.XX-stable/CHANGELOG.md) for previous changes.

  1. Update the RELEASE_NOTES.md You can use the following template for the new RELEASE_NOTES.md:

# Release Notes

## 1. Upgrade notes

As usual, we recommend that you have a full backup, of the database, application code and static files.

To update, follow these steps:

### 1.1. Update your Gemfile

```ruby
gem "decidim", github: "decidim/decidim"
gem "decidim-dev", github: "decidim/decidim"
```

### 1.2. Run these commands

```console
bundle update decidim
bin/rails decidim:upgrade
bin/rails db:migrate
```

### 1.3. Follow the steps and commands detailed in these notes

## 2. General notes

## 3. One time actions

These are one time actions that need to be done after the code is updated in the production database.

### 3.1. [[TITLE OF THE ACTION]]

You can read more about this change on PR [\#XXXX](https://github.com/decidim/decidim/pull/XXXX).

## 4. Scheduled tasks

Implementers need to configure these changes it in your scheduler task system in the production server. We give the examples
 with `crontab`, although alternatively you could use `whenever` gem or the scheduled jobs of your hosting provider.

### 4.1. [[TITLE OF THE TASK]]

```bash
4 0 * * * cd /home/user/decidim_application && RAILS_ENV=production bundle exec rails decidim:TASK
```

You can read more about this change on PR [\#XXXX](https://github.com/decidim/decidim/pull/XXXX).

## 5. Changes in APIs

### 5.1. [[TITLE OF THE CHANGE]]

In order to [[REASONING (e.g. improve the maintenance of the code base)]] we have changed...

If you have used code as such:

```ruby
# Explain the usage of the API as it was in the previous version
result = 1 + 1 if before
```

You need to change it to:

```ruby
# Explain the usage of the API as it is in the new version
result = 1 + 1 if after
```

  1. Push the changes git add . && git commit -m "Bump develop to next release version" && git push origin develop

Create the stable branch in Crowdin

  1. You will need to first create the version branch in Crowdin

    1. Sign in as manager in Crowdin.

    2. Go to the Content tab in the Decidim project

    3. Click on the dropdown of the "New folder" button (top right) and select "New Version Branch"

    4. On the modal, add the name of the branch. As it does not allow the slash character in the name ("/") you will need to change it to a dot ("."). So, for instance, "release/0.20-stable" would become "release.0.20-stable". See other releases if in doubt, it should be consistent.

    5. After creating the version branch, edit it and change the "Title as it appears to translators" to the name with the slash ("release/0.20-stable")

  2. Then you will be able to setup the GitHub integration in Crowdin

    1. Sign in as user "Decidim" in Crowdin. NOTE: this user is the only one that can do this.

    2. Go to the Integrations tab in the Decidim project

    3. Click on GitHub

    4. Click on the "Edit" button

    5. In the "Select Branches for Translation", search the branch. Click on it.

    6. Change the "Service Branch Name" to be consistent with the other branches. So, for instance, "release/0.20-stable" would become "chore/l10n/release/0.20-stable".

    7. Click on "Save"

    8. After a couple of hours, if there are any changes in Crowdin that would involve this branch, you will see the correspondent "Decidim bot" Pull Request in GitHub.

Producing the CHANGELOG.md

Look for the "Bump develop to next release version" commit sha1. You can do it either visually with gitk .decidim-version or by blaming git blame .decidim-version.

Here you have different options to see what happened from one revision to another:

git log v0.20.0..v0.20.1 --grep " (#[0-9]\+)" --oneline
git log <SHA>..HEAD --grep " (#[0-9]\+)" --oneline

Once you have checked the list of changes, it is time to actually generating the changelog.

bin/changelog_generator

In order to generate the changelog, you need to know the SHA hash of the first commit that was not part of the previous release. You can check the commit hash by inspecting the commit log of the .decidim-version file as follows when in the correct release branch:

git log -1 --format=oneline .decidim-version

Alternatively, you can find the first commit after the point of time that the two release branches have separated from each other as follows:

git log --reverse --pretty=format:"%H" $(git merge-base release/0.XX-stable release/0.YY-stable)..release/0.YY-stable | head -1

In the above command, replace 0.XX with the previous release and 0.YY with the current release you are generating the change log for. This command works only for major releases, not for patch or bugfix releases.

Running it as is, or passing it the --help flag, will render the help section for the script. In order to actually run the script, follow the instructions:

bin/changelog_generator <GITHUB_TOKEN> <SHA>

This command will create a temporary_changelog.md in the root of the project, so you can inspect this file and generated changelog.

If you have some elements in the Unsorted section of the output, you can review the PRs individually, fix the title and the tags and rerun the script. Those PRs with the tags fixed will be automatically sorted. Labelling the PRs as they are opened or merged is encouraged to save some time when producing the changelog.

You can copy-paste the contents of the temporary changelog file to the relevant sections of the Changelog file.

Release Candidates

Release Candidates are the same as beta versions. They should be ready to go to production, but publicly released just before in order to be widely tested.

If this is a Release Candidate version release, the steps to follow are:

  1. Merge all the Crowdin pull requests created by the user decidim-bot, specially the one that is going to be marged against the release branch release/x.y-stable that should be returned by the provided example search (pick the correct pull request for the release from the results).

  2. Checkout the release stable branch git checkout release/x.y-stable.

  3. Update .decidim-version to the new version x.y.z.rc1

  4. Run bin/rake update_versions, this will update all references to the new version.

  5. Run bin/rake patch_generators, this will update the Gemfile for the generators to the new version.

  6. Run bin/rake bundle, this will update all the Gemfile.lock files

  7. Run bin/rake webpack, this will update the JavaScript bundle.

  8. Run bin/rspec, this will check things like if all the officially supported languages translations are OK.

  9. Commit all the changes: git add . && git commit -m "Bump to rcXX version" && git push origin release/x.y-stable.

  10. Wait for the tests to finish and check that everything is passing before releasing the version. NOTE: When you bump the version, the generator tests will fail because the gems and NPM packages have not been actually published yet (as in sent to rubygems/npm). You may see errors such as No matching version found for @decidim/browserslist-config@~0.xx.y in the CI logs. This should be fine as long as you have ensured that the generators tests passed in the previous commit.

  11. Run git pull && bin/rake release_all, this will create all the tags, push the commits and tags and release the gems to RubyGems.

Usually, at this point, the release branch is deployed to Metadecidim during, at least, one week to validate the stability of the version.

During the validation period

  1. During the validation period, bugfixes must be implemented directly to the current release/x.y.z-stable branch and ported to develop.

  2. During the validation period, translations to the officially supported languages must be added to Crowdin and, when completed, merged into release/x.y.z-stable.

Major/Minor versions

Release Candidates will be tested in a production server (usually Metadecidim) during some period of time (a week at least). When they are considered ready, it is time for them to be released:

  1. Merge all the Crowdin pull requests created by the user decidim-bot, specially the one that is going to be marged against the release branch release/x.y-stable that should be returned by the provided example search (pick the correct pull request for the release from the results).

  2. Checkout the release stable branch git checkout release/x.y-stable.

  3. Update .decidim-version by removing the .rcN suffix, leaving a clean version number like x.y.z

  4. Run bin/rake update_versions, this will update all references to the new version.

  5. Run bin/rake patch_generators, this will update the Gemfile for the generators to the new version.

  6. Run bin/rake bundle, this will update all the Gemfile.lock files

  7. Run bin/rake webpack, this will update the JavaScript bundle.

  8. Update the CHANGELOG.md. Add the header with the current version and link like ## [0.20.0](https://github.com/decidim/decidim/tree/v0.20.0) and the headers generated by the changelog generator.

  9. Commit all the changes: git add . && git commit -m "Bump to v0.XX.0 final version" && git push origin release/x.y-stable.

  10. Wait for the tests to finish and check that everything is passing before releasing the version. NOTE: When you bump the version, the generator tests will fail because the gems and NPM packages have not been actually published yet (as in sent to rubygems/npm). You may see errors such as No matching version found for @decidim/browserslist-config@~0.xx.y in the CI logs. This should be fine as long as you have ensured that the generators tests passed in the previous commit.

  11. Run git pull && bin/rake release_all, this will create all the tags, push the commits and tags and release the gems to RubyGems.

  12. Once all the gems are published:

    1. Re-run the failed generators tests at the release branch.

    2. Create a new release at this repository, just go to the releases page and create a new one.

  13. Update Decidim’s Docker repository as explained in the Docker images section below.

  14. Update Crowdin synchronization configuration with Github:

    1. Add the new release/x.y-stable branch.

    2. Remove from Crowdin branches that are not officially supported anymore. That way they do not synchronize with Github.

  15. Update the version in the documentation system that we use, Antora. For the decidim repository this is done automatically when bumping the versions. For the documentation repository:

    1. Create and push the branch in that repository: git checkout release/0.XX-stable && git push origin release/x.y-stable

    2. Add the new version on the antora-playbook.yml branches key, for the both sources (decidim and documentation repository). Do not forget to also remove the oldest and change the new default in the start_page key. For instance for v0.28 this is how it’d look like:

diff --git a/antora-playbook.yml b/antora-playbook.yml
index 31be47c..75c7d5e 100644
--- a/antora-playbook.yml
+++ b/antora-playbook.yml
@@ -1,21 +1,21 @@
 site:
   title: Decidim Docs
   url: https://docs.decidim.org
-  start_page: v0.27@en:ROOT:index.adoc
+  start_page: v0.28@en:ROOT:index.adoc
 content:
   sources:
     - url:
       start_path: en
       branches:
+        - release/0.28-stable
         - release/0.27-stable
-        - release/0.26-stable
         - develop
       edit_url: "https://github.com/decidim/documentation/edit/{refname}/{path}"
     - url: https://github.com/decidim/decidim
       start_path: docs
       branches:
+        - release/0.28-stable
         - release/0.27-stable
-        - release/0.26-stable
         - develop
     - url: https://github.com/decidim/decidim-bulletin-board
       start_path: docs

After you commit this change in develop branch you will have to wait a couple minutes for the automatic deployment to see it live at https://docs.decidim.org.

Releasing patch versions

Releasing new versions from a release/x.y-stable branch is quite easy. The process is very similar from releasing a new Decidim version:

  1. Merge all the Crowdin pull requests created by the user decidim-bot, specially the one that is going to be marged against the release branch release/x.y-stable that should be returned by the provided example search (pick the correct pull request for the release from the results).

  2. Make sure that there are no more PRs to backport. Learn more about Backports.

  3. Checkout the branch you want to release: git checkout -b release/x.y-stable

  4. Update .decidim-version to the new version number.

  5. Run bin/rake update_versions, this will update all references to the new version.

  6. Run bin/rake patch_generators, this will update the Gemfile for the generators to the new version.

  7. Run bin/rake bundle, this will update all the Gemfile.lock files

  8. Run bin/rake webpack, this will update the JavaScript bundle.

  9. Update the CHANGELOG.md. Add the header with the current version and link like ## [0.20.0](https://github.com/decidim/decidim/tree/v0.20.0) and the headers generated by the changelog generator.

  10. Commit all the changes: git add . && git commit -m "Prepare VERSION release"

  11. Wait for the tests to finish and check that everything is passing before releasing the version. NOTE: When you bump the version, the generator tests will fail because the gems and NPM packages have not been actually published yet (as in sent to rubygems/npm). You may see errors such as No matching version found for @decidim/browserslist-config@~0.xx.y in the CI logs. This should be fine as long as you have ensured that the generators tests passed in the previous commit.

  12. Run git pull && bin/rake release_all, this will create all the tags, push the commits and tags and release the gems to RubyGems.

  13. Once all the gems are published:

    1. Re-run the failed generators tests at the release branch.

    2. Create a new release at this repository, just go to the releases page and create a new one.

  14. Update Decidim’s Docker repository as explained in the Docker images section.

Docker images for each release

Each release triggers a GitHub workflow that rebuilds and publishes the decidim/docker images to GitHub Container Registry and Docker Hub.