Release process

This page describes the current status of the release process.

Since 6.30.0, the automated release process is using Github Actions.

However, there are still a few steps, that need manual examination.

Overview

This page gives an overview which tasks are automated to do a full release of PMD. This knowledge is required in order to verify that the release was successful or in case the automated process fails for some reason. Then individual steps need to be executed manually. Because the build is reproducible, these steps can be repeated again if the same tag is used.

There is one special case in this project: The release of PMD is done in two steps:

1. All modules except pmd-cli and pmd-dist are released. That means, pmd-core and all the language modules are released. This is, so that these libs can be used by pmd-designer to create a new release.
2. pmd-cli and pmd-dist are released afterwards. Both depend on pmd-designer, and this two-step release process is used for now to break the cycling release dependency.

The three main steps are:

• Preparations (which creates the tags) - use do-release.sh for that
• The actual release (which is automated) - GitHub Actions will build the tags when they have been pushed.
• Prepare the next release (make sure the current main branch is ready for further development)

Preparations

This is the first step. It is always manual and is executed locally. It creates in the end the tag from which the release is created.

Make sure code is up-to-date and everything is committed and pushed with git:

$ ./mvnw clean
$ git pull
$ git status

As a help for the preparation task, the script do-release.sh guides you through the preparation tasks and the whole release process. The script requires a specific source code folder and additional checkouts locally, e.g. it requires that the repo pmd.github.io is checked out aside the main pmd repo:

• https://github.com/pmd/pmd ➡️ /home/joe/source/pmd
• https://github.com/pmd/pmd.github.io ➡️ /home/joe/source/pmd.github.io

The script do-release.sh is called in the directory /home/joe/source/pmd and searches for ../pmd.github.io.

Also make sure, that the repo “pmd.github.io” is locally up-to-date and has no local changes.

The Release Notes and docs

Before the release, you need to verify the release notes: Does it contain all the relevant changes for the release? Is it formatted properly? Are there any typos? Does it render properly?

As the release notes are part of the source code, it is not simple to change it afterwards. While the source code for a tag cannot be changed anymore, the published release notes on the github releases pages or the new posts can be changed afterwards (although that’s an entirely manual process).

You can find the release notes here: docs/pages/release_notes.md.

The date (date +%d-%B-%Y) and the version (remove the SNAPSHOT) must be updated in docs/_config.yml, e.g. in order to release version “6.34.0”, the configuration should look like this:

pmd:
    version: 6.34.0
    previous_version: 6.33.0
    date: 24-April-2021
    release_type: minor

The release type could be one of “bugfix” (e.g. 6.34.x), “minor” (6.x.0), or “major” (x.0.0).

The release notes usually mention any new rules that have been added since the last release. Please double-check the file pmd-core/src/main/resources/rulesets/releases/<version>.xml, so that all new rules are listed.

Add the new rules as comments to the quickstart rulesets:

• pmd-apex/src/main/resources/rulesets/apex/quickstart.xml
• pmd-java/src/main/resources/rulesets/java/quickstart.xml

The designer lives at pmd/pmd-designer. Update property pmd-designer.version in pom.xml to reference the new version, that will be released shortly. Note: This version does at the moment not exist.

Starting with PMD 6.23.0 we’ll provide small statistics for every release. This needs to be added to the release notes as the last section. To count the closed issues and pull requests, the milestone on GitHub with the title of the new release is searched. Make sure, there is a milestone on https://github.com/pmd/pmd/milestones. The following snippet will create the numbers, that can be attached to the release notes as a last section:

LAST_VERSION=6.33.0
NEW_VERSION=6.34.0
NEW_VERSION_COMMITISH=HEAD

echo "### Stats"
echo "* $(git log pmd_releases/${LAST_VERSION}..${NEW_VERSION_COMMITISH} --oneline --no-merges |wc -l) commits"
echo "* $(curl -s "https://api.github.com/repos/pmd/pmd/milestones?state=all&direction=desc&per_page=5"|jq ".[] | select(.title == \"$NEW_VERSION\") | .closed_issues") closed tickets & PRs"
echo "* Days since last release: $(( ( $(date +%s) - $(git log --max-count=1 --format="%at" pmd_releases/${LAST_VERSION}) ) / 86400))"

Note: this part is also integrated into do-release.sh.

Check in all (version) changes to branch master or any other branch, from which the release takes place:

$ git commit -a -m "Prepare pmd release <version>"
$ git push

The Homepage

The github repo pmd.github.io hosts the homepage for https://pmd.github.io. All the following tasks are to be done in this repo.

The new version needs to be entered into _config.yml, e.g.:

pmd:
  latestVersion: 6.34.0
  latestVersionDate: 24-April-2021

Also move the previous version down into the “downloads” section. We usually keep only the last 3 versions in this list, so remove the oldest version.

Then create a new page for the new release, e.g. _posts/2021-04-24-PMD-6.34.0.md and copy the release notes into this page. This will appear under the news section.

Check in all (version) changes to branch master:

$ git commit -a -m "Prepare pmd release <version>"
$ git push

The actual release

The actual release starts with one last local command: calling maven-release-plugin.

This plugin changes the version by basically removing the “-SNAPSHOT” suffix, builds the changed project locally, commits the version change, creates a new tag from this commit, changes the version of the project to the next snapshot, commits this change and pushes everything.

RELEASE_VERSION is the version of the release. It is reused for the tag. DEVELOPMENT_VERSION is the next snapshot version after the release.

RELEASE_VERSION=6.34.0
DEVELOPMENT_VERSION=6.35.0-SNAPSHOT
./mvnw -B release:clean release:prepare \
    -Dtag="pmd_releases/${RELEASE_VERSION}" \
    -DreleaseVersion="${RELEASE_VERSION}" \
    -DdevelopmentVersion="${DEVELOPMENT_VERSION}" \
    -DscmCommentPrefix="[release] " \
    -Darguments='-Pgenerate-rule-docs,!cli-dist' \
    '-Pgenerate-rule-docs,!cli-dist'

Once the maven plugin has pushed the tag, github actions will start and build a new version from this tag. Since it is a tag build and a released version build, the build script will do a couple of additional stuff. This is all automated in .ci/build.sh.

Note: The profile “cli-dist” is deactivated, so this release command doesn’t include pmd-cli and pmd-dist. They will be released separately after pmd-designer is released. Since pmd-dist is not included in this first step, no binaries are created yet.

Here is, what happens:

• Deploy and release the build to maven central, so that it can be downloaded from https://repo.maven.apache.org/maven2/net/sourceforge/pmd/pmd/. This is done automatically, if all unit tests pass and the build doesn’t fail for any other reason. The plugin nexus-staging-maven-plugin is used for that.
• Create a draft release on GitHub and upload the release notes from docs/pages/release_notes.md. Note: During the process, the release is a draft mode and not visible yet. At the end of the process, the release will be published.
• Render the documentation in docs/ with bundle exec jekyll build and create a zip file from it.
• Upload the doc zip file to the current github release under https://github.com/pmd/pmd/releases and to https://sourceforge.net/projects/pmd/files/pmd/.
• Upload the documentation to https://docs.pmd-code.org, e.g. https://docs.pmd-code.org/pmd-doc-6.34.0/ and create a symlink, so that https://docs.pmd-code.org/latest/ points to the new version.
• Remove the old snapshot documentation, e.g. so that https://docs.pmd-code.org/pmd-doc-6.34.0-SNAPSHOT/ is gone. Also create a symlink from pmd-doc-6.34.0-SNAPSHOT to pmd-doc-6.34.0, so that old references still work, e.g. https://docs.pmd-code.org/pmd-doc-6.34.0-SNAPSHOT/ points to the released version.
• Deploy javadoc to “https://docs.pmd-code.org/apidocs/*/RELEASE_VERSION/”, e.g. https://docs.pmd-code.org/apidocs/pmd-core/6.34.0/. This is done for all modules.
• Remove old javadoc for the SNAPSHOT version, e.g. delete https://docs.pmd-code.org/apidocs/pmd-core/6.34.0-SNAPSHOT/.
• Create a draft news post on https://sourceforge.net/p/pmd/news/ for the new release. This contains the rendered release notes.
• Copy the documentation to sourceforge’s web space, so that it is available as https://pmd.sourceforge.io/pmd-6.34.0/. All previously copied versions are listed under https://pmd.sourceforge.io/archive.phtml.

The release on github actions currently takes about 30-45 minutes. Once this is done, you can proceed with releasing pmd designer, see https://github.com/pmd/pmd-designer/blob/master/releasing.md. Make sure to release the version, you have used earlier for the property pmd-designer.version.

Once the pmd-designer release is done, you can proceed with part 2. We’ll checkout the release tag, add a new commit with the changed versions for pmd-cli and pmd-dist on top of it and create a new tag:

git checkout "pmd_releases/${RELEASE_VERSION}"
./mvnw versions:update-parent -DparentVersion="${RELEASE_VERSION}" -DskipResolution=true -DgenerateBackupPoms=false -pl pmd-cli,pmd-dist
git add pmd-cli/pom.xml pmd-dist/pom.xml
git commit -m "[release] prepare release pmd_releases/${RELEASE_VERSION}-dist"
git tag -m "[release] copy for tag pmd_releases/${RELEASE_VERSION}-dist" "pmd_releases/${RELEASE_VERSION}-dist"
git push origin tag "pmd_releases/${RELEASE_VERSION}-dist"
git checkout master
# make sure parent reference is correct
./mvnw versions:update-parent -DparentVersion="${DEVELOPMENT_VERSION}" -DskipResolution=true -DgenerateBackupPoms=false -pl pmd-cli,pmd-dist
git add pmd-cli/pom.xml pmd-dist/pom.xml
git commit -m "Prepare next development version [skip ci]"
git push origin "${CURRENT_BRANCH}"

Since pmd-cli/pmd-dist were not part of the first maven-release-plugin call, we might need to fix the parent references manually to set to the new development version.

The new created tag ends with the suffix -dist, and this is used as a marker for the GitHub action. The same build script .ci/build.sh is executed, but now, it does the following steps:

• Build only modules pmd-cli and pmd-dist
• Upload the new binaries to the existing draft release under https://github.com/pmd/pmd/releases.
• Upload the new binaries additionally to sourceforge, so that they can be downloaded from https://sourceforge.net/projects/pmd/files/pmd/, including the release notes.
• After all this is done, the release on github (https://github.com/pmd/pmd/releases) is published and the news post on sourceforge (https://sourceforge.net/p/pmd/news/> is published as well.
• The new binary at https://sourceforge.net/projects/pmd/files/pmd/ is selected as the new default for PMD.
• As a last step, a new baseline for the regression tester is created and uploaded to https://pmd-code.org/pmd-regression-tester.

Once this second GitHub Action run is done, you can spread additional news:

• Write an email to the mailing list

  To: PMD Developers List pmd-devel@lists.sourceforge.net Subject: [ANNOUNCE] PMD released

  • Downloads: https://github.com/pmd/pmd/releases/tag/pmd_releases%2F
  • Documentation: https://docs.pmd-code.org/pmd-doc-/

  And Copy-Paste the release notes

• Tweet about the new release

Tweet on https://twitter.com/pmd_analyzer, eg.:

PMD 6.34.0 released: https://github.com/pmd/pmd/releases/tag/pmd_releases/6.34.0 #PMD

Checklist

Prepare the next release

There are a couple of manual steps needed to prepare the current main branch for further development.

• Move any open issues to the next milestone, close the current milestone on https://github.com/pmd/pmd/milestones and create a new one for the next version (if one doesn’t exist already).

• Update version in docs/_config.yml. Note - the next version needs to have a SNAPSHOT in it otherwise the javadoc links won’t work during development.

  pmd:
      version: 6.35.0-SNAPSHOT
      previous_version: 6.34.0
      date: ??-??-2021
      release_type: minor
  

• Prepare a new empty release notes. Note, this is done by do-release.sh already.
  • Move version/release info from docs/pages/release_notes.md to docs/pages/release_notes_old.md.
  • Update version/release info in docs/pages/release_notes.md. Use the following template:

---
title: PMD Release Notes
permalink: pmd_release_notes.html
keywords: changelog, release notes
---

## {{ site.pmd.date }} - {{ site.pmd.version }}

The PMD team is pleased to announce PMD {{ site.pmd.version }}.

This is a {{ site.pmd.release_type }} release.

{% tocmaker %}

### New and noteworthy

### Fixed Issues

### API Changes

### External Contributions

{% endtocmaker %}

Finally commit and push the changes:

$ git commit -m "Prepare next development version"
$ git push origin master

Branches

Merging

If the release was done on a maintenance branch, such as pmd/5.4.x, then this branch should be merged into the next “higher” branches, such as pmd/5.5.x and master.

This ensures, that all fixes done on the maintenance branch, finally end up in the other branches. In theory, the fixes should already be there, but you never now.

Multiple releases

If releases from multiple branches are being done, the order matters. You should start from the “oldest” branch, e.g. pmd/5.4.x, release from there. Then merge (see above) into the next branch, e.g. pmd/5.5.x and release from there. Then merge into the master branch and release from there. This way, the last release done, becomes automatically the latest release on https://docs.pmd-code.org/latest/ and on sourceforge.

(Optional) Create a new release branch

At some point, it might be time for a new maintenance branch. Such a branch is usually created from the master branch. Here are the steps:

• Create a new branch: git branch pmd/5.6.x master
• Update the version in both the new branch and master, e.g. mvn versions:set -DnewVersion=5.6.1-SNAPSHOT and mvn versions:set -DnewVersion=5.7.0-SNAPSHOT.
• Update the release notes on both the new branch and master

The maintenance or bugfix branch could also be created later when needed from the actual tag. Then only the version on the maintenance branch needs to be set.