C1. Comment out all builds in crontab on central BioC 3.20 builder

IMPORTANT: No 3.20 build should start until:

• the bump and branch process is complete,

• and BBS_BIOC_GIT_BRANCH is switched from devel to RELEASE_3_20 on the central BioC 3.20 builder (this switch is documented in section F3 below).

Maybe no build is scheduled to start during the next 3 hours, in which case everything is fine. Otherwise, comment out all the builds that are at the risk of starting before step F3 below is completed. Commenting out the prerun and run entries for each build should be enough.

C2. Ask people to stop committing/pushing changes to the BioC git server

Announce or ask a core team member to announce on the bioc-devel mailing list that people must stop committing/pushing changes to the BioC git server (git.bioconductor.org) for the next 3 hours.

C3. Modify packages.conf to block all commits

The RELEASE_3_19 lines in gitolite-admin/conf/packages.conf were commented out when the release builds were frozen. At this point, only the devel lines are still active.

Deactivate all push access by commenting out the devel lines in gitolite-admin/conf/packages.conf.

NOTE: Do not change the branch from RELEASE_3_19 to RELEASE_3_20, it is not a good solution. Maintainers now will be able to push their own RELEASE_3_20 branch before we are able to create it at release time. This issue reflects the issue https://stat.ethz.ch/pipermail/bioc-devel/2019-May/015048.html.

Using vim, it is possible with a one liner:

:g/RW devel/s/^/#

Or Lori created sed commands see scripts/sedCommandsForVariousTasks.txt

Once packages.conf is updated, push to gitolite-admin on the git server to make the changes effective.

C4. Disable hooks

Log on to git.bioconductor.org as the git user.

• Comment out the hook lines in packages.conf.

• Remove the pre-receive.h00-pre-receive-hook-software file from the hook/ directory in each package, e.g., /home/git/repositories/packages/<PACKAGE>.git/hooks

  rm -rf ~/repositories/packages/*.git/hooks/pre-receive.h00-pre-receive-hook-software

Or (trying as of release 3.20) try using hooks exclusion file. See scripts/sedCommandsForVariousTasks.txt. This changes is made in the hook_maintainer repo.

C5. Log on the machine where you’ve performed the preliminary steps

Make sure to use the -A flag to enable forwarding of the authentication agent connection e.g.:

ssh -A [email protected]

See B. Preliminary steps above for the details.

C6. Checkout/update the RELEASE_3_20 branch of the manifest repo

cd ~/git.bioconductor.org/manifest
git pull --all
git checkout RELEASE_3_20
git branch
git status

C7. Set the BBS_HOME, WORKING_DIR and MANIFEST_FILE environment variables

• BBS_HOME:

  export BBS_HOME="$HOME/BBS"

IMPORTANT NOTE: The settings for WORKING_DIR and MANIFEST_FILE are specific to the type of packages. The following settings are for the software packages. Make sure to adapt for data-experiment packages, workflows, and books.

• WORKING_DIR: Point WORKING_DIR to the folder containing the software packages:

  export WORKING_DIR="$HOME/git.bioconductor.org/software"

  All the remaining steps in section C. must be performed from within this folder:

  cd $WORKING_DIR

• MANIFEST_FILE: Point MANIFEST_FILE to the manifest file for software packages. This must be the file from the RELEASE_3_20 branch of the manifest repo:

  export MANIFEST_FILE="$HOME/git.bioconductor.org/manifest/software.txt"

C8. Run bump_version_and_create_branch.py

We’re going to run the script on all the software packages. It will take care of applying and pushing the changes described in A. Introduction.

cd $WORKING_DIR
pkgs_in_manifest=`grep 'Package: ' $MANIFEST_FILE | sed 's/Package: //g'`

$BBS_HOME/utils/bump_version_and_create_branch.py --push RELEASE_3_20 $pkgs_in_manifest >bump_version_and_create_branch.log 2>&1 &

Alternatively, the bump_version_and_create_branch.py run above can be replaced with a 2-pass run:

# First pass (apply all the changes but do NOT push them):
$BBS_HOME/utils/bump_version_and_create_branch.py RELEASE_3_20 $pkgs_in_manifest >bump_version_and_create_branch.log1 2>&1 &
# Second pass (push all the changes):
$BBS_HOME/utils/bump_version_and_create_branch.py --push RELEASE_3_20 $pkgs_in_manifest >bump_version_and_create_branch.log2 2>&1 &

The 2-pass run can be useful if one wants to inspect the changes before pushing them.

Notes:

• You can follow progress with tail -f bump_version_and_create_branch.log.

• The BiocVersion package will automatically receive special treatment.

• In the 2-pass run, the second pass checks the packages and applies the changes only if needed (i.e. if a package does not already have the RELEASE_3_20 branch) before pushing the changes.

• If for some reason the bump_version_and_create_branch.py script stops prematurly, it can be safely re-run with the same arguments. This is because the script only does things to packages that need treatment. Packages that were already successfully treated will no longer be touched.

• A typical error is Error: duplicate commits (happened for affyPLM and Rdisop first time I tested this). Report these errors to gitolite experts Lori or Jen. Once the problem is fixed, re-run the script.

C9. Check bump_version_and_create_branch.log

After completion of the bump_version_and_create_branch.py run, open bump_version_and_create_branch.log and make sure everything looks ok.

D1. Data-experiment packages

Repeat steps C6 to C9 above but for C7 define the environment variables as follows:

export WORKING_DIR="$HOME/git.bioconductor.org/data-experiment"
export MANIFEST_FILE="$HOME/git.bioconductor.org/manifest/data-experiment.txt"

D2. Workflows

Repeat steps C6 to C9 above but for C7 define the environment variables as follows:

export WORKING_DIR="$HOME/git.bioconductor.org/workflows"
export MANIFEST_FILE="$HOME/git.bioconductor.org/manifest/workflows.txt"

D3. Books

Repeat steps C6 to C9 above but for C7 define the environment variables as follows:

export WORKING_DIR="$HOME/git.bioconductor.org/books"
export MANIFEST_FILE="$HOME/git.bioconductor.org/manifest/books.txt"

F1. Enable push access to new RELEASE_3_20 branch

This is done by editing the conf/packages.conf file in the gitolite-admin repo (git clone [email protected]:gitolite-admin).

• If not done already, replace all instances of RELEASE_3_19 with RELEASE_3_20.

• Uncomment all RELEASE_3_20 and devel lines.

See scripts/sedCommandsForVariousTasks.txt for help on re-enabling access

• If using hooks exclusion file for allowing version bump. Restore original hooks file.

• Run gitolite setup from /home/git/repositories to re-enable the hooks.

• Test that a non-super user can push access is enabled on the dummy package BiocGenerics_test.

  Check:

  git push
  git checkout RELEASE_3_20
  git pull

F2. Tell people that committing/pushing to the BioC git server can resume

Announce or ask a core team member to announce on the bioc-devel mailing list that committing/pushing changes to the BioC git server (git.bioconductor.org) can resume.

F3. Switch BBS_BIOC_GIT_BRANCH from devel to RELEASE_3_20 on central BioC 3.20 builder

DON’T FORGET THIS STEP! Its purpose is to make the BioC 3.20 builds grab the RELEASE_3_20 branch of all packages instead of their devel branch.

Log on the central BioC 3.20 builder as biocbuild and replace

export BBS_BIOC_GIT_BRANCH="devel"

with

export BBS_BIOC_GIT_BRANCH="RELEASE_3_20"

in ~/BBS/3.20/config.sh

Also replace

set BBS_BIOC_GIT_BRANCH=devel

with

set BBS_BIOC_GIT_BRANCH=RELEASE_3_20

in ~/BBS/3.20/config.bat

Then remove the manifest and MEAT0 folders from ~/bbs-3.20-bioc/, ~/bbs-3.20-data-annotation/, ~/bbs-3.20-data-experiment/, ~/bbs-3.20-workflows/, and ~/bbs-3.20-books/. They’ll get automatically re-created and re-populated when the builds start.

Once you’re done, DON’T FORGET TO UNCOMMENT ALL THE BUILDS THAT WERE TEMPORARILY COMMENTED OUT IN THE CRONTAB (see section C1 above).

F4. Sync all core Bioconductor packages hosted on GitHub

The Bioconductor organization on GitHub hosts repositories that may also be packages in Bioconductor. To identify and update these packages with the latest version bump from a Bioconductor release, use the ReleaseLaunch R package at https://github.com/Bioconductor/ReleaseLaunch.

First, create a fine-grained Personal Access Token (PAT) under User > Settings > Developer Settings > PATs > Fine-grained tokens.

When generating a new token, select Bioconductor (or organization) as the resource owner, then ‘All repositories’ under ‘Repository access’. Finally, click on the Generate token green button. Use gitcreds::gitcreds_set() to store the fine-grained PAT.

NOTES

• If you are renaming branches, select ‘Read and Write’ under ‘Contents’ in the ‘Repository permissions’ section.

• Depending on the organization’s settings, R / Bioconductor packages hosted on GitHub may also be updated with a classic PAT.

• These commands only work if the user has admin access to the organization’s GitHub account.

To update all packages, run the following command:

add_gh_release_branches(release = "RELEASE_3_20", org = "Bioconductor")

Be sure to edit the release argument in the function.

To update an individual package, run the following function:

add_gh_release_branch(
    package_name = "updateObject", release = "RELEASE_3_20", gh_branch = "devel"
)

Note that the gh_branch corresponds to the default branch on GitHub. On some repositories, the gh_branch can be devel.

Packages will be cloned into the current working directory.