---
title: "How to branch the Annotation package repository"
author: "Bioconductor Maintainer"
date: "`r format(Sys.time(), '%B %d, %Y')`"
vignette: >
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteIndexEntry{Branching the annotation package repository}
  %\VignetteEncoding{UTF-8}
output:
  BiocStyle::html_document:
    number_sections: yes
    toc: true
---

```{css, echo=FALSE}
.infobox {
  padding: 1em 1em 1em 1em;
  margin-bottom: 10px;
  border: 2px solid #008000;
  border-radius: 10px;
  background: #c7dbe4 5px center/3em no-repeat;
  color: #8B0000;
}

.center {
  text-align: center;
}
```


# Branching Annotation

The document describes how to branch the annotation repository for a new
release. This can be done the day of the release or several days after.

Before branching, Check and make sure there is enough room on master for the
annotations. If there isn't enough room follow the guidelines for moving old
[Bioconductor releases into
S3](https://github.com/Bioconductor/AWS_management/blob/master/docs/S3_website.md)
As of release 3.13 Annotations are around 113G.

These instructions use BioC 3.14 as the new devel version and nebbiolo2 as the
devel master builder.

As of 3.14, the linux builders have moved to DFCI in Boston. Personal accounts
and the jump account should be set up for access.

## Set up nebbiolo2

### Create new annotation repo on devel master builder (nebbiolo2)

```sh
ssh to nebbiolo2 as biocpush user
cd ~/PACKAGES/3.14/data
mkdir annotation
```

### rsync annotations from current release

Because 3.13 used RPCI mablec2 and 3.14 used DFCI nebbiolo2, we could only rsync
in one direction because of firewalls. It should work on either machine, in
either direction in the future if under the same firewall.

::: {.infobox data-latex=""}

**Notes**

* Run with `--dry-run` first
* Note the trailing slash `/` and dot `.`

:::

```sh
# From malbec2 to nebbiolo2 (logged into malbec2):
cd ~/PACKAGES/3.13/data/annotation
rsync --dry-run -ave ssh . biocpush@nebbiolo2:PACKAGES/3.14/data/annotation/
```

Live Run if all looks correct

```sh
rsync -ave ssh . biocpush@nebbiolo2:PACKAGES/3.14/data/annotation/
```

### Remove symlinks for old R versions for windows and mac

There may be symlinks in the windows and macosx folders. These will only be
present when the new version of BioC uses a new version of R which happens every
other version of BioC. If they are present, they will look something like this:

```sh
~/PACKAGES/3.7/data/annotation/bin/windows/contrib/3.5 -> 3.4
~/PACKAGES/3.7/data/annotation/bin/macosx/contrib/3.5 -> 3.4
```

Remove the symlinks and rename the folder with content to the version of R being
used by devel e.g.,

```sh
rm ~/PACKAGES/3.7/data/annotation/bin/windows/contrib/3.5 -> 3.4
mv 3.4 3.5
```

The PACKAGES and PACKAGES.gz files are empty. They are needed for
install.packages() even though we don't have binaries for annotations.

A rerun of the `rsync` command should only show changes in the files we just
edited:

```sh
rsync --dry-run -ave ssh biocadmin@malbec1:PACKAGES/3.6/data/annotation/ .
```

## Set up <master.bioconductor.org>

### Log on master as webadmin

```sh
ssh ubuntu@master.biocondcutor.org
sudo su - webadmin
cd /extra/www/bioc/packages/3.14/data/annotation
```

Products are sent to master from both nebbiolo2 and staging.bioconductor.org.
Because of this, the rsync from nebbiolo2 to master is not from the top level;
we want to leave the products deposited by staging.bioconductor.org untouched.

### Remove symlink and create empty annotation folder

There will likely be a simlink at

```sh
/extra/www/bioc/packages/3.14/data/annotation -> ../../3.13/data/annotation
```

pointing to the current release annotation repo. In recent years, the symlinks
have appeared underneath an already created annotation repostiory. The point is
to start with a non-linked empty annotation folder. Remove annotation and
recreate:

```sh
rm annotation
```

## Create a new annotation folder

```sh
mkdir annotation
```

## Back to nebbiolo2 to run the propagation scripts

1.  Remove any deprecated packages from the 3.13 annotation repo that should not
    be included in the devel 3.14 repo.

2.  Update and run cronjob by hand (should be commented out).

::: {.infobox data-latex=""}

Symlink must be gone on master before running this!!

:::

Run the crontab entry by hand. Monitor
\~/cron.log/3.14/propagate-data-annotation-\* log file.

3.  Test BiocManager::install() to see if it finds the new repo. Try installing
    a package that should be found and try installing a removed package that
    should fail.

## Update bioconductor.org/config.yaml to build landing pages for 3.14

Uncomment 'data/annotation':

```yaml
devel_repos:
- "bioc"
- "data/experiment"
- "data/annotation"
```
