Using git to manipulate website source code

Introduction

Git is a version control system that helps to manage having multiple people work on the same material simultaneously. Every user can hold a local copy of a main repository. The local copies can be on the same machine, or across the world. Users can then modify the local copy as they wish and when the modified material is ready, commit the changes and push them back to the main repository.

Git will not let you push a commit directly if the remote repository has any newer commits (modifications) than your local copy on the same branch. In such case where conflict takes place, please fetch and update your local copy first and rebase your new modifications on top of the latest commit as needed.

Git repository write access

The whole Debian website source code is managed under Git. It is located at https://salsa.debian.org/webmaster-team/webwml. By default, guests are not allowed to push commits onto the source code repository. You will need some kind of permission to gain write access to the repository.

Unlimited write access

If you need unlimited write access to the repository (e.g., You are about to become a frequent contributor), please consider requesting write access via the https://salsa.debian.org/webmaster-team/webwml web interface after logging in to Debian's Salsa platform.

If you are new to Debian's website development and have no previous experience, please also send a mail to debian-www@lists.debian.org with your self introduction before requesting unlimited write access. Please provide with something useful in your introduction, like which language or which part of the website you plan to work on, and who would vouch for you.

Write into repository via Merge Requests

If you do not intend to get unlimited write access to the repository or is unable to do so, you can always submit a Merge Request and let other developers to review and accept your work. Please submit Merge Requests using the standard procedure as provided by the Salsa GitLab platform via its web interface (read Project forking workflow and When you work in a fork for details).

The Merge Requests are not monitored by all website developers thus it may not always be processed in time. If you are not sure whether your contribution would be accepted, please send a email to the debian-www mailing list and request for a review.

Working on the repository

Get a local copy of the repository

First, you need to install git to work with the repository. Next, configure your user and e-mail details on your computer (please refer to general git documentation to learn how to do this). Then, you can clone the repository (in other words, make a local copy of it) in one of two ways.

The recommended way to work on webwml is to first register an account on salsa.debian.org and enable git SSH access by uploading an SSH public key into your salsa account. See the salsa help pages for more details on how to do that. Then you can clone the webwml repository using the following command:

   git clone git@salsa.debian.org:webmaster-team/webwml.git

If you don't have a salsa account, an alternative method is to clone the repository using the HTTPS protocol:

  git clone https://salsa.debian.org/webmaster-team/webwml.git

This will give you the same repository copy locally, but you will not be able to directly push changes back directly this way.

Cloning the whole webwml repository will require downloading about 500MB of data, thus it may be difficult for those with slow or unstable Internet connections. You may try shallow cloning with a minimum depth first for a smaller initial download:

  git clone git@salsa.debian.org:webmaster-team/webwml.git --depth 1

After obtaining a usable (shallow) repository, you can deepen the local shallow copy and eventually convert it to a full local repository:

  git fetch --deepen=1000 # deepen the repo for another 1000 commits
  git fetch --unshallow   # fetch all missing commits, convert the repo to a complete one

Partial content checkout

You can create a checkout for only a subset of the pages like this:

   $ git clone --no-checkout git@salsa.debian.org:webmaster-team/webwml.git
   $ cd webwml
   $ git config core.sparseCheckout true
   In webwml: Create the file .git/info/sparse-checkout with content like this
   (if you only want the base files, English, Catalan and Spanish translations):
      /*
      !/[a-z]*/
      /english/
      /catalan/
      /spanish/
   Then:
   $ git checkout --

Submitting local changes

Keep your local repo up-to-date

Every few days (and definitely before starting some editing work!) you should do a

   git pull

to retrieve any files from the repository which have changed.

It is strongly recommended to keep your local git working directory clean before performing "git pull" and following editing work. If you have uncommitted changes or local commits that are not present in the remote repository on the current branch, doing "git pull" will automatically create merge commits or even fail due to conflicts. Please consider keeping your unfinished work in another branch or using commands like "git stash".

Note: git is a distributed (not centralised) version control system. This means that when you commit changes they will only be stored in your local repository. To share them with others, you will also need to push your changes to the central repository on salsa.

Example on editing English files

An example on how to edit English files within the website source code repository is provided here. After obtaining a local copy of the repo using "git clone" and before starting the editing work, run the following command:

   $ git pull

Now make changes to files. When you are done, commit your changes to your local repository using:

   $ git add path/to/file(s)
   $ git commit -m "Your commit message"

If you have unlimited write access to the remote webwml repository, you may now push your changes directly onto the Salsa repo:

   $ git push

If you do not have direct write access to the webwml repository, please consider submitting your changes using the Merge Request function as provided by Salsa GitLab platform or asking other developers for help.

That's a very basic summary of how to use git to manipulate the Debian website's source code. For more information on git, please read git's documentation.

Closing Debian bugs in git commits

If you include Closes: #nnnnnn in your commit log entry, then bug number #nnnnnn will be closed automatically when you push your changes. The precise form of this is the same as in Debian policy.

Linking using HTTP/HTTPS

Many Debian websites support SSL/TLS, so please use HTTPS links where possible and sensible. However, some Debian/DebConf/SPI/etc websites either don't have have HTTPS support or only use the SPI CA (and not an SSL CA trusted by all browsers). To avoid causing error messages for non-Debian users, please do not link to such sites using HTTPS.

The git repository will reject commits containing plain HTTP links for Debian websites that support HTTPS or containing HTTPS links for the Debian/DebConf/SPI websites that are known to either not support HTTPS or use certificates signed only by SPI.

Working on translations

Translations should always be kept to be up-to-date with its corresponding English file. The "translation-check" header in translation files is used to track which version of English file the current translation was based on. If you change translated files, you need to update the translation-check header to match the git commit hash of the corresponding change in the English file. You can find that hash with

$ git log path/to/english/file

If you do a new translation of a file, please use the copypage.pl script and it will create a template for your language, including the correct translation header.

Translation changes with smart_change.pl

smart_change.pl is a script designed to make it easier to update original files and their translations together. There are two ways to use it, depending on what changes you are making.

To use smart_change to just update the translation-check headers when you're working on files manually:

  1. Make the changes to the original file(s), and commit
  2. Update translations
  3. Run smart_change.pl - it will pick up the changes and update headers in the translated files
  4. Review the changes (e.g. with "git diff")
  5. Commit the translation changes

Or, if you're using smart_change with a regular expression to make multiple changes across files in one pass:

  1. Run smart_change.pl -s s/FOO/BAR/ origfile1 origfile2 ...
  2. Review the changes (e.g. with git diff)
  3. Commit the original file(s)
  4. Run smart_change.pl origfile1 origfile2 (i.e. without the regexp this time);it will now just update headers in the translated files
  5. Finally, commit the translation changes

This is more involved than the previous one (needing two commits), but unavoidable due to the way git commit hashes work.

Getting notifications

Receiving commit notifications

We have configured the webwml project in Salsa so that commits are shown in the IRC channel #debian-www.

If you want to receive notifications via e-mail when there are commits in the webwml repo, please subscribe to the www.debian.org pseudopackage via tracker.debian.org and activate the vcs keyword there, following these steps (only once):

Receiving Merge Request notifications

If you wish to receive notification emails whenever there are new Merge Requests submitted on the webwml repository web interface on Salsa GitLab platform, you may configure your notification settings for the webwml repository on the web interface, following these steps: