Gitlab Ci Job Descriptions

This page is here to provide a description of current jobs ran on the Gitlab CI/CD pipeline.

Job Name

Time (Minute)

buildtestimage

39

static-analysis

5

test

6

docs

1

userdocs

~

wheel

.5

webrefimg

23

deploy

~

Images can be found at https://code.ornl.gov/rse/images

And the images for this repo are uploaded here: code.ornl.gov:4567/reflectometry/web_reflectivity/

buildtestimage

First this job performs the [func_rse_docker_cleanup](https://code.ornl.gov/rse-deployment/rse-sharables/raw/master/rse-bash-modules.sh) action, then builds a test docker image for the package under src/ and pushes it to gitlab with the ‘latest’ tag This is performed first so that subsequent jobs may reuse the same image and avoid unnecessary builds.

static-analysis

This job pulls the latest docker image and runs the battery of checks normally performed by the pre-commit hook At the time of writing this includes the following:

  • trailing-whitespace

  • check-docstring-first

  • check-json

  • check-added-large-files

  • check-yaml

  • debug-statements

  • requirements-txt-fixer

  • check-merge-conflict

  • end-of-file-fixer

  • sort-simple-yaml

  • black

  • flake8

For the an accurate list of hooks, please refer to the .pre-commit-config.yaml file

test

This job pulls the latest docker image with tag containing string buildimage. It then performs db migrations, runs unit tests, generating a coverage report and finally builds the wheel to confirm it can successfully and stores it for later.

docs

This job pulls the latest docker image and generates docs in the /docs/ folder using Sphinx

userdocs

Using a POST request, this job signals the readthedocs.org site to pull and publish docs from the latest image.

The instance for this project is located here: https://web-reflectivity.readthedocs.io/en/latest/

wheel

This job pulls the latest docker image and publishes the wheel created during the test step using the publish_wheel.sh script The script is just a python -m twine upload with credential checks, failing the job if data is missings.

The filename is configured in the pyproject.toml. The naming convention for the generated wheel is PREFIX-VERSION(.devDISTANCE) where

  • PREFIX: web-reflectivity

  • VERSION: is the most recent tag given by git describe. For developer versions this is one minor version ahead of the last release. VERSION will be of the form MAJOR.MINOR.PATCH(rcCANDIDATE) where rcCANDIDATE is missing from a full release.

  • DISTANCE: number of commits since latest git tag

An example name is web_reflectivity-1.2.0.dev507-py3-none-any.whl. This job only executes on protected branches such as next, qa, or main

webrefimg

This job builds the production docker image for Web Reflectivity It then pushes the image with the date appened to the tag and again with :latest-dev appended instead. i.e. this single image will have two tags associated with it, the former being its permanent tag, and the later a temporary tag. The temporary tag always refers to the latest version of this image. Finally it cleans up the images locally. This job only executes on protected branches such as next, qa, or main

deploy

This job attempts to deploy the docker image for the environment associated with the branch that trigged it.