Gitlab Ci Job Descriptions

This page is here to provide a description of current jobs ran on the Gitlab CI/CD pipeline.

Job Name

Time (Minute)

buildtestimage

39

static-analysis

5

test

6

docs

1

userdocs

~

wheel

.5

webrefimg

23

deploy

~

Images can be found at https://code.ornl.gov/rse/images

And the images for this repo are uploaded here: code.ornl.gov:4567/reflectometry/web_reflectivity/

buildtestimage

First this job performs the [func_rse_docker_cleanup](https://code.ornl.gov/rse-deployment/rse-sharables/raw/master/rse-bash-modules.sh) action, then builds a test docker image for the package under src/ and pushes it to gitlab using the name of the feature branch as tag, under subdirectory feature/. This is performed first so that subsequent jobs may reuse the same image and avoid unnecessary builds.

static-analysis

This job pulls the latest docker image and runs the battery of checks normally performed by the pre-commit hook At the time of writing this includes the following:

  • trailing-whitespace

  • check-docstring-first

  • check-json

  • check-added-large-files

  • check-yaml

  • debug-statements

  • requirements-txt-fixer

  • check-merge-conflict

  • end-of-file-fixer

  • sort-simple-yaml

  • black

  • flake8

For the an accurate list of hooks, please refer to the .pre-commit-config.yaml file

test

This job pulls the feature docker image. It then performs db migrations, runs unit tests, generating a coverage report, and finally builds the wheel to confirm it can successfully and stores it for later.

docs

This job pulls the feature docker image and generates docs in the /docs/ folder using Sphinx

userdocs

Using a POST request, this job signals the readthedocs.org site to pull and publish docs from the latest image.

The instance for this project is located here: https://web-reflectivity.readthedocs.io/en/latest/

wheel

This job pulls the feature docker image and publishes the wheel created during the test step using the publish_wheel.sh script The script is just a python -m twine upload with credential checks, failing the job if data is missings.

The filename is configured in the pyproject.toml. The naming convention for the generated wheel is PREFIX-VERSION(.devDISTANCE) where

  • PREFIX: web-reflectivity

  • VERSION: is the most recent tag given by git describe. For developer versions this is one minor version ahead of the last release. VERSION will be of the form MAJOR.MINOR.PATCH(rcCANDIDATE) where rcCANDIDATE is missing from a full release.

  • DISTANCE: number of commits since latest git tag

An example name is web_reflectivity-1.2.0.dev507-py3-none-any.whl. This job only executes on protected branches such as next, qa, or main

webrefimg

This jobs builds the latest docker image for branch next (tag latest-dev), qa (tag latest-test), or main (tag latest-prod), then pushes the image to the container registry. For stable release, it retags latest-prod as vX.Y.Z and pushes the image to subdirectory release/. Finally it cleans up the images locally. This job only executes on protected branches such as next, qa, or main

deploy

This job attempts to deploy the docker image for the environment associated with the branch that triggered it.