DevOps Guide
Environment “testenv”
This environment allow developers and power users to run additional testing such as automated system tests and manual tests. One of its main purposes is to uncover bugs and defects not detected by the CI.
One host machine acting as a GitLab runner runs the instructions specified in the deploy job of the GitLab CI. There’s only one specific machine allowed to pick and run the deploy job so that environment testenv is always deployed to the same machine. The app running in testenv is exposed to the WWW as reflectivity-test.sns.gov.
Currently, the database in this environment is not persistent, meaning a new deployment will erase whatever data has been stored since the last deployment. Also, testenv is currently listening and writing to livedata.sns.gov, as well as listening to oncat.ornl.gov. Development of a test environment for the web monitor will bring about test substitutes for both servers.
Database Management
We rely on Django’s manage.py for dumping the source database and loading it into the recipient database. Django’s manage.py allows for a fine control of what tables to dump and the command is agnostic of the database flavor (mysql, postgresql, sqlite) for both the source and recipient databases.
Dumping the Old Database
The database from reflectivity.sns.gov has quite a different schema than the database of the modernized application because models for the app and its dependencies have evolved.
Login to reflectivity.sns.gov and then:
cd /var/www/web_reflectivity/app
dumpfile=/tmp/webreflect_$(date +%F).json # e.g. /tmp/webreflect_2022-05-01.json
python manage.py dumpdata --verbosity 3 --natural-foreign --natural-primary -e contenttypes -e auth.Permission -e django_auth_ldap -e django_celery_results --indent 2 --database default --traceback > ${dumpfile}
For loading the resulting JSON file into the recipient database, jump to Loading onto the Modernized Database.
Dumping the Modernized Database
It is assumed that the container running the web_reflectivity app, as well as the container running the database are up and running.
The name of the container running the web_reflectivity app should be test_webref_1 if running in the TEST environment. One can make sure by listing the running containers:
$> docker container ls
CONTAINER ID MAGE COMMAND CREATED STATUS PORTS NAMES
e71b9b6c4a4e code.ornl.gov:4567/reflectometry/web_reflectivity/web_reflectivity:latest-dev /usr/bin/docker-ent… 6 hours ago Up 6 hours (healthy) 22/tcp, 8000/tcp test_webref_1
In this particular case, the name of the container is test_webref_1, and we can use the CONTAINER ID e71b9b6c4a4e in place of this name.
Open a shell to the container running the web_reflectivity app and execute the dumpdata make target:
$> docker exec -it test_webref_1 bash
(webrefl)$ make dumpdata # e.g. creates /tmp/webreflect_2022-05-20.json
A JSON dump file /tmp/webreflect_$(date +%F).json is generated in the container’s /tmp directory. An easy way to make it available to the host machine is to move this file to directory /var/log/ because this directory is mounted in the host machine as directory /tmp/log/web_reflectivity/web/
Loading onto the Modernized Database
We need to make the JSON dump accessible from within the container running the app. An easy way is to place the file in the host machine directory /tmp/log/web_reflectivity/web/ because is bind-mounted to container’s directory /var/log/.
Assuming we have file /tmp/log/web_reflectivity/web/webreflect_2022-05-01.json in the host machine, we need to open a shell in the running container servicing the application and execute the make loaddata target.
Details on how to find out the name of the running container are laid out in the previous section Dumping the Modernized Database.
docker exec -it test_webref_1 bash
(webrefl)$ make fixturefile=/var/log/webreflect_2022-05-01.json loaddata
This will update the recipient database. The command takes minutes to execute because it translates the JSON file into a large set of python objects. These objects are in turn translated into a long list of postgres commands to be executed on the recipient database.
Deployment for Testing
Deployment for the testing environment is handled by the web-relectivity-deploy repo. For details on how to deploy, read the Guide for the deployment to Testing Environment.
Access to the Test Environment
Access to the test server reflectivity-test.sns.gov via SSH is granted on a per user and per client-machine basis. Contact the DevOps engineer assigned to maintenance of the server. After granted access, SSH to the machine as user cloud
ssh cloud@reflectivity-test.sns.gov
The goal of accessing the test server is to troubleshoot any problems by directly manipulating the deployed application.
Deployment in Production
This deployment is scheduled to happen after a new version of the software is released and thoroughly tested in deployment testenv. For details on how to deploy, read the Guide for the deployment to Production Environment.