Scriptology

Ready ancient scripts for CI/CD testing

Urs Roesch

What is Scriptology

A term derived from Script and Archaeology. Describes the art of finding scripts hidden under layers of system rubble, decipher their long forgotten syntax, dust them off and bring them into the modern world.

A nagging problem

As a sysadmin I write a lot of scripts that are for the most part irreversably modify the system. Keen to better myself I always wanted to have some automated testing for these scripts. The same is true for scripts I see at the places I work. Mostly these are scripts with no or very rudimentary testing frameworks.

For the longest time I wondered how to test without making any changes. This short presentation gives a glimpse how I got there.

The Journey

SVN → Git migration

At the beginning I was tasked with a repository migration from Subversion to Git. Everything should be hosted on the internal Gitlab instance.

Easy enough! Right?

reality check

Challenges

  • Many interlinked repositories.

  • A plethora of scripts scattered all over.

  • None of the scripts are under version control.

  • Scripts running on AIX exclusively.

  • 🔥 Hot scripts, running them modifies state.

Goals

  • Don’t open Pandora’s box by changing too much at any given time.

  • Focus on moving from Subversion to Git, not on the stuff around it.

  • Go one repo at the time.

  • Test, test, test and test again!

Step by step approach

  • Move all scripts under version control

  • Add --dry-run option for faux-testing.

  • Remove hard coded path.

  • Refactor code into modules and / or functions.

  • Add testing to modules.

  • Write stub programs to prevent the system from 💣.

Stubby programs

Bash stub

For the most part when executing an external command the caller parses the content from STDOUT. With the stub that is exactly what is done. Arguments in, text out.

#!/usr/bin/env bash

declare -r BIN_DIR=$(cd $(dirname ${0}); pwd)
declare -- PATH=${PATH/${BIN_DIR}/}
export PATH

case "${@}" in
'branch') printf "  %s\n" "feature" "bugfix" "main" ;;
'rev-parse --show-toplevel') echo "/git/root" ;;
'rev-parse --abbrev-ref HEAD') echo 'bugfix' ;;
'checkout'*) echo "Switched to branch '${2}'" ;;
*) echo ${NAME} ${@} ;;
esac

Shell builtin stubs

When testing shell scripts there is the concept of builtins especially in bash. To override them a file like the one below needs to by sourced.

# override builtin shell functions
function cd {
  :
}

function kill {
  :
}

Gitlab CI-CD sample

With the stubs in place and checked into the repository the .gitlab-ci.yml file looks like the sample below:

stages:
  - test

stubby-test:
  stage: test
  script:
    - >
      PATH=$(pwd)/tests/helpers:$PATH
      perl hot-script.pl --mode all --debug
    - |2+
      source stub.sh
      bash hot-script.sh

Summary

With a bit of elbow grease, a lot of sweat and a tear here and there one can bring scripts the ancient Greek would have recognized.

Is it worth the effort. I leave that up to the participants to decide.

EOF

https://github.com/uroesch/Scriptology

Scripts 10 years and older were kindly donated to museums all around the world!