Stuck in the 70’s?

Sanitize shell script from the disco era

Urs Roesch

Why bash?

As part of my work I encounter many shell scripts which are poorly written but are essential for the infrastructure to function properly.

Most of the time the stake holders aren’t sufficiently trained to use a higher level language such as python, ruby or go.

By introducing modern bash features, the scripts can be maintained by all stake holders without creating high hurdles for entry.

Target audience

If you often encounter shell or scripts written in bash and want to learn a few tricks to make your and your co-worker’s life easier this presentation is for you.

A basic understanding of programming in general and shell scripting in particular is assumed.

Errors, Shmerrors

set -o errexit

Script without errexit option
#!/usr/bin/env bash

echo 'Still alive?' # Still alive?; exit rc 0
Script with errexit option
#!/usr/bin/env bash

set -o errexit

false               # exit rc 1
echo 'Still alive?'

To prevent exit on error, append || : after the command.

Prevent exit on error
#!/usr/bin/env bash

set -o errexit

false || :
echo 'Still alive?' # Still alive?; exit rc 0

set -o nounset

Script without nounset option
#!/usr/bin/env bash

rm -rf /home/jdoe/${temp_dir} # rm -rf /home/jdoe/
Script with nounset option
#!/usr/bin/env bash

set -o nounset

rm -rf /home/jdoe/${temp_dir} # bash: temp_dir: unbound variable; exit rc 1

set -o pipefail

Script without pipefail option
#!/usr/bin/env bash

find /fake-dir | awk '{print $1}' # exit rc 0
Script with pipefail option
#!/usr/bin/env bash

set -o pipefail

find /fake-dir | awk '{print $1}' # exit rc 1


Always use the option triplet errexit, nounset and pipefail in your scripts!

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o pipefail


To disable the option for a certain block or command use +o instead of -o.


# switch off exit on error
set +o errexit
# switch it back on
set -o errexit


I do declare!


Ensure the variable only holds integers.

declare -i int=1

int=string    # bash: string: unbound variable

int=2         # declare -i int="2"

Integer math

Fun fact, declaring a variable as an integer allows for simple integer math.

declare -i int

int=1+2      # declare -i int="3"    # addition
int=2-1      # declare -i int="1"    # subtraction
int=8/2      # declare -i int="4"    # division
int=9/2      # declare -i int="4"    # division rounded
int=9%2      # declare -i int="1"    # modulo
int=3*2      # declare -i int="6"    # multiplication

Zero padded numbers

Caution! The zero padded numbers 08 and 09 are interpreted as octal and throw an error.
declare -i int

int=01  # declare -i int="1"
# ...
int=07  # declare -i int="7"
int=08  # bash: 08: value too great for base (error token is "08")
int=09  # bash: 09: value too great for base (error token is "09")


Define a variable as a constant.

declare -r readonly=set-in-stone

readonly=change   # bash: readonly: readonly variable


The content of the variable is converted to lowercase.

declare -l lower=LOWER

echo ${lower}    # lower
Requires bash >= 4.0


The content of the variable is converted to uppercase.

declare -u upper=upper

echo ${upper}    # UPPER


The content of the variable is an array.

declare -a array=( 1 2 3 )

typeset -p array  # declare -a array=([0]="1" [1]="2" [2]="3" )


typeset -p array  # declare -a array=([0]="foo" [1]="2" [2]="3")

Associative Arrays

The content of the variable is an associative array.

declare -A hash=( [a]=foo [b]=bar )

typeset -p hash # declare -A hash=([b]="bar" [a]="foo" )


typeset -p hash # declare -A hash=([0]="c" [b]="bar" [a]="foo" )


typeset -p hash # declare -A hash=([c]="foo" [b]="bar" [a]="foo" )
Requires bash >= 4.0


There are few other switches to declare and the certainly can be combined. To create a readonly array use declare -ra for instance.

Using declare helps narrowing the scope of the values a variable can hold. This makes the script more predictable!

Operation plus equal

Integer incrementation

Classical way
while true; do
  counter=`expr ${counter} + 1`   # declare -- counter="1"
  # [...]
With += operator
declare -i counter=0
while true; do
  counter+=1                      # declare -i counter="1"
  # [...]
Without declaring the variable as an integer the behavior is not as expected.
while true; do
  counter+=1                      # declare -- counter="01"
  # [...]

Append to string

Classical way
string="${string}bar"  # declare -- string="foobar"
With += operator
declare -- string=foo
string+=bar            # declare -- string="foobar"

Push to array

Classical list
list="${list} bar" # declare -- list="foo bar"
With += operator
declare -a list=( foo )
list+=( bar )      # declare -a list=(foo bar)


The += operator helps with the assignment of existing and brings a bit more comfort to the previously wordy and often times ugly process of appending or incrementing numbers.

A -= assignment operator does not exist!

Substitution Revolution

Modify case

There are two basic modes, the first is to convert the case of the first letter. The second is to convert the whole string.

declare -- to_lower="ALL CAPS"
echo ${to_lower,}               # aLL CAPS
echo ${to_lower,,}              # all caps

declare -- to_upper="all lower"
echo ${to_upper^}               # All lower
echo ${to_upper^^}              # ALL LOWER
Requires bash >= 4.0

There is also the option of only matching a pattern to adjust the case.

declare -- fruits="banana apple pear"
echo ${fruits^a}                # banana apple pear
echo ${fruits^[bp]}             # Banana apple pear
echo ${fruits^^a}               # bAnAnA Apple peAr
echo ${fruits^^[ae]}            # bAnAnA ApplE pEAr
Requires bash >= 4.0
Real world example
if [ `echo ${string1} | tr "[A-Z]" "[a-z]"` = "${string2}" ]; then
  # [...]
With parameter expansion
if [[ ${string1,,} == ${string2} ]]; then
  # [...]


This one really gets me! I almost never see ${parameter/pattern/} substitutions in the wild. And is has been around for ages.

declare -- string="lagoon racoon"
echo ${string/oo/u}             # lagun racoon
echo ${string//oo/u}            # lagun racun
Real world example
string="lagoon racoon"
echo ${string} | sed 's/oo/u/g' # lagun racun
With parameter expansion
declare -- string="lagoon racoon"
echo ${string//oo/u}             # lagun racun


Parameter expansion in shell scripts can be a bit cryptic at first but it is definitely a lot faster then to run a sub shell for every little change made to a small string.

There are many more expansions available but the examples here are bash specific and won’t work in the predecessor shells such as Bourne or Korn shell.

( hip hip )

Modify case

Combining bash arrays with the upper case parameter expansion is mighty powerful. Let’s see how it’s done.

declare -a fruit=( banana apple pear )
echo ${fruit[@]^}               # Banana Apple Pear
echo ${fruit[@]^^}              # BANANA APPLE PEAR
echo ${fruit[@]^a}              # banana Apple pear
echo ${fruit[@]^[bp]}           # Banana apple Pear
echo ${fruit[@]^^a}             # bAnAnA Apple peAr
echo ${fruit[@]^^[ae]}          # bAnAnA ApplE pEAr
Requires bash >= 4.0


Applying substitution to each element of an array can also be done. Here a few examples.

General substitutions
declare -a array=( lagoon racoon )
echo ${array[@]/o/u}          # laguon racuon
echo ${array[@]//o/u}         # laguun racuun
Prefix substitutions
declare -a sshopts=( BatchMode=yes User=foobar )
echo ${sshopts[@]/#/-o }      # -o BatchMode=yes -o User=foobar
Append suffix
declare -a fruits=( banana apple pear )
echo ${fruits[@]/%/,}         # banana, apple, pear,
Remove suffix
declare -a fruits=( bananas apples pears )
echo ${fruits[@]/%s/}         # banana apple pear


Parameter expansion combined with bash arrays allows to make volatile changes to a list of values at the time of echoing. No sed, tr and awk constructs and excessive looping are required.

Perfect condition

Double square brackets

Double square brackets are more forgiving when it comes to dealing with empty strings in a comparison.

Classical shell script condition
[ ${foo} = foo ]      # bash: [: =: unary operator expected

Effectively the above conditional statement is expanded as [ = foo ] because the variable foo is empty. There are two common ways to prevent the error:

Classical shell script condition workaround
[ "${foo}" = "foo" ]  # exit rc 1
[ x${foo} = xfoo ]    # exit rc 1

Korn shell introduced the more robusts [[ which does not suffer the same limitations.

Korn shell style double square brackets
declare -- foo=
[[ ${foo} == foo ]]   # exit rc 1

Regex comparison

Bash has also a regex operator =~ for matching strings. This often overlooked feature can save you lots of typing.

Classical shell string match
if echo ${string} | grep -q '[Bb]ar'; then
  # [ ... ]
Bash style regex
declare -- string=foobar
if [[ ${string} =~ [Bb]ar ]]; then
  # [ ... ]


These are but a few small examples creating "perfect conditions" for your bash scripts. Although they are not POSIX compatible the fact that ksh, zsh and bash among other implement them makes them virtually portable.

Funky, funky, functions!

Special characters

With the exception of a few reserved characters such as $, |, ( and { among others pretty much everything else works.

function @() { echo '@'; }
@                                        # @
function /() { echo '/'; }
/                                        # /
function -() { echo '-'; }
-                                        # -
function :() { echo ':'; }
:                                        # :
function ü() { echo 'ü'; }
ü                                        # ü
function 照() { echo '照'; }
照                                       # 照

This opens possibilities of creating prefixes or fake namespaces for functions. Pretty useful for sourced files and libraries.

Functions with faux namespaces
function funky() {
  # [ ... ]

function test::funky() {
  # [ ... ]

funky            # execute funky
test::funky      # test funky function

Function within functions

It it is even possible to place functions within functions. Unfortunately there they are still accessible globally.

function parent() {
  function @child() {
    # [ ... ]
  # [ ... ]


Surprisingly bash functions can be creatively named and have fewer restrictions than most other scripting languages. Caution is advised when using special characters tho. I don’t think many bash scripters use them.

What the printf?

Don’t loop, printf!

The bash printf can be used instead of a loop for printing as it prints each additional argument not unlike a loop.

Print fruit loop :)
declare -a fruit=( banana apple pear )
for f in ${fruit[@]}; do echo ${f}; done    # banana\napple\npear
Printf it!
declare -a fruit=( banana apple pear )
printf "%s\n" "${fruit[@]}"                 # banana\napple\npear

Straight lines

Don’t manually write out lines use printf!

printf "%0.1s" -{1..16}         # ----------------

sprintf anyone?

One can even emulate the sprintf function by using the -v switch.

printf -v line "%0.1s" -{1..16}
echo ${line}                    # ----------------
Requires bash >= 3.1

Escape plan!

With the %q placeholder printf provides a shell escaping routine.

printf "%q" 'cat foo | grep "foo bar"'
# cat\ foo\ \|\ grep\ \"foo\ bar\"

What does the clock say?

Use printf instead of the date command to convert an Unix epoch timestamp to a human readable format.

printf "%(%Y-%m-%d)T\n" 1515151515
Requires bash >= 4.2


IMHO, one of most underused builtin in bash scripts is printf. It is very versatile and can help reduce code and complexity if used appropriately.

Real World Examples ™

A few examples on how to reduce clutter with examples taken from the Real World ™.

The power of read(ing)

Expensive parsing and splitting with awk
fname=`echo ${info} | awk -F : '{ print $1 }'`   # Joe
lname=`echo ${info} | awk -F : '{ print $2 }'`   # Doe
role=`echo ${info} | awk -F : '{ print $3 }'`    # Sysadmin
read with IFS
declare -- info="Joe:Doe:Sysadmin"
IFS=: read fname lname role <<< "${info}"

The same but read everything into an array.

read into array
declare -- info="Joe:Doe:Sysadmin"
IFS=: read -a person <<< "${info}"

typeset -p person   # declare -a person=([0]="Joe" [1]="Doe" [2]="Sysadmin")

May the case be with you

Too WET to maintain
if [ ${host} = jp-prod ]; then
  prod_host ${host}
elif [ ${host} = ch-prod ]; then
  prod_host ${host}
elif [ ${host} = jp-uat ]; then
  uat_host ${host}
# [ ... ]
  dev_host ${host}
Better with case
case ${host} in
  *-prod) prod_host ${host};;
  *-uat)  uat_host ${host};;
  *)      dev_host ${host};;

Poor man’s Ansible

Execute a locally defined function on a remote machine without first copying the code.

function bootstrap() { config_host; hostname; }

ssh "$(declare -f bootstrap); bootstrap"


No Sysadmin was hurt during the making of this presentation!