Recreating Python's argparse in Bash

Parsing arguments in Bash can be annoying

Picture this. You've just finished an amazing new piece of code. It's tested, it has nice logging messages, and you're proud of it. Now comes time to ship it to your coworkers or users. You give your code to your coworker, and she asks how to run it. Well that's obvious, you reply, you just invoke Python, pass in the name of the entrypoint, specify your command line arguments, denote the output folder, and press Enter. Your coworker looks befuddled. Fine, you conclude, I'll make a script to run the application. You create run.sh and add some command line options like -a for argument and -o for the output folder. Your coworker comes back with more questions. Can you use multiple -a arguments at once? Will they overwrite each other? Does -o check for an output folder before executing? Will you be updating the documentation regularly?

Isn't there a better way to do this? Almost every language ships with some form of argument parsing, or has a helpful library nearby. Bash has getopt, Python has argparse, and Node.js has yargs. In Python and Node.js, argument parsing is a pleasant experience. You can build autogenerated help menus, control option settings, create mutually exclusive parsing groups, and more. Is this available natively in Bash with getopt? Nope.

Created by convenience, honed by necessity

I often found myself writing Bash scripts to run my code. What would initially start out as a run.sh would evolve into a fully featured entrypoint into an application, and I found myself recreating some form of argument parsing in every script. Sometimes it would involve manually checking for a --help flag and printing out a predefined help message. Other times it would involve searching for a specific -a argument and capturing the next token. After some time I noticed how repetitive this process had become, and how it could be automated in the future.

This led to me to create koi, a fully-featured and automated argument parser for Bash.

Pleasant argument parsing in Bash with koi

Koi acts as a single Bash script that parses command line arguments, and was inspired by Python's argparse. It automatically sets up subcommand parsing on the command line and requires little effort to incorporate into existing scripts.

Let's say you'd like to write a simple script to say hello to someone. This is easy to document and set up with koi. Let's create a new file called say.sh.

#!/bin/bash
source koi

koiname=say.sh
koidescription="A script for saying things"

function hello {
    __addarg "-h" "--help" "help" "optional" "" "Greet a person by name"
    __addarg "" "name" "positionalvalue" "required" "" "The name of someone you like"
    __addarg "-s" "--salutation" "storevalue" "optional" "Hello" "The greeting to use"
    __addarg "-e" "--exclaim" "flag" "optional" "" "If included, greeting will be exclaimed"
    __parseargs "$@"

    local exclaim_text=
    if [[ $exclaim -eq 1 ]] ; then
        exclaim_text='!'
    fi

    echo "${salutation}, ${name}${exclaim_text}"
}

__koirun "$@"

What value does koi add here? Let's take a look.

First, koi sets up the script's name and description. These strings are used in all of the automated help menus. Next, koi exposes the __addarg function, which allows the script author to register command line arguments with a specific function. Any function created in the script can be used as a subcommand on the command line, so hello would effectively become a subcommand. Koi then registers a few types of arguments: a required positionalvalue for reading in the argument name, an optional storevalue for reading in an optional salutation with the default Hello, and an optional flag for whether or not the greeting should be exclaimed.

Based on the registered arguments, koi automatically sets up ./say.sh --help.

./say.sh --help

A script for saying things

Usage:
  say.sh COMMAND [args]

Available commands:
  hello
  help
  list

Hints:
  say.sh help --verbose    Show complete command documentation
  say.sh COMMAND --help    Show individual command documentation

Koi also sets up the help menu for each subcommand, such as ./say.sh hello --help.

./say.sh hello -h

say.sh hello [-h] [-s SALUTATION] [-e] NAME 
Greet a person by name
  name                           The name of someone you like 
  -s, --salutation SALUTATION    The greeting to use (optional) (default: Hello) 
  -e, --exclaim                  If included, greeting will be exclaimed (optional)

Now we have complex argument parsing without any additional logic!

./say.sh hello 'Guido van Rossum' --exclaim

Hello, Guido van Rossum!

Pretty cool!

A slew of advanced features

Automated help menus

One of the most powerful features of koi is its ability to autogenerate help menus (documentation). When you register an argument with __addarg, you are required to include help text about the argument, which koi can then use to generate a help menu (as in the example above).

Further documentation: Autogenerated help menus

Subcommand parsing

Another powerful feature of koi is the ability to parse subcommands.

You can either attach your arguments to your script:

./myscript.sh -a arg -b arg

Or, you can attach your arguments to subcommands:

./myscript.sh subcommand -a arg -b arg

This is powerful because it allows you to have more granular control over reading in arguments for different functionality.

Further documentation: Subcommands

Parsing mutually exclusive groups

Sometimes it is helpful to group a script's arguments together. This can be used to require two arguments to be used together, or to prevent more than one optional argument from being used.

For example, suppose myscript.sh had two available flags, --flagA and --flagB. We can use argument grouping to make rules like:

1. Either --flagA or --flagB can be present, but not both.
2. If --flagA is present, so must be --flagB, and vice versa.

Koi offers this functionality with the __addgroup function.

Further documentation: Argument Groups

Verifying arguments

Often we need to run some sort of validation on each command line argument. Koi allows you to specify a verifying function that will be run on each registered argument, and comes default with some helpful functionality like verifying that a file or directory exists. This allows us to reuse verifying logic on multiple arguments and keeps our code minimal and clean.

For example, consider the following function check_for_files, which checks for a number of files using the __verifyfile verifying function.

function check_for_files {
    __addarg "-h" "--help" "help" "optional" "" "Check that a number of different files exist"
    __addarg "" "--sourcefile" "storevalue" "" "Check that a source file exists" "__verifyfile"
    __addarg "" "--destinationfile" "storevalue" "" "Check that a destination file exists" "__verifyfile"
    __addarg "" "--anotherfile" "storevalue" "" "Check that another file exists" "__verifyfile"
    __parseargs "$@"
    # rest of function...
}

Further documentation: Verifying arguments

Default values

Sometimes when our scripts offer optional command line arguments, it's helpful for those arguments to have default values. If a user omits a command line argument that is needed to run the function, we can use koi to include a default for that argument.

Further documentation: Defaults

Dependent arguments

Sometimes it is helpful to have an argument's usage be tied to another argument's presence. Koi accomplishes this with dependent arguments.

For example, suppose myscript.sh has two available flags, --flagA and --flagB. We can use dependent arguments to define rules like:

1. If --flagA is present, --flagB must be present.
2. --flagB can be present without --flagA.

In this example, we say that --flagA is dependent on --flagB. Thus, we refer to --flagA as the dependent argument and --flagB as the dependency argument.

Further documentation: Dependent Arguments

Using koi

Koi was born out of necessity for a better argument parser in Bash, and is currently used in a variety of companies, projects, and applications. If you'd like to get started using koi, it's easy to install with Homebrew using brew install wcarhart/tools/koi. There are other installation options in koi's GitHub repository. If you'd like to read more about koi, check out its dedicated documentation subsite. You can also find the example script from this blog post in this repository. I hope you enjoy using it as much as I have.

🦉