argparsh allow shell scripts to have better argument parsing. However, it requires many repeated invocations of the argparsh process - how can we reduce the overhead of spawning argparsh processes? In this post we make argparsh 3.46x faster by using static linking and nostd.

The goal of argparsh is to make it easier for me to embed documentation into bash scripts. As a researcher, I find myself frequently cobbling together bash scripts out of commands I’ve already run interactively. As I’m often testing various different configurations to analyze how changes I’m making affect performance or other metrics, I often add flags and arguments to my scripts. While other scripting languages, like python, offer better ergonomics around argument parsing, it’s more overhead to port my workflow into python. Another option is to use a more modern shell, like fish, but the bash muscle memory is strong. Hence, argparsh.

argparsh works by building a parser program on stdout, then “compiling” that program and parsing your actual arguments. Here’s a basic hello world script using argparsh:

aneesh@earth:~/$ command cat hello.sh
parser=$(
    argparsh new "hello.sh" --description "hello world script"
    argparsh add_arg --helptext "name to greet" "name"
    argparsh add_arg --type int \
        --helptext "number of times to repeat greeting (default: 1)" \
        -- "-r" "--repeat"
)
eval $(argparsh parse $parser -- $@)

for i in $(seq 1 $repeat); do
    echo "hello" $name"!"
done
aneesh@earth:~/$ bash hello.sh --help
usage: hello.sh [-h] [-r REPEAT] name

hello world script

positional arguments:
  name                 name to greet

options:
  -h, --help           show this help message and exit
  -r, --repeat REPEAT  number of times to repeat greeting (default: 1)
aneesh@earth:~/$ bash hello.sh --foo
usage: hello.sh [-h] [-r REPEAT] name
hello.sh: error: option --foo: is not recognized
aneesh@earth:~/$ bash hello.sh foo
hello foo!
aneesh@earth:~/$ bash hello.sh -r 10 foo
hello foo!
hello foo!
hello foo!
hello foo!
hello foo!
hello foo!
hello foo!
hello foo!
hello foo!
hello foo!
aneesh@earth:~/$ bash hello.sh -r bar foo
usage: hello.sh [-h] [-r REPEAT] name
hello.sh: error: argument repeat: invalid int value: 'bar'

We can see that hello.sh prints useful and configurable helptext and even provides some argument validation.

However, argparsh comes with one downside - script launch overhead. For any script to begin executing it must first run all argparsh invocations. Previously, I’ve written about the techniques argparsh employs for speed, but in this post we’re going to take it even further.

What makes launching a process slow?

When a process is launched, it typically goes through the following workflow:

+------------------------------------------------+
|           Load process into memory             |
+------------------------------------------------+
                        |
                        v
+------------------------------------------------+
|           Go to process entry point            |
+------------------------------------------------+
                        |
                        v
+------------------------------------------------+
|            Initialize dynamic loader           |
+------------------------------------------------+
                        |
                        v
+------------------------------------------------+
|           Initialize libc runtime              |
+------------------------------------------------+
                        |
                        v
+------------------------------------------------+
|  Load and initialize other dynamic libraries   |
+------------------------------------------------+
                        |
                        v
+------------------------------------------------+
|                Invoke main()                   |
+------------------------------------------------+

Each of these stages adds some latency. To quantify this latency, I wrote the simplest possible c program - do nothing and exit. I compile this program with gcc. We measure how long this program takes under three settings:

• Default - compiled with GNU libc (linked dynamically)
• static - compiled with musl libc (linked statically)
• static_no_libc - compiled with musl libc (linked statically), and no libc

GNU libc no longer supports static linking, so musl libc was used instead. For the no_libc case, the program has to be a bit different. Instead of defining main, we define _start (which gcc will set as the program entry point). Additionally, we must explicitly invoke the exit syscall to terminate the process. This is the tradeoff of having a binary with no (libc) runtime. You pay nothing for the runtime features you don’t need, but need to provide any runtime features you do rely on. There is a point where the cost of reimplementation, or just the overhead of the features you need themselves, precludes any benefit from removing the standard runtime in the first place.

Below, I present the results of running each of these programs with hyperfine to measure performance.

aneesh@earth:~/$ hyperfine -N --warmup 10 ./test
Benchmark 1: ./test
  Time (mean ± σ):       1.4 ms ±   0.2 ms    [User: 0.3 ms, System: 1.0 ms]
  Range (min … max):     0.2 ms …   1.9 ms    1768 runs
 
aneesh@earth:~/$ hyperfine -N --warmup 10 ./test_static
Benchmark 1: ./test_static
  Time (mean ± σ):     899.5 µs ± 112.0 µs    [User: 293.4 µs, System: 500.8 µs]
  Range (min … max):   149.8 µs … 1644.0 µs    2881 runs
 
aneesh@earth:~/$ hyperfine -N --warmup 10 ./test_static_no_libc
Benchmark 1: ./test_static_no_libc
  Time (mean ± σ):     374.8 µs ± 167.3 µs    [User: 187.3 µs, System: 110.6 µs]
  Range (min … max):    79.0 µs … 922.8 µs    4231 runs
 

We see very significant speedups at each stage! Let’s apply these techniques to argparsh.

De-pythoning argparsh

For simplicity, argparsh used to defer the actual argument parsing to python. This was done by using the PyO3 crate which dynamically links python. In effect, every single invocation of argparsh, was loading and initializing the python dynamic library which adds significant overhead. Thus, it was time to remove python from argparsh entirely.

I’d always wanted to have a pure Rust implementation of argparsh, but the tedium of re-implementing python’s argparse in Rust was not an appealing task. But the world is a worse better different place today than a few years ago, so I tasked an agent with the rewrite.

Static’ing argparsh

With python removed from argparsh, getting a static build was very straightforward. Rust supports a static build target by linking against musl libc.

nostd

Rust has a nostd mode in which the application is compiled without a runtime. While most of the crates used for argparsh either support nostd or can be easily, implemented without std this is not the case for clap. clap is the argument parser that argparsh itself uses, and it’s a really nice argument parsing library. Unfortunately clap doesn’t support nostd yet today.

So, for now, I created a separate argparsh-nostd-demo crate that builds argparsh without std at the cost of having poor ergonomics (argparsh itself’s argument parsing errors/helptext not fully supported). Additionally, this binary does not support the parse functionality that instantiates the argument parser - it only supports building the parser on stdout.

Benchmarking

1. How do we determine if we made argparsh faster?
2. How do we know how each of the changes we made affected argparsh performance?

For (1), I defined the following benchmark script based off of one of the argparsh examples.

#!/bin/bash

PARSE_PARSER=${PARSE_PARSER:=$PARSER}

# Create a parser program
parser=$({
  $PARSER new $0 -d "argparsh example" -e "bye!"
  $PARSER add_arg \
    --choice a --choice b --choice c \
    --helptext "single letter arg" \
    -- "a"
  $PARSER add_arg --type int --default 10 -- "-i" "--interval"
  $PARSER add_arg --action store_true -- "-f"

  $PARSER add_subparser foobar --required
  $PARSER add_subcommand foo
  $PARSER add_subcommand bar

  $PARSER add_arg --subcommand foo "qux"
  $PARSER add_arg --subcommand bar "baz"
})

# Parse cli arguments as shell variables prefixg ed with "arg_"
#   cli arguments can be placed in the environment with "-e" or "--export"
#   cli arguments can be declared as local with "-l" or "--local"
eval $($PARSER parse $parser -p "arg_" -- "$@")

echo "Parsed args as shell variables"

Then (2), I ran the benchmark with each of the changes I listed above using hyperfine. The benchmarking environment ran on a Intel i9-149000K CPU running linux 7.0, GLIBC 2.39, rust 1.94.0.

# Baseline
Benchmark 1: env PARSER=argparsh bash bench.sh a -i 100 foo qux
  Time (mean ± σ):      64.4 ms ±  12.4 ms    [User: 37.8 ms, System: 27.0 ms]
  Range (min … max):    48.3 ms … 105.7 ms    27 runs

# No-python
Benchmark 1: env PARSER=./target/release/argparsh bash bench.sh a -i 100 foo qux
  Time (mean ± σ):      39.0 ms ±   1.7 ms    [User: 10.0 ms, System: 30.3 ms]
  Range (min … max):    29.2 ms …  42.4 ms    73 runs

# static
Benchmark 1: env PARSER=./target/x86_64-unknown-linux-musl/release/argparsh  bash bench.sh a -i 100 foo qux
  Time (mean ± σ):      31.7 ms ±   1.4 ms    [User: 8.1 ms, System: 25.0 ms]
  Range (min … max):    29.0 ms …  35.6 ms    83 runs

# nostd (except for parser)
Benchmark 1: env PARSER=./nostd-demo/target/x86_64-unknown-linux-musl/release/argparsh PARSE_PARSER=./target/x86_64-unknown-linux-musl/release/argparsh  bash bench.sh a -i 100 foo qux
  Time (mean ± σ):      18.6 ms ±   4.9 ms    [User: 5.8 ms, System: 14.0 ms]
  Range (min … max):     2.1 ms …  23.3 ms    118 runs

We can see that each change had an improvement. Just removing the dependency on python yielded an 1.65x speedup. Moving from dynamic to static linking had a more modest, ~20% impact, but going from static to nostd provided another 1.7x speedup. The overall speedup was 3.46x!

That’s all for now. Hopefully as I continue using argparsh there will be more to share here.