This is the third in a series of posts describing the system I built to monitor long-running commands.

The second post explained my twait script, which waits for a command in a separate tmux pane or iTerm window to complete, and then exits with the same code.

The easiest way to get notified when twait finishes is to chain the commands:

twait %22; notify

But this ignores the return value, which is a useful part of the notification.

You could solve this by passing the return value as an argument:

twait %22; notify $?

The $? variable contains the return code of the last command.

I mostly use a script called mon that takes the other command as an argument:

mon twait %22

Since mon is running the command, it has access to its exit code. It can also do some prep work before running the command. The downside is that you have to keep the command fairly simple; using pipes, multiple commands, or shell aliases does not work properly.

Here is the code to my monitor script, which lights my blink LED and sends a Pushover notification.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

#!/bin/bash
set -eu

function blink() {
  blink1-tool -m 0 "$@" > /dev/null
}

blink --rgb 170,170,170
echo "Monitoring $@"
"$@"
rc=$?

case $rc in
  0)
    blink --green
    title="Process succeeded"
    ;;
  1)
    blink --red
    title="Process failed"
    ;;
  *)
    blink --blue
    title="Process failed"
    ;;
esac

pushover "$title" "Exited with code $rc"

This sets the LED to a dim white at the beginning of the command, prints a message, and then runs the command. The $@ variable contains all of the arguments to the script, which should be the command to run, followed by its arguments. When we are done with the command, we capture the return code.

Depending on the value of the return code, I turn the LED green, red, or blue, and send a message to my phone and watch.

The pushover script uses curl to send a request to the pushover servers.

1
2
3
4
5
6
7
8

#!/bin/bash
set -eu
curl -s \
  --form-string "token=asdfasdfasdfasdf" \
  --form-string "user=asdfasdfasdfasdff" \
  --form-string "title=$1" \
  --form-string "message=$2" \
  "https://api.pushover.net/1/messages.json" > /dev/null

Those are not my token or userid. You have to get your own from pushover.

The final post in this series will show some tmux and AppleScript integrations to make this monitoring easier to launch.