This is the fourth and last in a series of posts
describing the system I built to monitor long-running commands.
The third post explained my mon script, which together
with my twait script from the second post,
lights up an LED when a command in a tmux pane or iTerm window completes.
If I want to monitor a command, I use mon twait ID, where ID is the pane or
window ID. Or I can run another command only if the first command is successful,
by writing twait ID && ./other_command.
Of course it would be annoying to type the window ID manually, so I have two
keyboard shortcuts. One to start mon twait ID in a separate window that closes
automatically when it is done. The other to open a new window with the text
twait ID && followed by a space.
Tmux integration
Tmux can be scripted through the use of unix-like commands, with arguments and
options. These can either be run from the command line or assigned keyboard
shortcuts.
I use Ctrl-A plus uppercase E for the first.
bind E run "tmux split-window -l 5 \"mon twait #D\""
This splits to form a new window, 5 lines high, and runs mon twait. The #D
parameter in a tmux run command gets translated into the pane ID.
I use Ctrl-A plus lowercase E for the second.
bind e run "tmux split-window; tmux send-keys \"twait #D &&\""
iTerm integration
You can do very similar things with iTerm using AppleScript.
To open the monitor in a temporary pane, I use this script.
It is possible to control the height of the new pane, but I haven’t
figured it out yet.
This is the third in a series of posts
describing the system I built to monitor long-running commands.
The second post explained my twait script, which
waits for a command in a separate tmux pane or iTerm window to complete,
and then exits with the same code.
The easiest way to get notified when twait finishes is to chain the commands:
twait %22; notify
But this ignores the return value, which is a useful part of the notification.
You could solve this by passing the return value as an argument:
twait %22; notify $?
The $? variable contains the return code of the last command.
I mostly use a script called mon that takes the other command as an argument:
mon twait %22
Since mon is running the command, it has access to its exit code. It can also
do some prep work before running the command. The downside is that you have to
keep the command fairly simple; using pipes, multiple commands, or shell aliases
does not work properly.
Here is the code to my monitor script, which lights my blink LED and
sends a Pushover notification.
This sets the LED to a dim white at the beginning of the command, prints a
message, and then runs the command. The $@ variable contains all of the
arguments to the script, which should be the command to run, followed by its
arguments. When we are done with the command, we capture the return code.
Depending on the value of the return code,
I turn the LED green, red, or blue, and send a message to my phone and watch.
The pushover script uses curl to send a request to the pushover servers.
This is the second in a series of posts
describing the system I built to monitor long-running commands.
The first post explained how I write a text file of the form
$HOME/.local/logs/return-$ID every time a command finishes.
The ID variable is the unique identifier for the tmux pane or iTerm window.
Each file contains a single line with the return code of the last command to
complete in that window.
Now that a file is being written every time a command finishes, it is not
difficult to wait for a command from a given window to finish.
I call this script twait, meaning “wait for terminal.”
It takes a window’s unique identifier as its only argument,
waits until the corresponding return file is modified,
then exits with the status read from that file.
This uses inotify (on my Linux machine at work) to wait until
someone writes to the return file. Then it reads the entire contents of the
return file (using cat) and uses that as the exit code for the script.
This is a suprisingly useful command. Although I originally built it with
only monitoring in mind, it serves as a useful way to queue up any follow-up
command. In one window I can write build,
and in a separate window I can write twait %22 && ./run. This will wait for
the build to finish, and as long as it is successful, run some other command.
It is
not uncommon for me to have a chain of three or four tmux panes stacked
vertically, each running one of a series of commands. It makes it very easy to
quickly visualize the progress of the command sequence.
In the next post, I will show how I use twait to do the monitoring,
and in the final post of the series I will share some tmux/iTerm integrations
that make it easier to use both twait and my monitoring script.
This is the first in a series of posts
describing the system I built to monitor long-running commands.
For me, this started as a way to light a blink 1 LED
at work when a long build completed,
bringing me back to the more important task from whatever may have
distracted me.
After a recent post by Casey Liss, I’ve added watch notifications to
the mix.
The problem, for me at least, is that I never know when a command is going to
take long enough that I lose interest and want a notification,
so appending a command at the end of the original is not an option:
build; notify
You could always run build, and then when you realize you want notified, type
notify blindly, but any stray keys or sometimes even mouse scrolling might
mess up the follow up command. (Or whatever long-running command you are using
might be capturing stdin.)
What I really wanted was a way for any terminal window to wait on any other
terminal window.
And of course, it would be nice to be able to do something different depending
on what the return value is.
The first step is to capture and log the return value of every command in every
window. Not long ago, this would have seemed ludicrous, but with SSDs, it really
isn’t a big deal to write to disk once per command.
Unique window identifiers
To be able to tell one command from another, you need some kind of an identifier
for each window.
At work, I always use tmux, which calls windows panes,
and gives each pane an incrementing identifier,
stored in the TMUX_PANE variable.
At home, I use iTerm2,
which sets the TERM_SESSION_ID variable.
This variable actually has two parts separated by a colon, and the unique id is
in the second part, which is accessible using parameter subtitution
as ${TERM_SESSION_ID#*:}.
Log the return value
Both zsh and bash can run an arbitrary command right before drawing the prompt,
that is, right after any command finishes. This is a perfect place to capture
and log the return value.
In zsh, this is precmd, so you can define something like this in your
zshrc (using tmux pane):