My weekend project was to fix a leaky valve and a leaky connection for the irrigation system at my house.

In the ten years I’ve lived here, I’ve replaced just about every sprinkler and a bunch of underground PVC, so I know some tricks, but I’m not an expert.

(Also my first “real” job as a 16-year-old was for my electrician neighbor, so I’ve been gluing PVC in some form or another for over 25 years.)

The most recent trouble spot was this valve manifold, where the incoming supply splits three ways for my three sprinkler zones, shown here after the project was finished.

A few years ago, the middle valve started leaking. I didn’t want to replace it because there wasn’t a shutoff valve, so I had a professional fix it and add a shutoff (not visible in the photo). Other than that change, he replaced it exactly as it had been, that is to say, a bunch of glued-together PVC.

One year ago, the left valve started leaking. There is no way to screw it off while it is attached, so I did the same thing the professional did, cutting and gluing new PVC, making use of the new shutoff valve.

As I mentioned, I’ve done this a lot at other places in the system, but one difference here is that one side of the valve is under constant pressure, unlike the sprinklers and underground connections, which are only under pressure when the sprinklers are running.

For my change a year ago, I didn’t quite screw the PVC connector hard enough into the valve. From the first day, it had a very slow drip. Barely enough to wet the ground, and not enough to convince me to recut and reglue, but enough to make me a little sad every time I walked past.

In the last month or so, the wet spot started to grow, both because the left connection was getting leakier, and the right valve was starting to fail as well.

I’m sure for a professional, it makes sense to just glue everything together, because they have figured out how to properly screw a connection in a way that doesn’t leak. But since I’m not a professional, I decided that this time I would give myself a way out.

Instead of gluing everything directly together, I installed two unions on each side. This way I can unscrew the entire unit and redo any connections if necessary. And the unions use an O-ring to keep the connection watertight, instead of my nemesis the NPT connection.

I didn’t replace the left valve since it was still good, but I did replace the connectors. And then of course I replaced the leaky right valve. I considered adding a union to the center valve, but decided to leave that for another day.

This time, I managed to get the left side tight enough (even though I wrapped the thread seal tape in the wrong direction), but I left the right side leaky. So my extra work paid off: I removed everything at the two unions, tightened up the leaky connection, and connected it together again. Voilà, for the first time in a year, the concrete is completely dry.

The lesson learned, I guess, is that as a newbie you sometimes have to over-engineer things to make up for your inevitable mistakes.

Yellow (white)

New road

South Coast Botanic Garden, Palos Verdes Penninsula, California

The Dodgers are guaranteed a playoff spot after tonight’s win, but the math isn’t straightforward. They need to be ahead of either the Diamondbacks or the Padres at the end of the year. The only way the Diamondbacks can catch the Dodgers is to win their last 12 while the Dodgers lose their last 12. In that scenario, the best the Padres can do is win 10 of their last 13, since they play three against the Diamondbacks. That would put the three in a 3-way tie, and the tiebreakers would go Diamondbacks first, Dodgers second, and Padres third.

I lived a real-life Californians sketch today picking up my kid from college for the weekend. On a 45-minute drive, I took the 405 to the 110 to the 91 to the 710 to the 105 to the 605 to the 10.

This week my oldest child begins studying civil engineering at Cal Poly Pomona. I’m really proud of and excited for them! The youthful energy of a college campus is something else.

Here is a little test you can do to play around with FIFOs (also known as Named Pipes) in bash.

For setup, we run

mkfifo test

which creates a FIFO named test in the current directory.

Then run

cat - < test

This opens test for reading and connects it to stdin, then runs cat -, which copies stdin to stdout. Nothing will happen (for now) because there are no writers for test.

In a separate window, execute

echo "hi" > test

You will see your first window print “hi” and then the cat command will exit.

The reason that cat exits is because as soon as the echo command finishes, there are no writers connected to test. In that case, anyone who tries to read from the pipe will get an EOF, which is what cat uses to know it should move on to the next file or exit. You can see this in action by rerunning the cat command and then running

sleep 60 > test &

Now you can run as many “echo” commands as you want (for 60 seconds) and cat will keep running and printing whatever is written to the pipe.

The reason that cat doesn’t just return before the first echo (after all, there are no writers at the beginning) is that Bash tells the OS to wait for a writer to connect before executing its command. You can see this by replacing cat - with

{ echo "starting"; cat - } < test

You will see that “starting” is not written to stdout until the first writer is connected (either the first echo or the sleep command).

Read-write file descriptor

A better way than sleep to keep a writer open is to use a read-write file descriptor:

cat - 3<> test < test

This starts by opening test for reading and writing and connecting it to file descriptor 3. Then it also opens it for reading and connects it to stdin.

We won’t actually use file descriptor 3 to pass data, but importantly, the <> operator is nonblocking, so it doesn’t matter that there are no other readers or writers yet. It also remains open for the life of the cat command, so cat will start immediately and never see EOF on its stdin.

Minecraft

I started on this investigation while trying to run a persistent Minecraft server for my kids. Minecraft works by writing logs to stdout and reading commands from stdin.

My runit script to keep this running in the background looks like this (omitting some details such as error handling and the java options):

#!/bin/sh

cd /var/opt/minecraft
mkfifo fifo

exec chpst -u minecraft java \
    -jar /opt/minecraft/server.jar \
    nogui 3<> fifo < fifo

Then if I need to send a command to the running server, I can run, for example,

echo "op user0293" | sudo tee /var/opt/minecraft/fifo

If you want to feel emotions and human connection, take your kid to the DMV drive test and chill with the family members of the other testers while your loved ones are out on their drives.

In yesterday’s post about sunrises, I made a snarky offhand comment about certain places being in the wrong time zone.

Today I drew a map that highlights the locations that are more than 7.5 degrees from the central longitude of their time zone. (US time zones are spaced every hour, which corresponds to 15 degrees of longitude.)

If you are in the correct time zone, then your average solar noon is between 11:30 a.m. (if you live on the far eastern edge) and 12:30 p.m. (on the far western edge) Standard Time.

A few states and parts of several others are clinging to the Eastern Time Zone when they should join Central. The Great Plains states and Western Texas should really join Mountain Time. My birth state of Idaho should observe Pacific Time.

Instead, these locations are all running 30-60 minutes later than they should be. And of course, if we do end up switching to full-time Daylight Saving Time, that will become 90-120 minutes, which seems pretty ridiculous to me.

Then again, they already spend two-thirds of the year in that condition, so this misalignment isn’t exactly new.

(You may notice that a tiny sliver of Maine is actually too far east for the Eastern Time Zone, but that one seems like not a big deal.)

Inspired by Dr. Drang’s plots illustrating the potential effects in Chicago of a move to permanent Daylight Saving Time in the US, I wanted to investigate which places would have the hardest time in the dead of winter.

I ended up with this map of the USA Lower 48, with regions shaded by the latest sunrise time that occurred between January 1 and January 15, 2026.

If any location were to switch to permanent DST, their latest sunrise would be an hour later than shown in this map.

I have very little sympathy for the orange and red zones, which currently see their latest sunrise after 8:00 and would move to a post-9:00 sunrise. Parts of Washington, Montana, and Michigan are very far north and there is only so much we can do to help. In all other cases, you are in the wrong time zone anyway; this one is on you.

I do feel for the green areas, which are properly situated but would have to suffer through at least a few sunrises between 8:30 and 9:00.

Of course, I think it is silly to use permanent DST rather than permanent Standard Time. Sure, a lot of our habits have been built up around astronomical noon being closer to 1 p.m. for most of the year, but I’m sure we would figure it out quickly enough.

I’ll be fine either way, since I’m right on the boundary between dark and light blue. I would survive shifting our worst-case winter 7:00 sunrise to 8:00.

Note that, ignoring DST, the latest sunrise is typically in January rather than at the winter solstice. But also in some southern areas (including where I live), the seasonal variation in sunrises is small enough that the latest sunrise time is actually in March or November, right after the shift to DST or before the shift back to Standard Time.

I’m not going to post the Python code I used for this because it was AI-assisted and sloppily thrown together. I used geopandas to read Natural Earth time zone data (both AI choices), and skyfield based on my previous experience with astronomical calculations. I used scipy to solve for the boundary points and then shaded the regions using matplotlib.

Earlier this week, I put together a quick Jupyter notebook to figure out who plays which third-place team in the Round of 32.

I parsed the Wikipedia table into a Pandas DataFrame with 20 columns:

• Twelve columns named 'A' through 'L' with values True or False, indicating that the third-place team from that group qualified for the Round of 32.

• Eight columns named 'vA', 'vB', etc., for the eight first place teams that are slated to play a third-place finisher. The value of these columns is 3A, 3B, etc., for the matchup that will result for the set of third place finishers determined by the values of the first eight columns.

For example, this is the row of the table assuming that the third place finishers from groups A, C, H, and K fail to advance (with True/False shortened to T/F for brevity):

Now I have been whittling down the options with the following Python code, where df is the name of the DataFrame.

from itertools import combinations
from collections import Counter
import pandas as pd

IN = 'FBEID'
OUT = 'H'
ORDER = 'CAG' 
ix = True
for x in IN:
    ix &= df[x]
for x in OUT:
    ix &= ~df[x]
for x, y in combinations(ORDER, 2):
    ix &= (~df[x] | df[y])

print(f'Options remaining: {sum(ix)}')
counters = []
index = ['vA', 'vB', 'vD', 'vE', 'vG', 'vI', 'vK', 'vL']
for x in index:
    counters.append(Counter(df[ix][x]))
dff = pd.DataFrame(counters, index=index).fillna(0).astype(int)
print(dff.replace(0, ""))

The ix variable is a Pandas index selector. For each character of IN, the corresponding column must be True. For each character of OUT, the corresponding column must be False. And finally, for each of the undetermined teams in ORDER, which are sorted by their current ranking, it cannot be that the lower-ranked team gets in while the higher ranked team does not. In Boolean logic terms, this is “x implies y”, which is the same as “(not x) or y.”

Then sum(ix) will get you the number of selected columns. Finally, I arranged everything into a new table that has as its rows the vX labels, as its columns, the 3X labels, and as its values, the number of scenarios left for which that matchup is possible:

Options remaining: 8

   3E 3C 3J 3G 3B 3D 3I 3A 3F 3L 3K
vA  7  1                           
vB        1  7                     
vD              8                  
vE                 8               
vG        2           2  4         
vI                          8      
vK  1                 3        4   
vL        1           3           4

You can see from this that one of the most disruptive scenarios left would be for 3C to play against 1A, but there is only one scenario left in which Scotland makes it through at all, which is that every single match today goes their way. That would mean Croatia losing by 3 or more, Congo failing to win, and some kind of lopsided win between Austria and Algeria.

I rent a post office box. It is an unnecessary expense, but I enjoy a walk every now and then to pick up the mail. My youngest child often joins me. He enters, takes a big sniff, and tells me it is his favorite smell. It smells like mildew.

I also have a return address stamp, typeset in Gorton Perfected, part of Marcin Wichary’s Shift Happens project.

If my 7-year-old ever sees me “hacking,” that is, typing terminal commands, he asks to help, which means me transcribing commands for him to type.

I started to explain IP addresses to him after he configured part of the network, but he told me he knows what an IP address is, he saw it on Magic School Bus.

With my new home server, I’ve made some changes (improvements, hopefully) to my backup system.

Previous backup systems

To back up my Mac, I’ve always started with Time Machine. It has good operating system support, and it can do frequent incremental backups, even when you aren’t connected to the backup disk. It isn’t great for long-term backups though, because every year or so, it gets corrupted and you have to start over from scratch, losing old versions of your files. It seems to be worse when backing up over the network, so I only use Time Machine to a directly connected hard drive.

I’ve used Backblaze, which does internet backups. You can also configure it to keep old versions of your files, although beyond a certain point you have to pay per gigabyte. Backblaze has been pretty reliable, although if you have multiple devices, the per-month price starts to add up.

I’ve used Arq, backing up either to a network drive or to Backblaze B2. This seems to work pretty well. The concept is very similar to Backblaze except that it is more flexible on where you put the data. I don’t use this on my main machine any more, but my kids do on their laptops because it is good about sending data over the internet to my home server (using RustFS to run an S3-compatible server).

ZFS network backups

All of these backup systems do somewhat complex things to allow you to store multiple versions of files, which is a very important feature for a backup system, in case you accidentally delete or change something and you want to get it back.

A different strategy is to make the backup client dumb but the storage system smart. Since I’m using ZFS for my home server, I can do snapshotting and versioning there. Not only does this simplify things for the client, it gives me more flexibility, for example, to keep many snapshots on the large home server but only send one or two versions to the smaller offsite backup drives.

SuperDuper

SuperDuper focuses on exact copies of the entire filesystem. It typically uses an APFS-formatted external drive, but it also has support for network shares. As of MacOS 26, these network backups use Apple’s new sparse image (ASIF) format. On the server side, this looks like a giant file big enough to store all of your data.

After each backup, I have SuperDuper trigger a ZFS snapshot, which allows me to travel back in time to see a day’s backup even after future SuperDuper “Smart Updates” have changed the live version of the sparse image on the server. And ZFS doesn’t have to store an entire copy of each sparse image, just the parts that have changed from one instance to the next.

You don’t get a fancy browser that can see all former versions of a file, the way that Time Machine does. From MacOS’ point of view, you have an entirely separate sparse image per backup, just stored in a way that they don’t have to duplicate unchanged data. You do get all of the Apple-native file features, since the backup targets are full APFS filesystems.

Carbon Copy Cloner

Carbon Copy Cloner is more flexible when making copies to an attached hard drive, including using APFS snapshots to store multiple versions of a file, the way that Time Machine does. But it is much more limited when using network storage: it copies individual files and doesn’t support versioning or other APFS features.

I find it useful, especially when combined with ZFS snapshots, because it makes it easy to see a file’s change history over time. It is more transparent than sparse images, even if it is less faithful.

Backups of a ZFS system

I also want an offsite backup, to make sure that I’m safe in case both my Mac and home server are taken out at the same time. For this, I use an old hard drive big enough to store just a few of the latest snapshots, and a Perl script called syncoid (part of the snapshot manager sanoid).

The backup script looks essentially like this:

syncoid --identifier=offsite --compress=none \
    --recursive --sendoptions=w \
    --delete-target-snapshots \
    main offsite/main

The first time this is run, it creates a snapshot of the ‘main’ dataset and then copies that snapshot to the offsite dataset, which is stored on the external drive. The identifier is used as part of the snapshot name, in case you want to set up multiple backup targets.

On subsequent runs, it creates a new snapshot, and then sends everything that has changed since the previous syncoid snapshot (including any snapshots that you make by other means) to the backup target. Then it deletes the previous snapshot.

Compression is turned off because I am copying to a USB hard drive, not over the network (although syncoid will work over SSH as well). The ‘w’ option sends raw encrypted datasets, which isn’t strictly necessary since you can always configure the target to re-encrypt everything. Deleting the target snapshots means that if you delete a snapshot on the source, it gets deleted from the target.

I keep two identical backups, one connected that is receiving daily syncs, and the other at my office. Every 2-3 weeks I swap them, so that the office copy isn’t more than a month out of date.