Of the many elements of the Phish.net feature set, one that often catches my curiosity is Trey's Notebook. It identifies songs most likely to be played at each show, given songs played in the previous year but not the previous three shows.

For upcoming shows, it's an algorithmic prediction ("Here's what you might expect to hear...") that often works remarkably well, such as predicting 68% of the 22 songs played three nights ago in Rochester. But for previous shows, focus on those percentages themselves rather than the list of songs, and Trey's Notebook becomes a measure of the extent to which Phish's setlists are predictable.

That varies widely, as this first chart illustrates. A handful of early shows were completely predicted (100%!), but many were predictive #fails (0%). Shows in 1990-93 were generally less predictable than shows before or since, largely as a function of the repertoire expanding during that period. And there's a general pattern, marked here with a fifth-order polynomial trendline, in maroon, though nothing stark. (Note that this scatterplot replaces an earlier, clunkier lineplot.)

Read more...