@jdonovan said:

"My previous research ( https://phish.net/blog/1539388704/setlists-and-show-ratings.html ) demonstrated that show ratings (with equal rater weights) are significantly correlated with setlist elements such as the amount of jamming in a show, the number and type of segues, the relative rarity of the setlist, narrative songs, and other factors."

If we believe to know the majority of setlist elements that typically correlate to a rating, would it make sense to list those elements for a user to individually rate when they go to rate a show? Subsequently a formula could crunch those numbers together to generate an "overall rating". Personally, I think I would stop and think twice about each category and probably be less impulsive with my rating after attending a show. Yes, it could drive some people away from rating due to the "survey-ness" of the approach, but at least you're able to understand and measure the impact of a rating based on those elements.

The findings in question were derived from 3.0 shows only, from 3/6/09 through 12/31/17. We don't know whether the same elements would correlate as reliably with higher ratings for shows from 1983 to 2004. We also don't know whether those elements have continued to exert the same effects on ratings from 2018 to present.

Similarly, we don't know whether a mathematical sum of ratings applied to each of the identified elements would be reliably representative of a given fan's global impression of a given show.

Ultimately it might be impossible to deviate from a simple average of user ratings, where each rater gives a single rating to a given show, without imposing distortive biases and introducing new layers of systematic inaccuracy.