◀︎ Methodology Contents

This started as a list at the bottom of my first post about representation of composers in orchestral music. Now I’ve made it into a page here so that I can keep adding to it. I’ve also added some commentary below the list which I think is interesting, but a little too dry and technical for the regular blog feed.

List of Studies

Baltimore Symphony / Ricky O’Bannon

US Orchestras • 2014/15 to 2016/17 • Size: 22–89 ensembles

When he was at the Baltimore Symphony Orchestra, Ricky O’Bannon wrote a series called “By the Numbers” analyzing the concert seasons of top US orchestras:

Women’s Philharmonic Advocacy

US Orchestras • 2016/17–present • Size: 21 ensembles

The Baltimore series stopped after three seasons, but Women’s Philharmonic Advocacy started doing their own season analysis in 2016/17 with a smaller list of 21 orchestras as a way to highlight the lack of women composers represented in orchestra programs.

Starting with the 2020/21 analysis, WPA has also been tracking gender balance among conductors, which is an interesting and valuable addition. The 2021/22 analysis also graphs out the changes over time with individual ensembles. This helps illustrate that the rate of change has been significant over the last five years, even though the overall proportion of women composers being programmed is still quite low.

Data collection during pandemic seasons has clearly been more challenging, though WPA has still done a great job of consistently collecting information that shows how things are changing over time.

DONNE – Women in Music

Global Orchestras • 2018/19–present • Size: 15–111 ensembles

Another organization, called DONNE, has been doing similar analysis for a more global set of orchestras:

DONNE also has a research page with links to other studies on representation in music, including one on (mostly-classical) French music festivals, which I have put on my reading list.

Institute for Composer Diversity

US Orchestras • 2019/20–present • Size: 120–133 ensembles

The Institute for Composer Diversity was founded within the School of Music at the State University of New York at Fredonia. Their analysis looks at representation of composers from underrepresented racial, ethnic, or cultural heritages in addition to gender.

I have used the ICD’s analysis frequently in my analysis of top classical playlists on streaming services because of its recency, and the depth and breadth of the composer demographics it studied. Notably, it is the only big study I found (before the release of the DONNE 2020/21 study) that examined race in addition to gender and whether composers are living.

Tom Hogglestock

US Orchestras • 2021/22–present • Size: 19–51 ensembles

Tom Hogglestock started blogging about repertoire numbers in 2021 when he was planning which concerts to see, and expanded his work in 2022 with a longer series of posts about the 2022/23 season:

2021/22 (19 Orchestras):

2022/23 (51 Orchestras):

Living Music Report (Australia)

The Living Music Report has been collecting data on programming by Australian orchestras since 2019, including information on living/deceased, nationality, gender, race, and Indigenous identity. A 2021 report is forthcoming.

Full data set on GitHub

Analysis tools to consolidate into one data table

Studies on Canadian Orchestras

So far, the best analysis of Canadian orchestras I’ve seen has been in Tom Hogglestock’s 2022/23 posts. I wrote some commentary about that in my 2022 repertoire report round-up.

I have also come across a couple interesting pieces of academic work examining the programming of Canadian composers:

In 2008, Robert John Fraser wrote a masters thesis which specifically looked at the programming of Canadian composers by Canadian orchestras.

This thesis catalogues performances of orchestral music written by Canadian composers, performed between 1980 and 2005 by six Canadian professional symphony orchestras (Victoria, Calgary, Winnipeg, London, Toronto and Montreal).

In 2020, Isaac Page wrote a masters thesis which looked at orchestral music of the Canadian Centennial.

In 1967, Canada celebrated its centennial anniversary of confederation. Celebrations were marked with many significant events in the decade leading up to the centennial, notably the adoption of a new Canadian flag, the construction of many cultural landmarks across the country, and Expo 67 in Montreal. In addition to these major cultural celebrations, there was a noticeable push to create and promote Canadian art. Approximately 130 compositions were written for the centennial year, with many commissions coming from Centennial Commission grants as well as the Canadian Broadcasting Corporation (CBC). Of those works, 51 were orchestral compositions that were intended to be performed by orchestras across the country. These works form an important collection that is ripe for study into compositional trends of the time.

I think the analysis in this paper of the history and impact of Indigenous cultures on Canadian music is particularly noteworthy. Canadian composers appropriated Indigenous melodies and language as part of their attempt to make music with a “Canadian” identity, while at the same time in Canada “for the first half of the twentieth century, the cultural practices of indigenous communities were either prohibited or severely restricted.”

Even after the Potlatch ban was written out of Canadian legislation in the 50s, the painful state-supported separation of Indigenous people from their cultures continued through the residential school and child welfare systems.

These issues are still going on. Indigenous children are still vastly over-represented in the Canadian foster care system. All this to say, any discussion about what constitutes a “Canadian” music identity has pretty real stakes.

What Each Study Measures

All of the analyses done by Baltimore, Women’s Philharmonic Advocacy, and the Institute for Composer Diversity only examine a subset of orchestra programming: mainstage “core classical” concerts. There’s a lot of interesting things to say about this choice, which is partly about methodology and partly about orchestra culture. I will write a post about it very soon.

The interesting, but more technical thing to say about each of these efforts is that they are all season analyses, but they don’t measure exactly the same things. I think it’s helpful to work through it if you want to understand how to do a season analysis well.

There are lots of different precise statistics you could create for an orchestra season, and which one you use depends on your goals and what you want to understand. I’ve listed each statistic that you could measure below, and given each a corresponding question that describes what each one answers.

I’ve simplified the descriptions below and just referred to “historically-underrepresented composers”, but in reality you would want use these statistics to measure several demographic traits, and some in combination so that your analysis is nuanced and intersectional.

Proportion of concerts featuring (at least one) historically-underrepresented composer

Question: “What’s the likelihood that if I pick a random orchestra concert I’ll hear a work by a historically-underrepresented composer?”

How to measure: Count up orchestra concerts that have at least one composer from a given demographic category.

Measured by: Baltimore occasionally reported this kind of figure (e.g. % of concerts featuring a living composer) but not for all of the demographic characteristics that they studied.

However, the Baltimore data is downloadable and (uniquely) has the concert and number of performances recorded for each piece, so it should be possible to use their dataset to generate this for women composers. It would also be possible to generate a statistic like this for composers of colour if you conducted additional research on the composers in their dataset.

Earlier DONNE reports also presented a figure like this (“% of concerts that include works by women composers”).

Pros and cons: Clearly reveals if curators only put under-represented composers on less popular or less played programs. However, would be overly favourable to curators who program “a mile wide and an inch deep” unless additional thresholds are measured (e.g. proportion of concerts featuring more than 50% under-represented composers in the season).

Proportion of concert programs featuring (at least one) historically-underrepresented composer

This is the same as the statistic above, but you count pieces in concert programs, not in individual concerts. It has a slightly different angle on what it measures about audience and curator intent.

I would argue that most of the curators behind an orchestra season think about deciding which pieces to put on each program as an artistic choice, and deciding how many concerts to have for each program as a logistical/business decision. So most curators might see this kind of statistic as being more in line with how they view the significance of their own role.

Phrasing the question this way also gives more agency and nuance to the audience - just because the target audience for a program is smaller (and therefore fewer concerts are planned during a week), doesn’t mean the concert can’t be incredibly meaningful for the people who are in that target audience.

Performances of pieces - proportion that are pieces by historically-underrepresented composers

Question: “How often are pieces by historically-underrepresented composers played?”

How to measure: Multiply each piece performed by the orchestra by the number of times it was played; calculate proportions based on these performance numbers.

Measured by:

  • Baltimore (their methodology states that “Calculations for the initial findings and infographic are weighed by the number of times a piece of music will be performed in concert.”)
  • Women’s Philharmonic Advocacy (“individual performances”)

Pros and cons: This one is like “proportion of concerts featuring at least…” and “proportion of programs featuring at least…” but not affected by whether the pieces are concentrated in a few programs or spread out across the season.

It’s also not so strongly influenced by whether the average program has three pieces or twenty, so if you are lumping Pops and core classical together into one statistic, this one would probably be better than either of the two above, unless you’re specifically looking at something like “% of concerts with at least 50% of pieces on the program by historically-underrepresented composers”.

Number of unique pieces in the orchestra season by historically-underrepresented composers

Question: “What proportion of the list of pieces played during the season is by historically-underrepresented composers?”

How to measure: Make sure that composers and pieces are consistently named (e.g. avoid cases where you might list the composer as Pyotr Tchaikovsky in one line and Peter Tchaikovsky in another), and do a unique count of unique Composer–Piece pairings in the season (in case there are two pieces with the same name by different composers). Calculate proportions based on these unique Composer–Piece numbers.

Measured by:

  • Not measured by Baltimore, but could easily be generated from their data
  • Women’s Philharmonic Advocacy (“individual works”)
  • DONNE (“works presented”)
  • Institute for Composer Diversity

Pros and cons: This is a much simpler statistic to put together than many of the ones already listed. All you need is a list of the pieces played during the season and their composers. I think a number of the Canadian orchestras that Robert John Fraser studied provided their data in this format. However, it won’t help you understand if pieces by under-represented composers are placed on programs with fewer performances or if they are concentrated into a small number of programs.

One interesting feature of this statistic is that it’s probably the closest conceptual equivalent to a playlist — especially a playlist designed to be played on random shuffle.

A key choice when generating a figure like this one about multiple orchestras is whether to use an average of each orchestra’s average, or to generate a figure based on the global unique count of Orchestra–Composer–Piece. I used the latter option when analyzing the data from the Institute for Composer Diversity.

The difference is pretty negligible, but conceptually, the average of averages approach weights each orchestra the same, whereas the unique count of Orchestra–Composer–Piece implicitly gives more weight to bigger orchestras because they present more performances. This distinction would matter more if, for example, smaller orchestras were much more active in selecting pieces by under-represented composers compared to large orchestras.

Number of unique pieces from seasons of multiple orchestras by historically-underrepresented composers

I’m not sure this is a particularly useful statistic, but it’s the first one that Women’s Philharmonic Advocacy reports in most of their posts — for example, in their 2019/20 report:

Of the 277 individual composers being programmed across all 21 ensemble’s seasons, 53 identify as women – a total of 19%.

The big downside of making a “global unique count” of under-represented composers from multiple orchestra seasons is that it provides no information about how often those composers are played. If a living composer is performed once, by one orchestra in the group, the composer is counted once. If Beethoven is collectively performed 400 times by all those orchestras, Beethoven is also counted once.

I suppose it shows something about how orchestras choose to put effort into adding under-represented composers onto their programs. If the percentage from this metric goes up over time and the percentage of performances of under-represented composers also goes up, then I suppose that would demonstrate that orchestras are mostly doing their own research and/or performing composers that are more local to their community. If performances go up, but the “global unique count” stays pretty static, then orchestras are probably just performing more of the small number of under-represented composers that they see their peers playing.

Other things to consider

So far, the Australian Living Music Report is the only study that I’ve seen include the duration of the pieces and their placement on the program. For the other studies, there’s nothing that would let you measure to what extent pieces by under-represented composers are being programmed outside of the “short feature piece” slot, which I’ve discussed before.

You would have to go back to the source material (season brochures and press releases) in order to gather data on program placement, but duration could theoretically be measured by conducting some simpler additional research using something like the Naxos Works Database.

A Final Thought

If you’re thinking about researching orchestra seasons, think carefully about what you measure, and try to include data about number of performances, program title, position on the program, etc. Even if you choose not to use this metadata in your own analysis, if you publish your data, other people will be able to find a broader range of uses for it.

This page is referenced by the following posts: