Monday, April 09, 2007

TAFTO 2007

As promised, today is the day my article Is TAFTO a good idea? Really? (new URL) (newer URL) has been published at Adaptistration. If you're interested in classical music, I encourage you to take a look and see if you agree with my thinking. If you're more interested in making sense of tough problems, I also encourage you to take a look. I've used some graphs I don't normally see in such modeling work. In either case, I'd be interested in your reactions.

If you happen to be here because you found me on Adaptistration, welcome! You must be a lover of classical music (you may also be an orchestra administrator, a musician, a board member, or all of the above). In that case, you might also enjoy a recent conversation I had with Greg Sandow called Making musical sense by email (new URL), and you might like a short exploration I did of a statement in the recent Knight Foundation Magic Of Music Final Report called Making sense with numbers (new URL).

I'd enjoy having you as a regular reader of Making Sense With Facilitated Systems, and I'd welcome your comments either here or on the Adaptistration article.

Labels: , ,

3 Comments:

Blogger buttsintheseats said...

Hi-

I came over here via the TAFTO posting on Adaptistration. I have read your entry over there twice and am starting to understand your model.

I am glad you posted this entry. I had come by the Magic of Music entry by a circuitous route and was afraid you wouldn't see my question back there at your Nov 20 entry.

When I had read the statistic that 74% of orchestra attendees had played an instrument or sung at one time, my assumption was that this was being presented as a trait of a person who might be inclined to attend later in life. I didn't see it as suggesting that handing out violins will create a highly efficient chemical reaction producing a music attendee.

My question is, is even this an erroneous assumption? Have I been taken in by statistics presented in just the right way?

It seems to me that if they asked how many people had ridden in a Mercedes or BMW at any point in their life, they would have arrived at a similarly high percentage. It appeared to me that they were simply identifying an affinity group.

One of the reasons I am concerned is because I had just cited those statistics from the report not two weeks ago to a group. I presented in the context I have mentioned above, but I am wondering if I was wrong in doing so.

I also wonder if people may have interpreted what I was saying the way you suggested in your entry. I agree that it would have been more helpful to know how many bought tickets given that they had played or sang.


One thing about the entry I do disagree with is the close connection of the 74% attendance later in life statistic with the observation there was a strong correlation between participatory music education programs and attendance. I thought it was a little misleading. There were about 20 lines of text between the them and the latter observation was clearly being used in contrast with the value of assembly based programs.

09 April, 2007 22:24  
Blogger Bill Harris said...

Thanks for your thoughtful comments and question. (While I don't yet see a comment from you on my Magic of Music entry, I do see your comments on your blog. I presume that's what you mean.)

To get a definitive answer to your questions, we should consider inviting Dr. Wolf, the author of that report, to participate in this discussion. Because of a lack of time this morning, I'll make a few guesses, but I'd welcome his setting us straight.

First, you raise a possible interpretation of the 74% claim I didn't really consider, one that's consistent with a comment from Greg Sandow in a recent conversation: "That is, people might go occasionally when they're young, then not go (or not go very often) for many years, and then resume going, much more often, when they're older."

I think the Knight Report was simply saying that there is a certain probable (and structurally unspecified) connection between playing music as a child and attending concerts as an adult. Both you and Greg have conjectured that people learn to like music, drift away, and then return at a later stage in life.

That could make the number of people, say, age 50 and up who played an instrument or sang when they were age 17 or younger a more appropriate statistic. I don't think I have the data to estimate that, but I can see intuitively that it might be a more sensitive indicator if your conjecture is true, for it wouldn't be clouded with all those people from 20 to 49 who have temporarily drifted away (according to the conjecture). The third point below still holds, though.

Second, you mention a potential correlation between concert-goers and people who had ridden in a Mercedes or BMW at some time. That's the challenge with statistics such as this: is the relationship between two events (playing an instrument and attending orchestra concerts) causal (that is, does one cause the other?), or is it simply correlational? In the latter case, both may be caused by some other, unspecified factor. I think that's what you mean when you write, "... this was being presented as a trait of a person who might be inclined to attend later in life. I didn't see it as suggesting that handing out violins will create a highly efficient chemical reaction producing a music attendee." I agree that the Knight Report doesn't really claim causation, but they don't stop us from falling into such an interpretation (see point 8 on page 50 of the report).

Third, the question still stands as to how to interpret the 74%. I think we see numbers such as that from time to time because those numbers are often easier to get than the number we want. In this case, they were presumably surveying people who had attended concerts to find out more about them. Surveying concert-goers meant they could limit the number of people they surveyed; they didn't need to survey the general population. While they were talking with concert-goers, it was easy to find out such facts as the probability that they played an instrument, given that they were concert-goers.

Unfortunately, as the analysis shows, that's not what we need to know. We'd be better served had they surveyed all the people who played an instrument or sang when they were young and asked them if they attended concerts now. That's hard and expensive, though. They'd have to come up with all possible factors of interest and then commission a separate survey for each one—a far bigger task than doing one survey of concert-goers.

If they had applied Bayes' theorem to the numbers, they'd have what we need at far less cost than doing multiple surveys.

Why don't people catch the distinction? First, I think the words we use don't make the distinction clear. I suspect that many of us, if we read "74 percent of them had played an instrument or sung in a chorus at some time in their lives" quickly as part of a longer report, wouldn't be certain whether that was the probability of a concert-goer being a former player or the probability of a former player being a concert-goer.

Second, I don't think we teach probability and statistics broadly, or, if we do, we tend to forget some of the material. If people haven't studied conditional probabilities and Bayes' theorem, they might not think there's a difference between the two probabilities. Even if they think there might be a difference, they might assume the writer is giving them the number they need.

Does that answer your question at all?

10 April, 2007 12:31  
Blogger buttsintheseats said...

Thanks for your response. What I submitted above I originally intended to write at your Nov 20 entry. I was glad you referenced it here because then it was appropriate to pose my question where you might see it. My blog entry was more a result of the thoughts churning after I wrote here.

I am glad to learn I viewed the information similar to how Greg Sandow did. At least then if I wasn't entirely right in my interpretation, I probably wasn't entirely wrong either!

My use of the BMW/Mercedes correlation was, as you surmised, not an assumption of a casual relationship. Rather, it was based on the idea that people with an affluent background were more likely to attend an orchestra performance. I was suggesting that if one were to survey an orchestra audience one would find a high number of people who had ridden in one of these vehicles at some time in their lives even if they now drove a Land Rover.

The point I was trying to make was that I didn't think handing out musical instruments was going to guarantee people would attend a concert any more than giving them rides on a Mercedes would.

I would, however, expect that if I were to attend a concert and asked if people had ridden in a BMW/Benz or participated in music in their pasts, there would be many people who answered yes to at least one of these.

As for statistics education. I confess I had classes in the subject as both an undergrad and grad student and my eyes glazed over in both. I am familiar with Bayes theorem but haven't had cause to apply it much since graduation.

Though honestly, that is probably more by choice than opportunity. I read studies like Magic of Music regularly enough and could perform the same analysis you did with the information at hand.

Thanks again for taking the time to make a response.

10 April, 2007 20:24  

Post a Comment

<< Home