Showing posts with label Left-Brain Specials. Show all posts
Showing posts with label Left-Brain Specials. Show all posts

February 27, 2010

2010 Olympics Week 2

I've been a negligent blogger this week due to other committments; but to be honest I've been enjoying my role as an Olympic spectator a lot more since I stopped writing about my medal prediction every day.

I promise (threaten?) that I'll have lots more to say when the Games are over about Own the Podium, the team's overall performance, and the post-mortem on my medal prediction. For the moment I am just going to give the last remnant of the prediction.

Now that we're down to the last two days of competition, and with Canada's (medal or non-medal) fate already decided in a few of the events, I don't really have the statistics of "large numbers" on my side. As a result, some of the numbers here are a bit meaningless.

Nevertheless, here they are. With teams already in the men's curling gold medal game, the men's hockey gold-medal game, and the men's speed skating team pursuit, team Canada is guaranteed to take at least 24 medals. I am showing the most likely number as 25 (45%), followed by 26 (30%). (The two best chances are in the four-man bobsleigh, and the men's snowboard parallel giant slalom, both today.) Altogether, there is an 83% chance that Canada will set a new national record for total number of medals at a Winter Olympics.

Canada has already set a new national record for the most gold medals won at the Winter Olympics, with 10. The previous best was 7 in Salt Lake City and Torino. I have not been able to confirm this, and I have not seen it written anywhere else — but as far as I have been able to determine, Canada has never won more than 10 gold medals in any Olympic Games, summer or winter. The previous highest total was 10 at the boycotted 1984 Olympic Games in Los Angeles.

February 19, 2010

2010 Olympics Day 6: Defiant Fatigue

The COC issued this flash quote from Canadian biathlete Megan Tandy after the 15 km race on Day 6:

I felt pretty decent on the ski trails, defiantly fatigued on the last loop though.

To all my friends on the Mission Team: I wish you defiance in your fatigue as you start Day 7!

On another somewhat superficial note, Canada witnessed another one of Maëlle Ricker's talents tonight. If you didn't hear it, the Snowboardcross Olympic champion stepped into the CTV broadcast booth for a chat and started giving great commentary on the half-pipe competition she was watching. She sounded like a pro.

The usual medal table and picture are presented below. Day 7 is looking like a great chance to get on the "plus" side of my original prediction, and also a great chance for the Whistler team's first medal. For those of you in North America who are supposed to be working today, I'll point you to the great Vancouver 2010 official web site, where (for example) you can monitor live race splits in progress in the Men's Super G this afternoon.

I also love the GeoView presentation of the 2010 Athletes.

Sport Event Athlete Category Result Impact
Initial Prediction         27.05
Day 1         -0.75
Day 2         +0.45
Day 3         -0.50
Day 4         +0.00
Day 5         -0.70
Speed Skating - LT 1,000 m - W Nesbitt, Christine Lock GOLD +0.10
Speed Skating - LT 1,000 m - W Groves, Kristina Outside 4th -0.10
Figure Skating Singles - M Chan, Patrick Possibility 5th -0.35
Snowboard Halfpipe - W Nicoll, Mercedes Outside 6th -0.10
Alpine Ski Super Combined Brydon, Emily Outside 14th -0.10
Day 6         -0.55
Current         25.00

February 18, 2010

2010 Olympics Day 5

Unnoticed Noteworthy Performance for Day 5

For Day 5 I had a tough time finding an unexpected, non-medal, overachieving performance for Canada. It was one of those days. Due congratulations to Marianne St. Gelais for her silver in the short track 500 m yesterday, but the medal disqualifies her from consideration.

I'm going to name two co-recipients for yesterday. Both of them probably had dreams of getting on the podium, and missed; nevertheless I think that the performances were noteworthy and overshadowed.

Whistler native Britt Janyk finished sixth in the women's downhill yesterday, the best Canadian result in the premier alpine event since 1994. American Lindsey Vonn absolutely dominated the field, with her teammate Julia Mancuso the only competitor to come within a second of her time.

The second co-recipient for Day 5 is short track speedskater Jessica Gregg. Gregg did not have a great race in the 500 m final last night, but she got into the final by being great in the preliminary rounds. She finished fourth after a restart. Gregg will turn 22 in March, and is in her first Olympic Games.

Medal Prediction

Alright, I know I'm falling behind! It's nice to know that somebody's reading … even if it is to nag.

Canada "lost" another 0.7 medals yesterday, compared to my prediction. The short track women's 500 m was a good chance for two medals, so to win one was not enough to boost us back over my predicted pace.

I suspect that there are a number of Canadians starting to think that reaching the top of the medal table is going to be very tough. I agree. In fact, I agreed from the outset. To add to that feeling, the US team has exceeded my expectations a bit. However, there is still lots of reason to think that the highly-publicized Own the Podium goal can be achieved.

First of all, let's look at my own prediction. It's true that so far the team is tracking a little bit under my initial prediction, as shown in the figure at the end of this post. But don't forget that I have tended (in the past) to be pessimistic about Canada's chances. And even according to my prediction, after the results of day 5 our chances of reaching 30 medals or more has decreased from about 25% to about 9%. A couple of days of winning two medals per day would recover essentially all of what's been "lost."

Second, let's look at some of the other predictions that were made before the Games. As previously noted, the Associated Press predicted that Canada would win 30 medals, and that they would have won 7 by the end of day 5. So we're only "down" by one on that front. Sports Illustrated also predicted 30 medals, but only six by the end of day 5, so we're even on that one. And the Canadian Press predicted a ridiculous 37 medals for Canada — remember that nobody won more than 29 in Torino — so the fact that we're "only" at six compared to the 10 predicted by Day 5 is not that disturbing.

In short, there are still lots of ways that Team Canada can get to 30 medals. I don't know for sure if that will be enough. As I said above, the US team is lapping the field at the moment. In Torino they "only" won 25 medals and they are going to have more than 15 medals after Day 6 this time. But I can say that Canadian experts fully expected to get their biggest medal surge in the last few days of the games.

Sport Event Athlete Category Result Impact
Initial Prediction         27.05
Day 1         -0.75
Day 2         +0.45
Day 3         -0.50
Day 4         +0.00
Alpine Ski Downhill - W Brydon, Emily Possibility 16th -0.35
Alpine Ski Downhill - W Janyk, Britt Outside 6th -0.10
Speed Skating - ST 500 m - W Roberge, Kalyna Strong 6th -0.65
Speed Skating - ST 500 m - W Gregg, Jessica Outside 4th -0.10
Speed Skating - ST 500 m - W St. Gelais, Marianne Outside SILVER +0.90
Speed Skating - LT 1,000 m - M Morrison, Dennis Outside 13th -0.10
Cross Country Ski Sprint - W Renner, Sara Outside 34th -0.10
Cross Country Ski Sprint - W Crawford, Chandra Outside 26th -0.10
Snowboard Halfpipe - M Lamoureux, Justin Outside 7th -0.10
Day 5         -0.70
Current         25.55

February 16, 2010

2010 Olympics Day 4

Svein Tuft of the Day for Day 4: JP Le Guellec

It was a bit tricky to find an unnoticed noteworthy performance today, but in the end I had to give it to Jean Philippe Le Guellec for the second time.

I didn't see any of it, but it sounds like Le Guellec had a very weird day. He officially finished eleventh — a very respectable finish on its face. But in fact, Le Guellec was fifth to cross the finish line. As I discussed on Sunday, Le Guellec was supposed to be the sixth athlete to leave the start line in today's 12.5 km pursuit. Instead, due to a starter's mistake, he left fifth, about 30 seconds before he was supposed to. He crossed the finish line in fifth place, but after a post-race adjustment to correct for the error, he ended up eleventh.

"The guys just let me out too soon. Why, I don’t know," said LeGuellec, 24, of Quebec City. "I was just like, well, if worse comes to worse I’ll be disqualified or there’ll be a time adjustment. Whatever, do your race, have fun and that’s what happened."

After 2.5 kilometres and five-out-of-five shooting, LeGuellec appeared to be in second place. But his coaches were scrambling to make sure he knew he really wasn’t, that 30 seconds had to be added to his score. At the end of the race, with a decent 18-for-20 shooting, LeGuellec seemed solid for fifth, with no one in his sights behind him. He slowed down at the end to acknowledge the boisterous crowd.

But when he crossed the finish line and organizers added the 30 seconds to his time, he dropped to 11th.

"I am upset," said LeGuellec. "I came in fifth and I’m 11th. There’s nothing much we can do, it’s done."

LeGuellec forgave the official who let him go early. "You can’t blame the guy. With the hype of the Olympics and everything, there’s things that can happen," said LeGuellec.

Jean Philippe gets extra credit for handling the offical's error with such equanimity.

Medal Prediction

Nothing much happened to my medal prediction today. Maëlle Ricker's gold medal gave my prediction a boost of 0.35, but Dominique Maltais' fall took it away. My prediction still stands at 26.25 medals (the update table is at the end of this post).

Since there's nothing that interesting to report about my prediction today, I thought I might look at progress so far in a different (and more optimistic) way. As I noted in my original prediction, there were three major media organizations that also made medal predictions on an event-by-event basis. How are they doing so far at predicting Canada's results?

The Associated Press predicted 30 medals for Canada. They were wrong about Charles Hamelin in the short track 1500 m, wrong about Manuel Osborne-Paradis in the alpine downhill, and wrong about Dominique Maltais in women's snowboardcross. They also missed Kristina Groves in the long track 3000 m, and Mike Robertson in the men's snowboardcross. So the AP predicted that Canada would have six medals by now, an overestimate of one.

Sports Illustrated also predicted 30 medals, and was also wrong about Osborne-Paradis. They also missed Mike Robertson, and Alexandre Bilodeau. So SI predicted that Canada would have four medals so far, an underestimate of one.

The Canadian Press predicted the astonishingly high total of 37 medals for Canada. They were wrong about Hamelin, Osborne-Paradis, and Maltais. They missed Mike Robertson, too. The CP predicted that Canada would have seven medals so far, an overestimate of two.

Sport Event Athlete Category Result Impact
Initial Prediction         27.05
Day 1         -0.75
Day 2         +0.45
Day 3         -0.50
Snowboard Snowboardcross - W Ricker, Maëlle Strong GOLD +0.35
Snowboard Snowboardcross - W Maltais, Dominique Possibility 20th -0.35
Day 4         +0.00
Current         26.25

February 14, 2010

2010 Olympics Day 2

Svein Tuft for Day 2: Jean Philippe Le Guellec

Jean Philippe Le Guellec, ranked 33rd in the World Cup sprint standings in biathlon, finished sixth in today's 10 km sprint event. The previous best-ever for Canada at the Olympics was an eighth-place finish by Steve Cyr in 1992.

It seems that Le Guellec was considerably smarter about his chances in this event than most of the pundits. Here's his take on the Whistler course, taken from an interview in the Province that ran this morning:

Just the trail here and the whole way the system is set up is that anything can happen. When you look at results, even on the World Cup, you can see the top 60 athletes and if three-quarters had shot one more target, they'd be on the podium or in the top 10. It's that drastic.

Le Guellec missed one of ten targets today. The order of finish in the sprint now determines the start order and interval in the 12.5 km pursuit event, which runs on Tuesday. In the pursuit event, racers are assigned staggered start times — in this case, based on the results of today's sprint — and the first competitor past the finish line is the winner. If you've never seen this, its very dramatic. Le Guellec will start in sixth position, 50 seconds behind the leader. Many of the event's biggest names will be starting behind him.

Honourable Mention: Sam Edney

When I first heard about Jean Philippe Le Guellec's result today, I thought the Svein Tuft for today would be a no-brainer. But luger Sam Edney had a great day, too.

Edney was 11th after run 1, 10th after run 2, and 8th after run 3. On run 4, he had the third-best time, behind only the powerhouse Germans, to vault into seventh place. That's a best-ever Olympic result for Canada in men's luge.

Medal Prediction

Of course Canada had a good day today overall. Kristina Groves picked up a bronze medal in the women's 3,000 m, and Clara Hughes finished 5th. And it was a super night at Cypress, with Alexandre Bilodeau winning Canada's first gold medal of these Games — or of any "home" Olympic games. Joining Bilodeau in the top 5 were Vincent Marquis and Pierre-Alexandre Rousseau, who were sitting 1-2 with four skiers to go.

The medal prediction has shifted upward since yesterday, and is still down slightly overall. The data are in the table and figure below.

Sport Event Athlete Category Result Impact
Initial Prediction         27.05
Day 1         -0.75
Freestyle Ski Moguls - M Bilodeau, Alexandre Strong GOLD +0.35
Freestyle Ski Moguls - M Marquis, Vincent Possibility 4th -0.35
Freestyle Ski Moguls - M Rousseau, Pierre-Alexandre Outside 5th -0.10
Speed Skating - LT 3,000m - W Groves, Kristina Possibility BRONZE +0.65
Speed Skating - LT 3,000m - W Hughes, Clara Outside 5th -0.10
Day 2         +0.45
Current         26.75

February 13, 2010

2010 Olympics Day 1

I don't have a lot of editorial comment to make about Day 1 of the 2010 Winter Olympics. I watched a lot of it on television, I followed team media announcements on Twitter, and I enjoyed myself. There was a great variety of stuff going on, covering many of my favourite Winter Olympic things.

Canada didn't have a great day, but it wasn't terrible either. Jennifer Heil won a medal in moguls, which will unfortunately fall into the disappointed-not-to-get-the-gold category. Our short track speed skating team didn't win a medal in the men's 1500 m, along the way making a pretty good case for the usefulness of a statistical approach to prediction. Charles Hamelin, considered a very strong contender, ended up in the same semifinal with the eventual gold and silver medalists and failed to advance to the final. Olivier Jean, who I gave an outside chance at a medal, was "advanced" to the final after a crash, and ended up fourth.

Those events have had a detrimental effect on my medal prediction, but not an unrecoverable one. Overall the mean of my predicted medal total has decreased from 27.05 to 26.30, with a small decrease in the standard deviation. The prediction is also impacted by the delay of the men's downhill until Monday.

Sport Event Athlete Category Result Impact
Freestyle Ski Moguls - W Heil, Jennifer Lock SILVER +0.10
Freestyle Ski Moguls - W Richards, Kristi Outside 20th -0.10
Speed Skating - ST 1,500m - M Hamelin, Charles Strong 8th -0.65
Speed Skating - ST 1,500m - M Jean, Olivier Outside 4th -0.10
TOTAL         -0.75

You can see from the figure below that I am predicting Canada will bring in a steady 1-2 medals per day until Day 9.

February 10, 2010

Amateur Medal Prediction for 2010

If I was a journalist writing for a newspaper or sports magazine, I'd have to open this story like this: Canada will win 27 medals at the upcoming 2010 Winter Olympics.

The nice thing about that kind of prediction is that it's easy to encapsulate the result in a single sentence. The bad thing is that the single number '27' doesn't really tell the whole story.

Back in 2006, you'll recall, I took a different approach at predicting Canada's medal haul. I predicted that Canada would win 21±3 medals at the 2006 Winter Olympic Games in Torino. Canada ended up winning 24 medals, overshooting my predicted total but well within my assessment of the uncertainty.

I repeated the exercise for Beijing in 2008, but didn't publish my prediction since I was using the Canadian Olympic Committee's internal assessment of our athletes. The results were actually quite similar. My pre-games prediction was for 15.5±2.5 medals, and the final total was 18.

The "±" part of my predictions comes from the statistical approach. To predict the total number of medals, I first assign each Canadian entry a probability of winning a medal in their event. Through some fairly straightforward math, that leads to a probability distribution for all the possible medal totals for Canada. The only parameters you can fiddle with are the medal probabilities you assign to each Canadian entry up front.

I've done the same thing for the upcoming 2010 Winter Olympics. I put Canadian entries into one of five probability categories: "Lock" (90% chance); "Strong Possibility" (65% chance); "Possibility" (35% chance); "Outside Possibility" (10% chance); and "No Possibility" (0% chance). I should comment at this point that for 2010 I do not have access to any COC evaluation of the team — I'm back in my armchair for these games. My assessment comes from a few days of looking over World Cup and World Championship standings and results from 2010. So in detail, the probabilities are only as good as my abilities as an analytical winter sports fan, and its probable that I've missed some things. Past evidence also suggests that I tend to have a pessimistic bias against Canada's chances, so my probability assessments may be too low.

My probability assessments are presented in the table at the end of this post. The table identifies the day of each event, the sport, the event, and the athlete, with my medal category and the corresponding probability for that event. (I have left the "No Possibility" entries out of the table.) I have also included the predictions that three major media outlets have released in the last couple of weeks. The Associated Press (AP) and Sports Illustrated (SI) have predicted all the medal winners in every event, and the Canadian Press (CP) has predicted all the Canadian medals. The Associated Press and Sports Illustrated both predict 30 Canadian medals, and the Canadian Press predicts 37.

It is clear that my (independent) assessment of the medal probability lines up qualitatively with the three media predictions, although there are some quantitative differences. The Canadian Press, for example, predicts that every single one of the "Strong Possibilty" Canadian medalists will actually win a medal.

With the probability values above, I can calculate a probability distribution for the total number of medals Canada will have won after each day of competition, including the final total after Day 16. The results are shown in the figure below.

The left-hand plot in this figure shows my prediction (today) of how Canada's medal total will evolve over the 16 days of the Games, assuming all events go as scheduled. The image shows the probability distribution, and the green lines show the mean and standard deviation of the probability distribution. The right-hand plot shows the full probability distribution for the medal count at the end of the games.

If I had to encapsulate this picture with a single number, I would have to say … 27. But I don't have to do that, so let's put it this way: the most likely total number of medals for Canada, assuming that my characterization of individual probabilities is not biased in one direction or the other, is 27. The probability that Canada will win between 24 and 31 medals is about 75%. The probability that Canada will win 30 or more medals is about 24%.

One of the reasons to be especially interested in Canada's medal total this year is that the COC has a well-publicized goal for its athletes to win more medals in 2010 than those of any other National Olympic Committee. On the face of it, my prediction makes that unlikely; it will probably take at least 30 medals to finish atop the medal table, and I am putting that at three-to-one odds. But then again, I've got a history of getting that wrong.

I'll plan to update my prediction after each day of competition, and post thoughts about the Games here as well.

Day Sport Event Athlete Athlete Probability AP SI CP
1 Freestyle Ski Moguls - W Heil, Jennifer Lock 0.90 GOLD GOLD GOLD
6 Speed Skating - LT 1,000m - W Nesbitt, Christine Lock 0.90 GOLD GOLD GOLD
9 Speed Skating - LT 1,500m - W Nesbitt, Christine Lock 0.90 GOLD GOLD GOLD
13 Ice Hockey Ice Hockey - W Team Lock 0.90 GOLD GOLD GOLD
15 Speed Skating - LT Pursuit - W Groves, Klassen, Nesbitt, Schussler Lock 0.90 GOLD GOLD GOLD
1 Alpine Ski Downhill - M Osborne-Paradis, Manuel Strong 0.65 GOLD SILVER SILVER
1 Speed Skating - ST 1,500m - M Hamelin, Charles Strong 0.65 SILVER   BRONZE
2 Freestyle Ski Moguls - M Bilodeau, Alexandre Strong 0.65 GOLD   BRONZE
4 Snowboard Snowboardcross - W Ricker, Maëlle Strong 0.65 BRONZE GOLD GOLD
5 Speed Skating - ST 500m - W Roberge, Kalyna Strong 0.65 SILVER SILVER SILVER
7 Skeleton Skeleton - W Hollingsworth, Mellisa Strong 0.65 GOLD SILVER GOLD
8 Speed Skating - ST 1,000m - M Hamelin, Charles Strong 0.65   BRONZE SILVER
9 Speed Skating - LT 1,500m - W Groves, Kristina Strong 0.65 SILVER SILVER SILVER
9 Freestyle Ski Ski Cross - M Del Bosco, Christopher Strong 0.65 SILVER SILVER GOLD
10 Figure Skating Dance - Mixed Moir, Virtue Strong 0.65 BRONZE SILVER SILVER
11 Freestyle Ski Ski Cross - W McIvor, Ashleigh Strong 0.65 GOLD SILVER SILVER
12 Bobsleigh Two-Man - W Humphries (Pilot) Strong 0.65   BRONZE SILVER
12 Speed Skating - ST 3,000m Relay - W Gregg, Maltais, Roberge, St.Gelais, Vicent Strong 0.65 BRONZE BRONZE BRONZE
14 Curling Team - W Bartel, Bernard, McRorie, Moore, O'Connor Strong 0.65 GOLD GOLD BRONZE
14 Speed Skating - ST 5,000m Relay - M Bastille, Hamelin, Hamelin, Jean, Tremblay Strong 0.65 SILVER SILVER GOLD
14 Speed Skating - ST 500m - M Hamelin, Charles Strong 0.65 GOLD GOLD GOLD
15 Curling Team - M Enright, Hebert, Kennedy, Martin, Morris Strong 0.65 GOLD SILVER GOLD
15 Snowboard Parallel Giant Slalom - M Anderson, Jasey-Jay Strong 0.65 GOLD GOLD SILVER
16 Ice Hockey Ice Hockey - M Team Strong 0.65 SILVER GOLD SILVER
2 Freestyle Ski Moguls - M Marquis, Vincent Possibility 0.35      
2 Speed Skating - LT 3,000m - W Groves, Kristina Possibility 0.35   BRONZE BRONZE
3 Figure Skating Pairs - Mixed Davison, Dubé Possibility 0.35      
4 Snowboard Snowboardcross - W Maltais, Dominique Possibility 0.35 GOLD   SILVER
5 Alpine Ski Downhill - W Brydon, Emily Possibility 0.35     BRONZE
6 Figure Skating Singles - M Chan, Patrick Possibility 0.35 BRONZE SILVER BRONZE
7 Alpine Ski Super G - M Guay, Erik Possibility 0.35      
7 Skeleton Skeleton - M Montgomery, Jon Possibility 0.35 GOLD BRONZE SILVER
8 Speed Skating - LT 1,500m - M Morrison, Dennis Possibility 0.35 SILVER SILVER BRONZE
8 Speed Skating - ST 1,000m - M Hamelin, Francois Possibility 0.35      
9 Bobsleigh Two-Man - M Rush (Pilot) Possibility 0.35     BRONZE
11 Freestyle Ski Ski Cross - W Serwa, Kelsey Possibility 0.35   BRONZE BRONZE
12 Bobsleigh Two-Man - W Upperton (Pilot) Possibility 0.35 BRONZE    
12 Speed Skating - LT 5,000m - W Hughes, Clara Possibility 0.35     BRONZE
13 Figure Skating Singles - W Rochette, Joannie Possibility 0.35   BRONZE BRONZE
14 Speed Skating - ST 500m - M Tremblay, François-Louis Possibility 0.35 BRONZE   BRONZE
15 Snowboard Parallel Giant Slalom - M Lambert, Michael Possibility 0.35      
15 Speed Skating - LT Pursuit - M Giroux, Makowsky, Morrison, Parrot (Turgeon) Possibility 0.35 BRONZE BRONZE BRONZE
1 Alpine Ski Downhill - M Dixon, Robbie Outside 0.10      
1 Alpine Ski Downhill - M Guay, Erik Outside 0.10      
1 Freestyle Ski Moguls - W Richards, Kristi Outside 0.10      
1 Speed Skating - ST 1,500m - M Jean, Olivier Outside 0.10      
2 Alpine Ski Super Combined - W Brydon, Emily Outside 0.10      
2 Freestyle Ski Moguls - M Rousseau, Pierre-Alexandre Outside 0.10      
2 Speed Skating - LT 3,000m - W Hughes, Clara Outside 0.10      
3 Cross-Country Ski 15km Free - M Kershaw, Devon Outside 0.10      
3 Snowboard Snowboardcross - M Fagan, Rob Outside 0.10      
3 Speed Skating - LT 500m - M Gregg, Jamie Outside 0.10      
5 Alpine Ski Downhill - W Janyk, Britt Outside 0.10      
5 Cross-Country Ski Sprint (Classic) - W Crawford, Chandra Outside 0.10      
5 Cross-Country Ski Sprint (Classic) - W Renner, Sara Outside 0.10      
5 Snowboard Half Pipe - M Lamoureux, Justin Outside 0.10      
5 Speed Skating - LT 1,000m - M Morrison, Dennis Outside 0.10   BRONZE BRONZE
5 Speed Skating - ST 500m - W Gregg, Jessica Outside 0.10      
5 Speed Skating - ST 500m - W St.Gelais, Marianne Outside 0.10      
6 Snowboard Half Pipe - W Nicoll, Mercedes Outside 0.10      
6 Speed Skating - LT 1,000m - W Groves, Kristina Outside 0.10      
7 Alpine Ski Super G - M Dixon, Robbie Outside 0.10      
7 Alpine Ski Super G - M Osborne-Paradis, Manuel Outside 0.10      
7 Skeleton Skeleton - M Pain, Jeff Outside 0.10      
8 Alpine Ski Super G - W Janyk, Britt Outside 0.10      
8 Speed Skating - LT 1,500m - M Makowsky, Lucas Outside 0.10      
8 Speed Skating - ST 1,500m - W Maltais, Valérie Outside 0.10      
9 Freestyle Ski Ski Cross - M Duncan, David Outside 0.10      
9 Freestyle Ski Ski Cross - M Hayer, Stanley Outside 0.10      
9 Speed Skating - LT 1,500m - W Schussler, Brittany Outside 0.10      
10 Cross-Country Ski Team Sprint - W Gaiazova, Renner, Webster Outside 0.10      
11 Freestyle Ski Ski Cross - W Murray, Julia Outside 0.10      
12 Cross-Country Ski 4 x 10km Relay - M Goldsack, Jewett, McMurtry (??) Outside 0.10      
12 Speed Skating - LT 5,000m - W Groves, Kristina Outside 0.10 SILVER    
13 Freestyle Ski Aerials - M Omischl, Steve Outside 0.10      
13 Freestyle Ski Aerials - M Shouldice, Warren Outside 0.10      
14 Snowboard Parallel Giant Slalom - W Loo, Alexa Outside 0.10      
14 Speed Skating - ST 1,000m - W Roberge, Kalyna Outside 0.10      
15 Alpine Ski Slalom - M Cousineau, Julien Outside 0.10      
15 Alpine Ski Slalom - M Janyk, Michael Outside 0.10      
15 Bobsleigh Four-Man - M Rush (Pilot) Outside 0.10      

November 29, 2008

Olympic Team Google Map, 2008 Edition

Just before the Olympic Games started in Beijing, I put together an interactive Google map of the 2008 Canadian Olympic team. It was too late to get it incorporated into the COC's own excellent 2008 Beijing web site, but here it is for your entertainment. The map is interactive in the usual Google Maps way, and clicking on the icons will bring up a capsule on the athlete in question. Within the capsule, the link will take you to the full athlete profile on the COC site.

This was a repeat of my 2006 effort, although it required significant modification to make it work with the new Google Maps API, and I also went to the effort of making it look like the COC pages.

November 15, 2008

How Fast Could Usain Bolt Have Run?

File this one in the category of Left-Brain specials from somebody else’s brain (thanks to SportsFilter for picking up on this, many weeks ago).

In September, four physicists from the University of Oslo submitted a scientific paper cleverly titled Velocity Dispersions in a Cluster of Stars: How Fast Could Usain Bolt Have Run? As the abstract describes, the paper investigates the question of what Bolt’s 100 metre time might have been in the Olympic final, had he not spent the last 20 metres of the race celebrating his victory:

We revisit this question by measuring Bolt’s position as a function of time using footage of the run, and then extrapolate into the last two seconds based on two different assumptions. First, we conservatively assume that Bolt could have maintained Richard Thompson’s, the runner-up, acceleration during the end of the race. Second, based on the race development prior to the celebration, we assume that he could also have kept an acceleration of 0.5 m/s2 higher than Thompson. In these two cases, we find that the new world record would have been 9.61 ± 0.04 and 9.55 ± 0.04 seconds, respectively, where the uncertainties denote 95% statistical errors.

I will have more to say about Bolt's astounding performance at a later date, assuming that I find time to keep this up.

October 29, 2006

Why False Positives Mean So Much

Why do some people believe that Floyd Landis is innocent, while others believe that he is guilty? Why do the same people reach the opposite conclusions about Barry Bonds, or Marion Jones?

Let's talk about Bayesian Inference. Bayesian Inference is a mathematical approach that can be used to estimate how evidence influences belief in a hypothesis.

The basis of Bayesian Inference is an equation known as Bayes' Rule, which is widely applied in the study of probability. Bayesian Inference is the somewhat controversial application of Bayes' Rule to discussions of belief. In Bayesian Inference a person's degree of belief in a hypothesis is treated mathematically as if it was an estimate of the probability of the hypothesis.

Rather than go through the general background, I'll leave you to the Wikipedia article and get right to our specific example. In this case, the hypothesis in question (which I'll call H) is that an Athlete is guilty of doping. The hypothesis is either true or false, but the truth is usually not known with one hundred percent confidence. The evidence in the case (E) will be the results of anti-doping tests. The quantity that I want to calculate is a person's degree of belief in H in the face of the collected evidence E.

To do this we need to know three things. First, we need to quantify the person's prior belief in the athlete doping hypothesis. That is, what probability would you assign to the hypothesis before the presentation of the anti-doping test results? I'll denote the prior belief by P(H).

The other two quantities we need to know concern the conditional probabilities of the evidence. We need to know the probability that the evidence would arise if the athlete was guilty, which we might call the probability of detection (PD). We also need to know the probability that the evidence would arise if the athlete was innocent, which we might call the probability of false positive (PFP).

The posterior belief in H — in other words the probability of H being true after the evidence has been taken into account — is written as P(H | E), and can be calculated as:

   P(H | E) = PD × P(H) / [PD × P(H) + PFP × (1 – P(H))]

The following calculator allows you to enter these three inputs as percentages, and calculates your revised belief in the doping athlete hypothesis.

The Doping Athlete Bayesian Inference Calculator
P(H): Prior belief, before you saw evidence E, that Athlete was doping: %
PD: Probability that evidence E would be found against a doping athlete: %
PFP: Probability that evidence E would be found against an athlete who is not doping: %
Revised Belief that Athlete is Doping: %

Let's look at a hypothetical example. Let's say there's a positive test in a sport where you have reason to believe that 10% of athletes are doping. And let's say that the test that the athlete has failed is known to be 60% effective in detecting cheaters. And finally, let's say that the probability of a false positive is 0.1%. If we put these three numbers into the calculator we find that our revised belief in the athlete's guilt has risen to 98.5%. That's probably more than enough to meet the burden of proof in an anti-doping case.

In fact, as long as the probability of false positives is low, the other parameters really don't make much difference to the outcome. For example, consider the case where PFP = 0.01%, equivalent to one false positive test result out of every 10,000 tests. Then even if the prior belief of guilt is only 1%, and the probability of detection is only 20%, the posterior belief of guilt is greater than 95%. If we take this to the extreme case where there is zero probability of a false positive, then the posterior belief of guilt will always be 100%!

You can try some examples for yourself, and you will see that the weight of the evidence in the case depends very strongly on the probability of false positives. This number must be kept small if we want to treat positive tests as conclusive.

Now, what happens if you really would rather not modify your belief based on the new evidence? Is there any way, in the face of a positive anti-doping test, that you can go on believing in your favourite athlete's innocence?

Bayesian inference does allow for this possibility, if the evidence is poor enough; and in fact the mathematical treatment defines for us what "poor enough" means. If the test's probability of false positive rises, or the probability of detection sinks, to the point where both are equal, then the posterior belief in the athlete doping hypothesis will be exactly equal to the prior belief.

In most doping cases, the public receives very little hard information about the reliability of the evidence; neither PFP nor PD are well-known. In the Landis case, for example, anti-doping authorities were quick to state that the chances of a false positive on the IRMS test are very small. Many of Floyd's fans still doubt that claim.

If anybody out there tries this calculator for the Landis case or any other, let me know in the comments. I'd like to hear your assessments of your prior belief and the conditional probabilities, and to see if the Bayesian inference of posterior belief at all matches with your actual belief.

October 17, 2006

Floyd Landis: the Evidence

(See the preamble to this post.)

The case assembled by Floyd Landis' support team aims to prove that the athlete's alleged positive test result should be invalidated. In particular, Dr. Baker's PowerPoint presentation and Howard Jacobs' argument for dismissal present a number of points of evidence supporting that claim. I have combed through the laboratory documents to see whether that evidence might lead to Landis' exoneration in this case.

I should say up front that I used the aforementioned two documents (and especially Dr. Baker's slide set) as my guide; I did not go through page by page assembling evidence from scratch. As a result, this is not necessarily a complete argument, as Landis' team claims that there are "dozens of problems" with the tests.

I should also say that I am a physicist, not a biochemist, so I do not have a very deep understanding of the technical data presented in the laboratory documentation. I understand the basic idea of mass spectrometry, but I can't really interpret the raw data. I am not really sure if this is a great hindrance, though; most of the people who review this case (either at the AAA or ultimately at the CAS) will not be medical or scientific experts, either.

In what follows the references of the form "USADA 0000" are to individual pages in the laboratory documentation package. These pages have been provided on archive.com by the blogger at Trust But Verify.

As I noted before, Landis' defence here is based on Article 3.2.1 of the WADA Code. I will repeat that article here to remind you who bears the burden of proof in this matter:

WADA-accredited laboratories are presumed to have conducted sample analysis and custodial procedures in accordance with the International Standard for laboratory analysis. The athlete may rebut this presumption by establishing that a departure from the International Standard occurred.

If the athlete rebuts the preceding assumption by showing that a departure from the International Standard occurred, then the anti-doping organization shall have the burden to establish that such departure did not cause the adverse analytical finding.

The arguments in Dr. Baker's PowerPoint presentation and Howard Jacobs' letter are divided into four rough categories. I will deal with each of these in turn.

1. Bad laboratory documentation: Dr. Baker's presentation shows a number of specific examples of sloppy record keeping by the lab, and I have found at least one more which I will discuss later. The laboratory identification number was incorrect on Landis' T/E results page. The athlete identification number was incorrect — that is, not Landis' — on the same document. The athlete identification number was also incorrect on the specimen transport record, which records the chain of custody of the sample between the collection point and the lab, and on the lab's summary record of the test results.

Another page shows that the sample identification number has been overwritten in a manner inconsistent with the Laboratory Internal Chain of Custody (reference above) — although technically the modification was not made on the Chain of Custody documentation, so the cited reference is not really applicable.

Nevertheless, these mistakes are very embarassing for the lab, and somebody's wrist should definitely be slapped. However, it's probably not going to get Landis off the hook. The lab and sample identification numbers appear dozens (if not hundreds) of times throughout the document; a few errors here and there do not create a significant doubt that the tested samples came from Landis.

To put this in terms of Article 3.2.1, Landis has shown that a departure from the international standard has occurred, but the UCI will not have to work very hard to demonstrate that these errors did not cause his positive tests. If an arbitration panel dismisses Landis' positive over this issue it will be the most technical of technicalities. It's not impossible, but I consider it unlikely.

2. Unexplained variability in the T/E ratio:

In all there were nine vials of Landis' urine subjected to GC/MS (Gas Chromatograph / Mass Spectrometer) analysis at LNDD.

Table 1 — Floyd Landis urine samples subjected to measurement of T, E, and T/E.
Vial Testosterone Concentration
(ng/mL)
Epitestosterone Concentration
(ng/mL)
T/E
(Raw)
T/E
(using Concentration)
References Comment
A Sample Screening Procedure (GC/MS), Aliquot ES04, 2006-Jul-21
11 60.60 13.70 4.9 4.9 USADA 0054, 0055, 0056 T/E exceeds screening threshold (> 4)
A Sample First Confirmation Analysis (GC/MS), Aliquot EC24D, 2006-Jul-22
10 172.23 17.59 10.7 9.8 USADA 0212, 0213 Sample with hydrolysis
11 1.06 0.10 11.2 10.2 USADA 0214, 0215 Sample without hydrolysis
A Sample Second Confirmation Analysis (GC/MS), Aliquot EC24D, 2006-Jul-24
4 61.37 5.20 11.4 11.8 USADA 0092, 0093 Sample with hydrolysis
A Sample Re-Screening Procedure (GC/MS), Aliquot Unknown, 2006-Jul-25
2 49.70 11.10 5.1 5.1 USADA 0057, 0058, 0059 Test not described in aliquot chain of custody table
B Sample Confirmation Analysis (GC/MS), Aliquot EC24D, 2006-Aug-3
4 63.15 5.94 10.9 10.6 USADA 0278, 0277 Sample with hydrolysis
5 61.64 5.75 11.0 10.7 USADA 0279, 0280 Sample with hydrolysis
6 60.18 5.55 11.1 10.8 USADA 0281, 0282 Sample with hydrolysis
7 1.22 0.44 3.6 2.8 USADA 0283, 0284 Sample without hydrolysis

In the table above the "Raw" T/E ratio is calculated using the response amplitudes from the instrument, and the "Concentration" values are calculated after first converting the instrument response to a concentration using a calibration curve. The values don't differ significantly in any case.

The first sample noted was subjected to the Screening Procedure for Natural Hormones. Screening procedures for threshold substances like testosterone are meant to flag a sample for further study; in this case, when the T/E ratio exceeds 4. A sample could also be flagged as suspicious if it showed a high concentration of either testosterone or epitestosterone (> 200 ng/mL). There are no numerical accuracy requirements for Screening procedures.

The sample I have called the "A Sample Re-Screening Procedure" is odd for a couple of reasons. First, it was performed after all of the confirmation tests, including the IRMS confirmation. At that point in time, the screening procedure would appear to be more or less irrelevant, so I am not sure why this was done. The handwritten note on the summary results page (USADA 0057) appears to read "vial de conf reinjecte pr screening." Another odd fact is that this test does not appear on the Aliquot chain of custody table (USADA 0011, 0012, repeated USADA 0255, 0256). This is a very clear violation of the Laboratory Internal Chain of Custody (reference above) and therefore another error in the documentation. Add it to the pile in item 1 above.

The Confirmation Analyses are used to definitively assess the T/E ratio in a sample once it has been identified as suspicious by the screening procedure. The samples I noted as "with hydrolysis" are the ones we should focus on, for now. These samples are subjected to hydrolysis to cleave testosterone and epitestosterone from the various compounds that they form in the urine sample (that's this physicist's understanding). The GC/MS then detects the total concentration of T and E in the urine.

The Confirmation Analysis (with hydrolysis) was run on five different vials of Landis' urine sample; two from the A sample and three from the B sample. Of those, four were very self-consistent: the A sample aliquot tested on 24 July, and the B sample aliquots tested on 3 August all showed T concentrations between 60.18 and 63.15 ng/mL, E concentrations between 5.20 and 5.94 ng/mL, and T/E ratios between 10.6 and 11.8. The LNDD claims (e.g. USADA 0101) that the uncertainty for these results is 30% for E, 20% for T, and 30% for T/E. All four of these measurements fall well within those expectations.

The fifth sample, which was actually the first one tested, is way, way off, with T = 172.23 and E = 17.59. It is true that the ratio of T/E is consistent with the rest of the samples, but the variation in steroid concentration remains unexplained.

It is possible that the LNDD will respond with a valid explanation for this variability. At any rate, by itself I do not think it is sufficient to doom the case. It might suggest that the confirmation procedure was run incorrectly in one instance, but it does not directly imply that all of the other results are invalid.

This brings me, however, to Dr. Baker's third claim.

3. Contamination of the sample: Dr. Baker makes a case that Floyd's B Sample was not suitable for testing. The Reporting and Evaluation Guidance for Testosterone (reference above) has this to say about microbial contamination of urine samples (emphasis mine):

The urine Sample is not collected under sterile conditions, and where the circumstances are favourable, the microbes present in the Sample can cause changes to the profile of the urinary steroids. Initially there is cleavage of the glucuronides and sulfates followed by modifications of the steroids’ structure by oxido-reductive reactions. To report an Adverse Analytical Finding of an elevated T/E value, testosterone or epitestosterone concentration or any other endogenous steroid parameters, the concentration of free testosterone and/or epitestosterone in the specimen is not to exceed 5% of the respective glucuroconjugates. Elevated amounts of 5ALPHA- and 5ß-androstan-3,17-dione in the free form also indicate microbial degradation.

The "free" testosterone in a urine sample is provided by running the Confirmation procedure on a sample without subjecting it to hydrolysis. Only T and E that exists "free" in the urine shows up in the GC/MS traces.

So let's go back to Table 1. There were two aliquots subjected to the confirmation procedure without hydrolysis. One of these was the first A sample confirmation on July 22, which showed acceptable levels of free T and free E (about 0.6% in both cases). The second was from the B sample on August 3 — but that sample showed 2% free testosterone and 7.7% free epitestosterone, calculated as a percentage of the mean of the results from the samples with hydrolysis.

Unless I am misreading the documentation or misinterpreting the science (both possibilities) it seems fairly clear to me that Floyd's B Sample should have been ruled too contaminated for testing. The paragraph I quoted above is quite clear on this: with a concentration of free epitestosterone this high, the B sample could not be used to report an adverse analytical finding.

And if the B Sample cannot confirm the A Sample results, the International Standard for Laboratories (reference above) states clearly that the test is negative:

5.2.4.3.2.3 The B Sample result must confirm the A Sample identification for the Adverse Analytical Finding to be valid. The mean value for the B Sample finding for Threshold Substances is required to exceed that threshold including consideration of uncertainty. (International Standard for Laboratories)

If the “B” Sample confirmation does not provide analytical findings that confirm the “A” Sample result, the Sample shall be considered negative and the Testing Authority notified of the new analytical finding.

It is also interesting to consider the hypothesis of contamination with respect to the sample-to-sample variability I discussed in point 3 above. No measurement of free T and E was made in the A sample second confirmation analysis, so there is no evidence of contamination in that sample. However, the concentration results are consistent with the B sample results, which do show evidence of contamination. It might be that the only uncontaminated sample was the first confirmation of the A sample. Note that the aliquot used in the second confirmation of the A sample was prepared for confirmation on the afternoon of July 23 and not tested until almost 24 hours later (ref. USADA 0256).

This speculative explanation does not cover the B sample contamination; however, any microbes present in the sample had an additional 10 days to act on the refrigerated B sample while it was stored at LNDD. Perhaps this was sufficient time to degrade the sample.

4. CIR results did not support a "positive" finding: I've already stated that I think that Landis is ultimately going to win his appeal due to the evidence that his B sample was contaminated when it was tested.

That outcome, however, might still leave the public with the feeling that Landis was a cheater who "got lucky" on the test. After all, his clearly uncontaminated A Sample showed a T/E ratio of approximately 11/1, and the IRMS results on that same sample showed evidence of exogenous testosterone. The IRMS results have been treated by most (including me) as the "smoking gun" in the case.

Landis has an answer for that, too: the IRMS test results do not show evidence of exogenous testosterone, by WADA's own standards.

The argument hinges on the fact that the IRMS test was performed on four different hormone ratios, and more than half of them came up negative when the laboratory standard is applied.

Table 2 — Floyd Landis urine samples subjected to IRMS testing
Metabolite-Reference Blank Urine Δ‰ Sample Δ‰
A Sample IRMS, Aliquot EC31, 2006-July-23 (reference USADA 0186)
Etio - 11 Kétoétio -0.87 -2.58
Andro - 11 Kétoétio -0.48 -3.99
5ß Adiol - 5ß Pdiol -0.55 -2.15
5α Adiol - 5ß Pdiol -1.59 -6.14
B Sample IRMS, Aliquot EC31, 2006-Aug-3 (reference USADA 0352)
Etio - 11 Kétoétio -1.08 -2.02
Andro - 11 Kétoétio -0.08 -3.51
5ß Adiol - 5ß Pdiol -0.67 -2.65
5α Adiol - 5ß Pdiol -1.60 -6.39

The threshold for a positive IRMS finding for exogenous testosterone is Δ‰ = -3.0. The IRMS uncertainty, according to LNDD, is 0.8. That means, according to the International Standard for Laboratories (reference above), that a positive must show Δ‰ of -3.8 or less.

By this standard, the A sample shows two metabolites as positive, and the B sample shows only one.

Landis' team asserts that this is not enough; they argue that all four ratios must exceed the -3.8 threshold to make a positive test. The Reporting and Evaluation Guidance (reference above) is ambiguous on this point; it neither specifies which metabolites should be measured, which reference steroids should be used, nor does it specify how many or which pairs must exceed the threshold.

Depending upon the nature of the endogenous steroid suspected to have been administered, the metabolites analysed could be testosterone, epitestosterone, androsterone, etiocholanolone, the androstanediols, DHEA, or other relevant metabolites while the urinary reference steroid usually analysed by the Laboratories is one of, pregnanediol, pregnanetriol, cholesterol, 11-hydroxyandrosterone or 11-ketoetiocholanolone. …

The results will be reported as consistent with the administration of a steroid when the 13C/12C value measured for the metabolite(s) differs significantly i.e. by 3 delta units or more from that of the urinary reference steroid chosen.

It seems pretty clear, under the Reporting and Evaluation Guidance, that the LNDD could legitimately have tested for a single ratio; and if they had chosen to use the 5α Adiol - 5ß Pdiol ratio as their single measure, it would have returned an unambiguously positive result.

But they didn't do that, and I can't really predict how things are going to go here. It will come down to a question of precedent, I think; whether there is an undocumented standard for the choice of metabolites and the application of the threshold, and whether the LNDD can prove it. This will become an important question if my prediction is incorrect and the B sample results are ruled to be valid.

August 29, 2006

In the Wind

This weekend CBC Sports broadcast the ICF World Flatwater Championships from Szeged, Hungary. The broadcast was not live — competition was held the weekend before — but that's about the only complaint I can come up with about the coverage. Commentators Scott Russell and Scott Logan did a very good job putting things in context. They made a few factual slip-ups, but you would have to be a real know-it-all to point those out. Best of all, we got to see every final Canada was in — two and a half hours in all.

The local organizing committee should also be commended for giving the championship some good exposure. There were two things that I wanted to mention in particular. The first was the use of some unusual camera angles during the races. There was one mobile overhead camera in particular that gave a few fabulous shots from right on top of the boats. It provided some very dynamic views of the athletes as they passed by during the races.

The second item that was very cool was the event web site. If you go to the Results page and click on "Download Full Results," you can download a CD Image file of all of the race results — heats, semis, and finals, including all of the finish photos. The .iso file can be burned onto a CD and you will then have a web-browser-ready database of all of the race results.

Speaking of race results, there were lots of interesting things that happened at the World Championships this year. I won't talk too much about the Canadian team results, except to say congratulations to the team and all you sports fans should be ashamed of yourselves for the attention you don't give our amateur athletes.

Kidding, I'm kidding. But since I have all those results on CD now, let me try to dig something interesting out of the data. Specifically, I wanted to take a look at the influence that the wind conditions had on some of the races, since that point was raised a few times by Russell and Logan during the broadcast.

Flatwater canoe-kayak races are seeded from the middle out, just like swimming races and athletic sprints. The course has nine lanes, and the (nominally) fastest crews are put in the middle lanes, with the (nominally) slower boats in the outside lanes. When everything works properly, you get a photo finish that looks like this:

Finish photo from 2006 world flatwater canoe-kayak championships, showing typical pattern of lanes

In Szeged, lane 1 is furthest from the judges' tower, at the top of the photo, and lane 9 is closest, at the bottom. If we look at the ensemble of all races at the World Championships, including preliminary rounds, we can see that the seeding is generally pretty good:

Position of finish versus lane: all races at 2006 world flatwater canoe-kayak world championships

This picture is arranged to mimic the photo finish above; position of finish goes from first place on the right, to ninth place at left, and the lanes go from 1 at the top to 9 at the bottom. The brightness of the image at each point represents the percentage of competitors racing in each lane who finished in each position. The green numbers on the right-hand edge show the percentage of competitors in each lane who finished first in their race. The green curve shows the average position of finish for each lane (1-9) for all 220 races.

You might wonder exactly how this seeding works. For the first round of races, known as heats, competitors are seeded based on prior season results (World Cups etc.). As you can imagine, this is not really an exact science, but the people who do the ranking are pretty effective. Seeding for the semifinals is based on performance in the heats, so the athletes self-seed themselves for the second round. That works even better:

Position of finish versus lane: heats and semis at 2006 world flatwater canoe-kayak world championships

In this picture, the top image includes the 103 heats, and the bottom image shows the 68 semifinals. You can see that the seeding in the heats and semifinals is a very good predictor of performance. In these preliminary rounds, more than 60% of winners come from lane 5, and only a few from lanes 1, 8, or 9.

In fact, over two and a half days and 171 races, there wasn't a single winner from lane 9. And then came the afternoon of Saturday, August 19.

The finals don't go quite as strongly according to seed as the preliminary rounds, even when everything goes right. That's because the separation between the fastest and slowest paddlers shrinks when you get to the final. It's also because the seeding has a random component to it. For example, in men's K-1 there are four semifinals, and the winners go into the four middle lanes; only one of them can have lane 5, and that selection is made in a pseudo-random manner so that athletes can't easily choose their lane for the final.

But on average, you would still expect my probability diagrams to be fairly symmetric around lane 5, even if they're not as strongly peaked as they are for the semis. The upper image in the plot below shows the statistics for the 33 finals held on Sunday, August 20 (all 500 m and 200 m finals). I wasn't there in person, but from the broadcast it looked like pretty ideal conditions on Sunday — flat calm or slight headwind. There is not a strong asymmetry in the expected position of finish, which supports the idea that the wind conditions were fair.

Position of finish versus lane: finals at 2006 world flatwater canoe-kayak world championships

On Saturday afternoon, though, it was pretty clear that some lanes were better than others. The lower plot shows the statistics for the 16 finals (A and B finals) held on Saturday afternoon. As you can see, the distribution of finishes was significantly altered. Competitors in the near lanes had enhanced performances relative to competitors in the far lanes. There were even three winners out of lane 9, one in a 'B' final and two in championship finals.

From the broadcast it appeared that there was a significant cross-headwind during the 1000 m finals. Again, it's difficult to assess the conditions from the pictures on TV, but the statistics seem to show a pretty clear advantage for the high-numbered lanes, where the Szeged course is more sheltered by the shore and the grandstand. My point here is not to make excuses for anybody, nor to discount the two world championships earned by paddlers in lane 9. Canoe-kayak is not like swimming, where most environmental factors can be rigorously controlled. That's just the nature of our sport; sometimes the conditions work in your favour, sometimes they work against you, and hopefully most of the time it's neutral.

July 17, 2006

WADA Statistics 2005, Part 3

In Part 2 of this post, I discussed the statistics of anti-doping tests conducted by WADA from 2003 to 2005. Specifically, I showed the positive test rates for 2003, 2004, and 2005 for all summer and winter Olympic sports, plus a few selected other sports. As I noted in that post, there was a significant increase in the positive test rate in 2005 compared to the previous two-year period. In the first part of this discussion, which was several months ago, I discussed WADA's publicly-stated hypothesis that the year-over-year increase was due to an increase in the effectiveness of their testing, and not due to an increase in the rate of cheating.

In this post, I want to explore the statistics of the increase in more detail. My goal here is to provide a crude test of the hypothesis that the effectiveness of anti-doping programs increased between 2004 and 2005.

Measured positive test rate changes

The figure above shows some of the key data summarizing that increase. The upper plot shows the positive test rates for the period 2003-2004, and for 2005, for all of the sports. The sports are arrayed on the vertical axis, although the names are not shown in this plot — see Figure 1 from Part 2 for more detail. The positive test rate, defined here as the number of A-sample adverse analytical findings divided by the total number of tests, is displayed on the horizontal axis. The error bars on the positive test rate are calculated on the basis of the total number of tests performed. Large error bars mean there were few tests performed, and small error bars mean that there were many tests (see this post for details).

The lower plot shows the change in the positive test rate (2005 rate minus the 2003-04 rate) plotted as a function of the 2003-04 positive test rate. As I noted last time, the curve is consistent with the hypothesis that the increase is the same for all sports, having a mean value of 0.53%.

Now it is interesting to note that almost all of this 0.53% increase can be attributed to a single banned substance: testosterone. The number of adverse analytical findings for testosterone in 2004, in all sports, was 392. The number of adverse analytical findings for testosterone in 2005, in all sports, was 1132. An increase of 740 positive tests, out of 146,539 total tests, would result in an increase in the positive test rate of 0.50%. That's an interesting coincidence.

It also turns out that the test for testosterone was changed between 2004 and 2005. Specifically, the threshold for the testosterone test was lowered:

The 2005 figures include several elevated T/E ratios [a urine parameter used in anti-drug tests] over 4, which were not reported in previous years when the threshold was 6, partially accounting for the increased number of AAFs in 2005. (source, PDF)

Now, lowering your failure threshhold is guaranteed to accomplish two things: it will increase your probability of detecting cheaters, and it will increase your probability of "catching" innocent athletes by mistake. The degree of each change will depend on how many cheaters are below the current threshhold, and how many non-cheaters are above the new one. To be honest, I don't know the details of those distributions, but the observations we do have (provided by WADA) might be of some help in assessing the change. I don't have the testing breakdown by sport by substance, so I'll have to look at the global data for all banned substances.

To quantify this a little bit further, I'll assume that within each sport, a fraction DS of the population of tested athletes are using banned substances, where DS is different for each sport. I'll assume that when one of those cheating athletes is tested, there is a probability Pd (the probability of detection) that the test will give an adverse analytical finding, and that Pd is the same for all forty sports. (In other words, drug testing, overall, is equally effective in all sports. I don't think that's really true, but I'll do my best to gloss over that fact when I'm finished.) Finally, I'll assume that when any athlete (clean or dirty) in any sport is tested, there is a probability Pfp that the test will generate a false positive.

Under these assumptions, when N athletes are tested, the expected number of positive tests will be given by:

      A = N (Pfp + Pd DS) ,

and the positive test rate will be simply A/N. Since we're talking about positive tests for all banned substances combined, the probabilities and rates in the equation above have to be averaged over all banned substances as well.

I've run some simulations using this equation to try to estimate the effect of changing the probabilities Pd and Pfp. I set N equal to the number of tests actually performed, using the 2005 numbers as a basis, and then ran multiple Monte Carlo simulations to estimate the statistics of the distribution of the number A of observed positives. The underlying doping rates DS are unknown, but once the values of Pd and Pfp are specified, then the doping rates have to be set to match the observed positive test rates for 2003-04; from there I can simulate the effect of changing either Pd, or Pfp, or both, while holding the DS fixed.

If drug testing was more effective in 2005 than it was in 2004, I can model that as an increase in Pd for all sports. To make things simple, I'll assume for the moment that the probability of false positives is zero. I don't really have any idea what to use as the value for Pd in 2003-04, but for starters I'll assume that the probability of catching a doping athlete with a drug test was 25%.

If that probability increased to 36.5% in 2005, then the simulated positive test rates would be as shown in the figure below:

Simulated positive test rate changes

The upper plot shows the simulated positive test rates for the two proposed values of Pd, where the doping rates DS are set to match the 2003-04 observations, and then held constant for the 2005 simulation. The error bars on the positive test rates indicate the standard deviations of the Monte Carlo runs, and the red circles indicate the means.

The increase in Pd was chosen to match the mean observed increase of 0.53% in all sports, indicated by the dashed blue line on the lower plot. The same result is obtained from any similar proportional change in the probability of detection. In other words, an increase from 25% to 36.5% looks exactly the same as an increase from 65% to 95%, or from 10% to 14.5%.

As you can see, if the probability of detection is increased for all sports, then the greatest increase in the positive test rate should be in sports with the highest positive test rates, and the sports with very low positive test rates show the smallest increase. The expected increase is proportional to the initial positive test rate. That doesn't actually match the observed changes very well. In general the sports with the highest positive test rates showed little or no change in 2005 compared to 2003-04.

But what if, instead of increasing Pd, we simulated an increase in Pfp, the false positive rate? Then the picture looks quite different:

Simulated positive test rate changes

The figure above shows the effect of increasing the false positive rate in all sports from zero to 0.53%. In this case, the positive test rate is increased uniformly for all sports, regardless of the underlying doping rate. (I used a probability of detection of 25% for the simulation, but that parameter turns out to be irrelevant to the increase.)

Modelling the 2005 increase as an increase in the false positive rate looks like it matches the observations quite a bit better than an increase in the detection rate. Given the sample uncertainty, as I stated at the outset of this post, the observed increase is consistent with a uniform change in all sports.

One small problem with this theory is that the postulated false positive rate is pretty high. If Pfp = 0.0053 for all sports in 2005, then that should set a lower threshold on the observed positive test rates. Both luge (0%) and softball (0.37%) came in under the limit in 2005, but the number of tests involved was very small. With 178 tests in luge, we would have expected 0.94 positive tests, whereas none were observed. In softball, 542 tests should lead to 2.87 positives, whereas two were observed. So we're talking about less than one "missing" positive test in each case — I don't think we can rule out a false positive rate of 0.53% quite yet, and I think we have to at least consider the possibility that the false positive rate increased significantly in 2005.

Of course the assumptions I've used in my analysis are pretty simplistic; certainly all sports don't have the same kind of doping problem, and therefore their anti-doping programs are not all the same either. But my conclusions are based on a comparison of broad trends, and don't really depend critically on that assumption, I don't think. For a future analysis, it would be really nice if I could study the positive test rate changes for testosterone by itself, since I think that the increase in positive test rates was largely due to the changes in the testosterone test. Unfortunately, WADA doesn't provide that breakdown.

Some people might also reject my very first assumption, which was that athlete behaviour (specifically doping rates) didn't change between 2004 and 2005. What if they did change? Well, assuming that the tests were consistent from year to year, the observed changes in positive test rates suggest that the rate of doping increased more or less uniformly across the board. I can't really tell the difference between that event and a change in the false positive rate.

I have to live with the limitations of the analysis, which means that there's no "smoking gun" here. But everything I've looked at in the 2003-2005 statistics suggests that the increase in the positive test rate in 2005 came largely at the expense of athletes that weren't cheating in the first place.