Pro Virtual Cycling Has a Massive Verification Problem
Blog #22
I hate talking about performance verification. It’s just so deflating. When you watch an IRL race and Puck Pieterse somehow bests Demi Vollering in La Flèche, the second she crosses the line you know she’s won.1
Not so in cycling esports. And I actually think this is worst part of our sport. Every time I watch a race, the excitement I feel seeing an upset or a thrilling ending is always tempered by the voice in the back of my head saying “well, let’s see if it gets verified.”
This came to a head earlier this season when, in consecutive Zwift World Series races, Kate McCarthy and Gabriela Guerra had wins annulled due to trainer malfunctions out of their control. But despite these issues apparently being fixed, the epidemic of annulments and controversy over verification has continued. Earlier this month, Riccardo Panizza was relegated from second place to 17th in Stage 4 of the Zwift World Series for reasons Zwift hasn’t bothered to tell the public.2 And as I sit here writing this early in the morning on Monday, May 5 (11 days after the final ZWS stage took place on March 24) we still don’t have verified results for Stage 5. So as much as I’ve tried to avoid this conversation, I think it’s time we have a serious chat about how verification issues are undermining the sport.3
The Verification Process
Let’s start with how the two major platforms, Zwift and MyWhoosh, approach performance verification.4
Zwift
Zwift outsources performance verification to AthleteAnalytiX, an independent third party. To start with the obvious: in the abstract, this is the ideal approach. The conflicts of interest - perceived or real - that arise when verification is kept in house are undeniable, and relying on an independent, third-party commissaire is the best way to do this stuff.
But just because Zwift hands off the reins to “an independent commissaire who is in charge of rules, regulations, and governance” doesn’t mean Zwift has no responsibility for the results of that process.
First, Zwift can and should set standards for the timeliness of verification. I see no reason whatsoever that the process should routinely take more than a week, and given the consistency with which riders are penalized or annulled it’s a constant source of frustration for fans that we can’t just watch a race and at least know within a couple of days who won (setting aside the promise of in-race verification, which feels like a pie-in-the-sky dream at the moment but has been discussed).
Second - and this is where I get most frustrated with Zwift - is the lack of transparency. In Stage 4 of ZWS, Panizza lost his podium position due to a commissaire action. Why? No clue. You watch the race and see an exciting finish where Panizza barely holds off Mickael Plantureux and Johan Norén for second, and then more than a week later look at the official results and he’s down to 17th. No explanation as to why.
This isn’t an isolated incident though. During Zwift Games, Stefan Kirchmair, Roberta Bussone, and Christine Schneider all received GC point deductions for procedural infractions (e.g. incorrectly inputting their names). But there was zero public mention of this by Zwift, and the only way I was able to figure it out was by noticing the discrepancy between the stage and GC standings and reaching out to Zwift to ask why they weren’t the same.
I really hate harping on this kind of stuff, but it’s these small things that really make it difficult for the sport to grow. Fans can’t just watch a race and know that what they see is the actual result. That can’t really be solved for in the immediate term, and that’s fair enough. It’s something we need to learn to live with in the sport until in-race verification becomes a thing. But the long wait times for official results and lack of transparency over penalties is absolutely something that can be addressed now.
MyWhoosh
In contrast, MyWhoosh doesn’t have either of the problems identified above. SRC results are generally verified within a couple of days, and annulments are clearly provided in the results.
The flip side, of course, is that the MyWhoosh verification process is done in-house using their “Power Passport” method. There has been a fair bit of publicity and controversy over how MyWhoosh verifies races, including last year when they annulled results from Lionel Vujasin, Ollie Jones, and Michal Kaminski and then flew them out to the UAE to verify their performances in person. There was also the back-and-forth bickering between Zwift and MyWhoosh after Zwift put ZWS on pause due to trainer malfunctions identified by AthleteAnalytiX and, in a not so subtle way, implied that the same trainer issues were probably happening on MyWhoosh as well. I don’t know if Zwift was the right party to be hinting at those issues, but it was an obvious question: if Zwift was willing to cancel one of its flagship events over hardware issues, and the same hardware was being used on MyWhoosh, wouldn’t MyWhoosh have the same issues?
Regardless, from a fan perspective, I strongly prefer the MyWhoosh approach. Or if not the approach, at least the speed and transparency. Performance verification is something that the sport absolutely needs to get right. But it’s not something I want to constantly be thinking about as a fan. Because it’s a bummer! It sucks when a McCarthy or a Guerra loses a win due to something completely out of their control. So I like that MyWhoosh doesn’t make us wait weeks and that it actually tells us what the heck happened when a rider gets penalized.
The Path Forward
In the short term, Zwift desperately needs to speed up their verification process and be more transparent about it. That’s low hanging fruit, and should be easy to accomplish.
But in the long term, there really is only one obvious solution: robust, third-party verification that is quick, accurate, and (perhaps most importantly) consistent across all platforms putting on professional races. Unfortunately, that’s really tough to do.
It could happen through the UCI - and indeed, the UCI has been looking at smart trainer homologation for well over a year now through Purdue University. But in esports, the UCI is famously slow to act. And I’m not sure how it happens outside of the UCI. Back in the post-ZADA days when Zwift was working with indieVelo as the independent commissaire, I had some hope that indieVelo (or something like) it would emerge organically as the gold standard that all platforms would use. But that didn’t happen, indieVelo got acquired by TrainingPeaks, and MyWhoosh seems to have further developed and become attached to its own system in the meantime. The competition between platforms and smart trainers is just fundamentally at odds with unifying around a single body and approach to conducting verification.
But something needs to change. Frankly, this isn’t an issue I want to be thinking about. It’s sort of like the baseball pitch clock - fans want games to go faster, but they don’t want the viewing experience to be dominated by thinking about the clock instead of the game. In the same way, professional cycling esports has a problem right now in that verification is always in the back - if not the front - of fans’ minds when watching races. That can’t be the case if we want the sport to grow.
Yeah, yeah, there are sometimes sprint relegations. But it’s not the same.
I reached out - on this question and a couple others discussed in this blog - but got no response.
I hope readers understand these critiques are coming from a place of love. I started a freaking ranking website and blog to track and write about virtual cycling as a professional sport. I love this stuff. But that doesn’t mean we, as a community, can’t have honest conversations about where it is falling short.
What follows is a pretty high-level summary of how this works. Chris Schwenker at The Zommunique has delved quite deeply into these issues, so if you are interested in more detail please use the hyperlinks in this section.