Witbe QoE Snapshot - Who Is The BEst video operator in North America and Europe?
Introduction & setup
- North America:
- Altice Optimum One in New York
- AT&T Uverse in San Jose
- Bell Fibe in Montreal
- Comcast Xfinity in Denver
- Cox Contour DTA in Phoenix
- DirectTV in New York
- Dish Hopper in New York
- Rogers Ignite TV in Toronto
- TWC Spectrum in New York
- Verizon Fios in New York
- Videotron illico in Montreal
- Europe:
- Bouygues BBox in Paris
- Brandt DVB-T in Paris
- BT UHD in London
- Canal + Cube in Paris
- Free Revolution in Paris
- Orange Livebox in Paris
- SFR Box in Paris
- Sky Q Mini in London
- Telefonica Movistar in Madrid
- Virgin V6 in London
- Vodafone Portugal in Lisbon
Witbe Remote Eye Controller
- Because the robot waits for both audio and video motion, measurements will be more severe than most operator’s standards.
- STBs tuning to 1 or 2-digit channel codes will take longer than STBs tuning to 4-digit channel codes because there will be no wait time for an eventual next digit input.
We measured and verified the following KPIs:
- Channel Availability: The channel was considered available when audio started and video was moving
- Channel Change Time: Time between the last remote code digit input and the start of both audio and video motion on the channel.
- Numerous other perceptual KPIs (Time to First Frame, Time to Audio, Time to Video Motion, etc.)
- Numerous other network KPIs (Network Channel Change Time, RTP Join Duration, Unicast Burst Time, etc.)
Every channel change was performed using our new Channel Change engine which includes reengineered perceptual and network algorithms, a dynamic scheduling system, and a new visualization widget in Witbe Datalab – our visualization interface used for fault finding, root cause analysis and advanced analytics.
Global Overview
The image above ranks all the STBs tested, by Channel Change Time average (only the top 3 operators in each region are made public — please contact us if you want to know your position in this ranking):
The first thing we can see by looking at the average Channel Change Time is that the best-in-class are:- in North America:
- AT&T Uverse in San Jose
- Bell Fibe in Montreal
- Rogers Ignite TV in Toronto
- in Europe:
- Virgin V6 in London
- Vodafone in Lisbon
- SFR in Paris
We can see that the top 4 performing operators for this channel change test are North American, with a Channel Change Time under 2 seconds. It can partially be explained by the fact that European operators have more 1 and 2-digit channel codes in their test lineup.
It is also interesting to note that both AT&T, Bell and Vodafone use Mediakind’s “fast channel change” technology, helping them reach the top of their respective rankings.Lastly, we measured that 3 of the 5 operators that have the longest Channel Change Time rely on satellite technology, with which slower tuning times are usually measured.
Channel change 101
- Broadcast stream: one emitter (a satellite or a Hertzian antenna) sends the signal to every connected receiver
- Unicast stream: one emitter (a video server) sends the signal to a single receiver
- Multicast stream: one emitter (a video server) sends the signal to a specific set of receivers (users watching the same channel in the same area)
- How long does the STB need to buffer the video before being able to play it (this is a STB software and encoding optimization challenge)
- How long does the server take to respond to the IGMP join query and send the video packets in the associated multicast stream (this is a server and network optimization challenge)
Examples of best-practices
Improving Channel Change Times is one of an operator’s many challenges. One of the ways to reduce the duration of channel change is to deploy “fast channel change” technology inside a network (Mediakind technology for example). This requires heavy investments and network modifications, but it produces excellent results.
Two improvements led to the very fast Channel Change Time seen in the video above:
- Software was optimized to attain a Video Buffering Time of 383ms
- New servers were added to the network to first deliver the channel in a “unicast burst” stream, creating an instant connection between the server and the STB, allowing video packets to be sent instantly before seamlessly transitioning to a multicast stream.
There are other ways to improve Channel Change Time that rely on smart engineering techniques rather than heavy investments:
In this example, the network architecture was not modified. However, the software of the STB was highly optimized and was able to replace the black screen entirely by freezing the last frame of the previous channel and then the first frame of the upcoming channel. By doing so, the Time to Video Motion of 4s is the same as if there was a black screen, but the Time to First Frame of 3.1s was effectively improved by 22%. Compared to using a black screen, this method has a far more positive impact on the end-users’ perceived QoE.
Conclusion
In conclusion, we can see that when it comes to Channel Change Times operators in North America offer a better Quality of Experience than their European counterparts, most notably by attributing 4-digit channel codes to their top viewed channels.
That being said, who would have thought that an operation as simple as changing a channel would be so technically challenging and require so much investment! Channel Change Times is a critical KPI that video operators monitor closely because they know it matters to their end users.
Stay tuned for the next part of this Snapshot series!About Witbe QoE Snapshots
In the same way that a consumer report tests a product and publishes an analysis of its overall quality, the Witbe QoE Snapshots test digital services to make available to the market information on the true Quality of Experience delivered internationally. These QoE Snapshots should not serve as benchmarks nor as rankings of operators by service or by device. Rather, the goal of these QoE Snapshots is to provide a global overview of digital services with multiple configurations, and in various environments. The public will thus be able to better understand the technological complexity inherent to today’s services, like the distribution of video content. It is quite a technical feat – considering the efforts and means implemented – to broadcast videos on different devices and networks, with a quality that is acceptable by consumers with high expectations.
Since its origin, Witbe relies on a non-intrusive technology, based on Robots measuring the quality truly delivered. The Witbe Robots are placed at the edge of distribution, and connected to test devices, the same ones as those used by real users. The Robots measure the Quality of Experience actually delivered to the end-users by providing KPIs on the availability, performance and integrity of the service.
Each snapshot is composed of several analytical frames, highlighting interesting findings about the KPIs that were measured. In our previous QoE Snapshot, we did a drive test in London, measuring video quality for mobile operators. This time, we are publishing a look at the channel change performance of video operators across North America and Europe.