Introduction & setup
The environment for this Snapshot consisted of several Witbe “Video & Media” robots hosted on Witbe Cloud in 11 cities across North America and Europe: Denver, London, Lisbon, Madrid, Montreal, New York, Paris, Phoenix, San Jose and Toronto. The Witbe robots were each connected to an operator’s STB.
A total of 22 different operators were tested:
North America:
- Altice Optimum One in New York
- AT&T Uverse in San Jose
- Bell Fibe in Montreal
- Comcast Xfinity in Denver
- Cox Contour DTA in Phoenix
- DirectTV in New York
- Dish Hopper in New York
- Rogers Ignite TV in Toronto
- TWC Spectrum in New York
- Verizon Fios in New York
- Videotron illico in Montreal
Europe:
- Bouygues BBox in Paris
- Brandt DVB-T in Paris
- BT UHD in London
- Canal + Cube in Paris
- Free Revolution in Paris
- Orange Livebox in Paris
- SFR Box in Paris
- Sky Q Mini in London
- Telefonica Movistar in Madrid
- Virgin V6 in London
- Vodafone Portugal in Lisbon
Witbe Remote Eye Controller
The image above displays these configurations in Witbe Remote Eye Controller, showing several streams on a single screen.
Each robot was tuning to a set of 10 channels, among the most popular in its region, using direct access (pressing the channel code’s digits on the remote, with a validation code when necessary). The channel was considered changed when both the audio started and the video was in motion. This replicates actual end-user experience, but creates two side effects:
- Because the robot waits for both audio and video motion, measurements will be more severe than most operator’s standards.
- STBs tuning to 1 or 2-digit channel codes will take longer than STBs tuning to 4-digit channel codes because there will be no wait time for an eventual next digit input.
We acknowledge that this methodology is biased to some operators (especially European ones who have less 3-digit channel codes in their lineup), but we made this choice to measure the actual wait time perceived by end-users.
We measured and verified the following KPIs:
- Channel Availability: The channel was considered available when audio started and video was moving
- Channel Change Time: Time between the last remote code digit input and the start of both audio and video motion on the channel.
- Numerous other perceptual KPIs (Time to First Frame, Time to Audio, Time to Video Motion, etc.)
- Numerous other network KPIs (Network Channel Change Time, RTP Join Duration, Unicast Burst Time, etc.)
The KPIs were measured on each channel every 5 minutes over a 7-day period, from September 4th to 11th, for a total of more than 350,000 channel changes.
Every channel change was performed using our new Channel Change engine which includes reengineered perceptual and network algorithms, a dynamic scheduling system, and a new visualization widget in Witbe Datalab – our visualization interface used for fault finding, root cause analysis and advanced analytics.
Global Overview
The image above ranks all the STBs tested, by Channel Change Time average (only the top 3 operators in each region are made public — please contact us if you want to know your position in this ranking):
The first thing we can see by looking at the average Channel Change Time is that the best-in-class are:
in North America:
- AT&T Uverse in San Jose
- Bell Fibe in Montreal
- Rogers Ignite TV in Toronto
in Europe:
- Virgin V6 in London
- Vodafone in Lisbon
- SFR in Paris
We can see that the top 4 performing operators for this channel change test are North American, with a Channel Change Time under 2 seconds. It can partially be explained by the fact that European operators have more 1 and 2-digit channel codes in their test lineup.
It is also interesting to note that both AT&T, Bell and Vodafone use Mediakind’s “fast channel change” technology, helping them reach the top of their respective rankings.
Lastly, we measured that 3 of the 5 operators that have the longest Channel Change Time rely on satellite technology, with which slower tuning times are usually measured.
Channel change 101
There are three ways of sending video to a STB:
- Broadcast stream: one emitter (a satellite or a Hertzian antenna) sends the signal to every connected receiver
- Unicast stream: one emitter (a video server) sends the signal to a single receiver
- Multicast stream: one emitter (a video server) sends the signal to a specific set of receivers (users watching the same channel in the same area)
Multicast streams are the most widely used because they balance cost effectiveness and performance, but they require more time to start sending the video packets than unicast streams.
What exactly happens when you press a digit on your remote? With an IPTV STB, first the STB unsubscribes from the previous channel (that’s called an IGMP leave). This leads to the apparition of a black screen. Then the STB subscribes to the new channel (that’s the IGMP join), connects to the associated multicast stream, starts to buffer the video, and eventually displays the new channel.
The video above shows a typical channel change for IPTV STBs with a black screen in between channels (Witbe robots record video traces for every test in order to replay the scenario and understand what the robot measured — this video was re-encoded and compressed).
The duration of a black screen is usually linked to two factors:
- How long does the STB need to buffer the video before being able to play it (this is a STB software and encoding optimization challenge)
- How long does the server take to respond to the IGMP join query and send the video packets in the associated multicast stream (this is a server and network optimization challenge)
But how can operators decrease the duration of black screens?
Examples of best-practices
Improving Channel Change Times is one of an operator’s many challenges. One of the ways to reduce the duration of channel change is to deploy “fast channel change” technology inside a network (Mediakind technology for example). This requires heavy investments and network modifications, but it produces excellent results.
Two improvements led to the very fast Channel Change Time seen in the video above:
- Software was optimized to attain a Video Buffering Time of 383ms
- New servers were added to the network to first deliver the channel in a “unicast burst” stream, creating an instant connection between the server and the STB, allowing video packets to be sent instantly before seamlessly transitioning to a multicast stream.
There are other ways to improve Channel Change Time that rely on smart engineering techniques rather than heavy investments:
In this example, the network architecture was not modified. However, the software of the STB was highly optimized and was able to replace the black screen entirely by freezing the last frame of the previous channel and then the first frame of the upcoming channel. By doing so, the Time to Video Motion of 4s is the same as if there was a black screen, but the Time to First Frame of 3.1s was effectively improved by 22%. Compared to using a black screen, this method has a far more positive impact on the end-users’ perceived QoE.
Conclusion
In conclusion, we can see that when it comes to Channel Change Times operators in North America offer a better Quality of Experience than their European counterparts, most notably by attributing 4-digit channel codes to their top viewed channels.
That being said, who would have thought that an operation as simple as changing a channel would be so technically challenging and require so much investment! Channel Change Times is a critical KPI that video operators monitor closely because they know it matters to their end users.
Stay tuned for the next part of this Snapshot series!
About Witbe QoE Snapshots
In the same way that a consumer report tests a product and publishes an analysis of its overall quality, the Witbe QoE Snapshots test digital services to make available to the market information on the true Quality of Experience delivered internationally. These QoE Snapshots should not serve as benchmarks nor as rankings of operators by service or by device. Rather, the goal of these QoE Snapshots is to provide a global overview of digital services with multiple configurations, and in various environments. The public will thus be able to better understand the technological complexity inherent to today’s services, like the distribution of video content. It is quite a technical feat – considering the efforts and means implemented – to broadcast videos on different devices and networks, with a quality that is acceptable by consumers with high expectations.
Since its origin, Witbe relies on a non-intrusive technology, based on Robots measuring the quality truly delivered. The Witbe Robots are placed at the edge of distribution, and connected to test devices, the same ones as those used by real users. The Robots measure the Quality of Experience actually delivered to the end-users by providing KPIs on the availability, performance and integrity of the service.
Each snapshot is composed of several analytical frames, highlighting interesting findings about the KPIs that were measured. In our previous QoE Snapshot, we did a drive test in London, measuring video quality for mobile operators. This time, we are publishing a look at the channel change performance of video operators across North America and Europe.