How DAZN and TAG Approach Failover, Sync, and Stream Resiliency
对于高风险的, 大规模流, redundancy is critical for ensuring smooth and reliable delivery. 这意味着要仔细监控, 良好的决策, and seamless network switching–also known as hitless failover–and making sure streams are precisely synced so the switch is invisible to the end user. 在这段视频中 流媒体连接2023主席马克·德容 CDN联盟, discusses the challenges of ensuring stream resiliency and strategies for maintaining it with Bob Hannent, 首席架构师:技术运营, DAZNMichael Demb，产品策略副总裁， TAG VS.
德容说, “If you look at the workflow up to the CDN in general, 如何确保冗余, failover, [ensuring that] you make the right decision to failover at the right time and also don't have switch back and forth, which I have also seen happening in different occasions.” He asks Hannent, “Can you talk a little bit on how you do this on your end?”
“Automatic failover is something that we shy away from,” Hannent says. “这是你可以拥有的东西, 但正如你所说, 你必须非常小心振荡. So if you end up in a flapping situation where you are going backwards and forwards, 你不会想这么做的. 但如果你在设计中加入弹性, 例如, (而且)你有高度弹性的联系, 你不明白. When we're talking about flipping sources, you want to avoid that. But up to that point, you've made sure that even if there is a failure, it's resilient in itself. 它没有影响. 但我来自一个传统的广播背景. 我以前在 BBC and other places where we could have – even in the analog era – failures that you couldn't even see less than one frame of glitch because things were timed and carefully synchronized. 我们在云计算领域并没有那么多. We do a lot with timing to ensure that it is synchronous through things, but the headend vendors have gone away as we've moved into the cloud from supporting synchronous headends in diverse geographic locations. 但实际上, that message is getting through now to some of the vendors that we would like synchronous headends in the cloud, which then comes back to the other problem with how you avoid coupling, so one failure in one location affects the failure in another location. But if you are synchronizing, they're inherently coupled. But also, as you get that high geographic distance, how do you ensure that synchronization is real? How do you ensure that the definition of time is the same? And so make sure that your sources of time are reliable and that you are measuring [it] accurately. Because the worst thing you could have is a stream switching between the two headends and time jumping backward and forwards.”
De Jong asks Demb, “What's your take on this from your perspective? 你可以对多个客户这样做.”
“我同意Bob的说法，”Demb说. 自动化和切换, 特别是CDN交换, 是不是来自不久的将来. If AI ML tools become smarter than humans and they can make the right decisions when it's really needed, 然后我们会看到(更多的自动化). 冗余就像汽车的备胎. 你没有第二辆车, so you can't afford to have fully redundant everything, 但是你的车里有一些备件. 所以如果一个轮胎瘪了, you just switch it to another tire because tires are the ones that usually go flat, 其他部分可能会断裂, 但不是经常. So, you need to identify what parts are more sustainable or more prone to fail. So, how do you ensure that everything stays in sync and switches seamlessly?
Bob提到了时间同步. We're working with a number of partners to help them enable better time synchronization technologies, 尤其是在云中. On-prem inside the same data center, we have very well-timed synchronization mechanisms today. P2P现在还不能在云上工作. So, we are working on a project with another vendor to enable P2P distribution inside cloud environments. And again, it brings me to the point: it's constant monitoring, knowing that something went wrong. It's looking for all the aspects of video quality being in the content quality. So you can make a switch between the video sources if something happens. 所以内容质量本身——视频, audio, metadata, 它是流媒体的质量, 提供服务的质量, 随便哪一个. 当然，还有体验的质量. You can collect that in real-time with different tools and know what exact experience your customers are having. 所以当监控告诉你一些事情, 它会立即提醒你, 它就像汽车里的传感器. 每个人都讨厌汽车里的黄色引擎灯, 但每个人都需要这样，因为如果有什么失败了, 你想了解它. There are several different technologies in place to help with seamless switching, and yes, you need to have all the streams time synchronized and have the right timestamps in the streams.”
Watch 完整的会议 来自流媒体连接2023年11月. We'll be back in person for 纽约流媒体 on May 20-22, 2024. 更多细节请点击这里.
相关文章
保证令人满意的最终用户体验, 无论你是提供直播内容还是视频点播, 需要弹性, ensuring that the stream doesn't break down regardless of the scale, bursts, 或其他交付需求的波动. 直播和视频点播面临的挑战是不同的, with live proving significantly more challenging in most instances. TAG视频系统公司的Michael Demb说, DAZN的鲍勃·汉内特报道, and the CDN联盟's Mark de Jong discuss the key challenges and how to address them in this clip from 流媒体连接2023.
2024年1月8日
Philo TV's Reed Barker sits down with Tim Siglin to discuss the economics of live linear streaming in this interview from 流媒体 East 2023.
2023年6月20日
Jef Kethley of LiveSports talks about how his organization develops solid resiliency models in varying production environments through the use of multiple data centers and cost-effective cloud production.
2022年9月6日
提及的公司及供应商