Viacom is a media conglomerate that creates entertainment experiences to drive conversation and culture. While digital experiences are most commonly associated with consumer-facing applications, the move toward cloud-hosted and Internet-dependent technologies has had a profound impact on internal employees, as well. As employee productivity moves into an IT teams’ direct line of responsibility, they are held accountable for ensuring quality digital experience delivery across infrastructures they increasingly do not own or control.
At a recent ThousandEyes Connect in New York City, Viacom’s IT infrastructure team shared how they are using ThousandEyes to ensure that consumers have exceptional entertainment experiences, and, that internal employees have unwavering access to critical business services. This blog post is part one of two in a series on delivering digital experiences both internally and externally at Viacom.
Business Continuity Hinges on Infrastructure
With more than 10,000 employees distributed throughout the world, providing a consistent and reliable internal user experience is critical for business continuity and productivity. According to Brendan Peterson, Infrastructure Architect at Viacom, they needed to ensure that its existing infrastructure enables employees to accomplish their day-to-day work over any number of SaaS services and applications—whether they’re using Office 365, ServiceNow, or BlueJeans. Likewise, core services (like DNS and corporate web browsing) also need to be running flawlessly to maximize employee productivity.
Central to maintaining business continuity is the ability to communicate effectively, particularly for global organizations at Viacom’s scale. So when they began hosting a CEO town hall event for employees called Bob Live (aptly named after its CEO, Bob Bakish), having visibility into critical video workflows became imperative to ensuring everyone could participate. This recurring event is streamed from a server in the studio out to partner CDNs, which then turn that stream around and deliver it to thousands of Viacom employees across the globe.
What’s the problem? Delivering real-time video to such a large audience across a number of geographic regions introduced performance inconsistencies that could impact end-user experience. After all, the video has to traverse Viacom’s corporate networks in addition to multiple ISPs and partner CDNs in order to reach the users. The complex nature of the workflow makes it very difficult to troubleshoot performance issues without end-to-end visibility. This could become a problem in a situation where users experience poor video quality because it would be difficult to determine whether the root cause is related to an ISP, their CDN, or their own network.
Gaining Visibility into a Complex Workflow
After assessing the situation, Peterson and the team decided to deploy a combination of Enterprise Agents, Cloud Agents, and Endpoint Agents to run tests from strategic locations with near end-to-end coverage of the workflow. Enterprise Agents were deployed within the corporate network environment, Cloud Agents were deployed based on CDN PoP locations, and Endpoint Agents were deployed on local workstations throughout campus offices and remote branch offices. According to Peterson, “We deployed a full complement of agents throughout our infrastructure. Starting with the Enterprise agents right next to our video stores and at the data center edge, to Cloud Agents outside our network to provide perspective on regional availability and then Endpoint Agents on workstations located in branch offices.”
After setting up a few tests, the team was able to isolate problem areas and narrow down where issues were occurring. First, they set up a basic HTTP test from the Enterprise Agent closest to the video source server targeting the CDN edges to get a good overall baseline of performance between their network and the CDN. Next, they broke that down by setting up a test in two segments: from the Enterprise Agent next to the source to the Enterprise Agent at the data center edge, and then from the Enterprise Agent at the data center to the target CDN. Peterson explains, “That gave us rapid insight into external performance versus internal performance.”
A complex problem requires a thorough solution, however, so the team did not stop there. They chose Cloud Agents in strategic locations based on where their CDN PoP’s were and set up tests between Viacom’s internet edge and those Cloud Agents, and then those Cloud Agents to partner CDNs. This provided perspective on regional availability and performance for those partner CDNs and now covered most of the workflow of getting the video to the partners. For the video coming back, they deployed Endpoint Agents on workstations throughout campus offices in the city, a few remote branch offices, and some Internet users. When those users watched the Bob Live event, the agent software reported back to ThousandEyes.
“The metrics that we collected from that event were invaluable,” asserts Peterson. The Viacom team was able to make the necessary changes both internally as well as work with their partners to resolve the video issues. They were even able to debunk some false assumptions they had around user behaviors contributing to the video issues. The resulting changes drastically improved the quality of experience for users for future events. Today, similar agents run consistently in the Viacom network and provide a good monitoring platform for similar workflows and the ability to quantify the performance of these events as they go along.
ThousandEyes Brings Clarity to Complexity
Addressing network performance issues can be particularly challenging when you consider the wide visibility gap that occurs once you cross into the public Internet. For Viacom, traffic routing across CDN providers to global endpoints created significant performance blind spots. Using ThousandEyes, Viacom is able to visualize their network performance with dashboards that are built into the platform and receive alerts when something isn’t right. According to Peterson, “The dashboards are beautiful; they are telling; and, they show a lot of data with very little need for explanation.” Having this visibility enables Viacom to monitor performance across the expanse of the Internet and to resolve video issues much more quickly than they could previously.
Viacom also benefits from the next-level service flexibility that ThousandEyes provides. According to Peterson, “We wanted a solution that is efficient, that is lean, and that is going to do what we want it to do, but not bill us for a lot of things that we don't use.” That is exactly what ThousandEyes provides. It is easy to deploy lightweight agents, run tests and access that data from virtually anywhere, anytime, on almost any device. Peterson adds, “ThousandEyes’ scalability is amazing. Our agent deployment is elastic, and it scales up and down as needed.” Using Ansible, elastic agent deployment was made easier as Viacom automated agent spin up/spin down and test execution (more on this in Part 2).
ThousandEyes has continued to provide results to Viacom in many ways. This includes enabling the team to resolve several issues, and (in a personal aside) Peterson says the team can “sleep a little better at night” as ThousandEyes continues to monitor their infrastructure. Continue reading the Viacom story in Part 2, where we cover how Viacom took what they learned to bring real-time video to millions of consumers.