Match Cutting at Netflix - Biweekly Engineering - Episode 7

Hello there! Welcome back to yet another issue of Biweekly Engineering! Hope you have been enjoying the curated blog posts shared in this newsletter so far. If you have missed any of the previous issues, feel free to check them out here-

If you find the newsletter useful, don’t hesitate to share it with your friends and colleagues! 🙂 

This week, we are back with an interesting bunch of articles from Netflix and LinkedIn.

Let’s get going!

Scalable Annotation Service at Netflix

#netflix #scalability #microservices #elasticsearch #cassandra

Netflix built Marken, an annotation service that stores and serves annotation data to the clients at high scale. The service allows multiple users to add and retrieve annotations on video content in real-time, making it a powerful system for improving the user experience of the products at Netflix. Marken is designed to handle high write and read traffic using a combination of technologies such as Cassandra, Elasticsearch, Iceberg, and Zookeeper. The use of Cassandra allows for high write and read throughput while Elasticsearch is used to provide a search backbone for the annotations.

The article also discusses the challenges faced during the development of Marken, including dealing with high write and read traffic, providing a search interface for the annotations, correctly designing generic schema for variety of use cases, etc.

Netflix engineering blog is an awesome place for quality engineering articles. I personally liked this post a lot as it started with giving proper context to the readers, and then moving towards explaining the complex parts. A well-written article undoubtedly.

Match Cutting at Netflix

#netflix #machinelearning #computervision

Here is another one from Netflix. I personally found the idea of match cutting and the approach to solve it as a machine learning problem very interesting.

In this heavily technical and well-written article, Netflix discusses a process used at the company to edit video content in a seamless manner. This process, known as "match cutting", involves finding cuts in video footage that maintain visual continuity and create a smooth transition between shots. The goal of this process is to create a viewing experience that is seamless and engaging for the audience. Match cuts are heavily used in teasers, trailers, or advertisements.

The article explains that the match cutting process at Netflix involves the use of a machine learning model that analyses the visual content of the video footage to find cuts that will maintain continuity. The model considers factors such as lighting, camera movement, and shot composition to determine the best cuts.

The blog post also explains that the match cutting process is an important part of the post-production process at Netflix, as it helps to ensure the quality and consistency of the content that is released to viewers.

Overall, a very interesting read!

The story of scale at LinkedIn

#linkedin #scalability #monolith #microservices #soa #kafka

This article provides a comprehensive overview of LinkedIn's journey towards scaling its architecture to meet the growing demands of its user base. The authors discuss the various stages of LinkedIn's growth, from its early days as a startup to becoming a major player in the social networking space, and the technical challenges and solutions they encountered along the way.

As a backend engineer, I found the article quite intriguing. The authors provide a detailed look into the various stages of LinkedIn's growth and the technical challenges they faced, such as increased traffic, data growth, and performance issues. The post highlights the importance of having a well-designed architecture that can accommodate growth, and the various approaches and technologies that LinkedIn built and used to overcome these challenges, like Kafka (Kafka was born at LinkedIn), caching layers, Rest.li, etc. Just like Netflix or Uber, LinkedIn innovated a lot to solve their unique problems to scale their systems well.

One very important lesson from such stories is the idea of starting simple. Notice how LinkedIn like many other successful companies started as a monolithic application and broke out of the monolith when they saw the need. I have seen examples of companies building microservice-based architecture from the early stage, thinking that they are doing the “right” thing. Unfortunately, this could easily become a bottleneck due to the complexity of managing microservices.

That would be all for today. Hope you have found the articles insightful. See you again soon! 👋