How YouTube uploads and streams billions of videos with ease?


Did you know that there are about 500 hours of videos uploaded on YouTube every minute? 

That's huge. It's like watching 230 movies. 

But, this is possible as YouTube has over 2 billion users. It is the second most visited website after Google and the second most used social platform after Facebook.

You can find more of such mind-boggling stats here.

But now let's focus on how YouTube does all of this? Let's discuss some functional aspects of YouTube which make it a God of video streaming.


When you upload a video

Once you upload a video you must have seen a processing bar.


Video Processing

People upload videos on YouTube with different file types, resolutions, and formats from their favorite devices. YouTube focuses on making these videos playable on every possible device around the globe.

Now you might be thinking why not play the original video?

So it might be a case that the uploaded video is so big that it becomes impossible to distribute the video due to constant buffering and also due to high data usage it might break the internet !!!

To avoid this YouTube makes the video smaller. In this process, they take an original high-resolution video which they call "Mezzanine" and slice it into chunks of maybe 5 seconds. These chunks (5 sec videos) are passed to different machines for generating compressed versions of themselves in different resolutions (1080p, 720p, 480p, 360p, 240p, 144p). These chunks are then stitched back together. 


Codec (Compression & Decompression)

YouTube further makes the video smaller by compression.

Ahh, compression sucks !!

The videos are compressed up to 95% in some cases, but thanks to our brain that is still able to figure out correctly what we are looking at. 

The video made from a high-resolution camera captures a huge amount of details that a human eye cannot perceive. Codec simplifies this by compressing and filtering those details which are useless for the human eye making it watchable and smaller at the same time.

These videos are further compressed by not refreshing the stuff that isn't changing in each frame. 

That means if you upload a video sitting in a place with a non-moving background, the frames are refreshed only for the moving parts of the video. This makes a video lot smaller.


When you Play a video

Imagine what would happen if you play a video on YouTube and it downloads before it actually starts playing. 

In the initial days, Youtube did this very thing to stream videos and the videos used to buffer a lot.

Adaptive Bitrate Streaming

YouTube came up with an algorithm that selects the video according to your network status, how big is your screen, and how much video is already downloaded. It is called Adaptive Bitrate Streaming.

Suppose, you are watching a video and your brother comes and starts using some bandwidth by downloading a huge file. YouTube will analyze this and downgrades the file quality to avoid buffering.

Transcoded videos with different quality will be shown as your network bandwidth is compromised or vice versa. YouTube's main focus here is the video should not buffer even if the quality is low. The users tend to leave the app if the video takes more time to buffer. The idea behind this is to retain a customer base.

You must be very familiar with this grey bar that leads the red progress bar shown above.

This is the amount of video downloaded in the browser's cache. YouTube chunks the video into smaller pieces (e.g. 2 MB each) and downloads them in the browser's cache which makes it more manageable when you want to go to the beginning of the video. In that case, it is able to retrieve that chunk from the browser's cache itself without contacting YouTube for that.


Content Distribution Network

To optimize the time for video retrieval, YouTube has partnered with all the major Internet Service Providers (ISPs). They have put a box with a rack of hard drives within the ISP datacenters which temporarily stores the popular videos which have already been streamed in that area. These are called Edge Nodes or Google Global Cache or GGC.

YouTube sends the popular videos to the CDN which is then replicated to GGC which are very close to the users. The videos which are previously streamed or popular in a particular area surely make it to the GGC.

So the next time when you play a popular song, your request is highly likely to be served by a nearby GGC and it won't go to YouTube's central service. If the video is not available with GGC, it asks for the video from the central service which is then replicated on the edge node and streamed to you.


Will YouTube run out of space

Do you really think that will happen?

No, YouTube has a full-proof plan for that. They have large data centers to take care of this problem. 

Over time the storage devices have significantly reduced in physical size, they can be used very easily to store huge amounts of data with less physical space needed. Nowadays there are modern racks of hard drives which are installed in a data center.

YouTube has a distributed network of its data centers. They are present all around the globe to serve the requests faster. For more information on Google's data centers follow this

Also, they have multiple copies of backups of the data present in different geographical locations. This is done to have high availability even in case of a disaster. 

If a data center goes down due to an earthquake, the data will still be present at other locations which are then used as a primary copy or a primary server.


Conclusion

YouTube does a whole lot of other stuff other than what I described here in this article but these were a few things I could get my hands on. There are other aspects as well such as Copyright and Content ID management etc. which I want you to explore.

So the next time you upload a video or play one you would exactly know what just happened behind the scenes.

See you next time.




Comments

Popular posts from this blog

What happens when you watch stuff on Netflix?

DHCP: Dynamic Host Configuration Protocol

Concurrency vs Parallelism