A/B testing & machine learning
When you are surfing Netflix how do you pick what to watch? Is it a catchy title? Maybe it the synopsis or trending right now. Perhaps it’s the unwillingness or rather willingness to watch another episode of your favourite rerun.
Just maybe, a particular thumbnail catches your attention – but go on your friends Netflix account and you will probably find that the cover art, or thumbnails, doesn’t interest you. In fact, what you see, on average, is completely different and that isn’t a mistake.
Your thumbnails are tailor-made just for you.
At its core, one of Netflix’s unique selling points (USP) is hyper-personalising experiences or rather calculating them. They will help you choose content to watch with curated trailers of upcoming programmes, highlight new episodes or seasons based on previously watched shows and even gauge your interest with match scores.
While streaming services are notoriously private about sharing their data – over the years, Netflix has given the world glimpses into how their technology works. According to their internal studies, a typical viewer spends a mere 1.8 seconds considering each title and they believe it only takes 9 seconds to get your attention before you move on.
Among all the things that could potentially catch your attention and make you binge a show. Netflix found the biggest influence are still the thumbnails. Our eyes move 3 to 4 times a second to process new information; and because Netflix’s goal is to catch and hold your attention, the company has allocated a lot of resources into choosing every single thumbnail you see. Before they decide what thumbnail will work for each user, there is more data mining than you can imagine that needs to be analysed.
Take for example an hour of your favourite show, say ‘The Witcher’. Its hour-long run time has almost 86,000 video frames. To figure out which one would work best as thumbnails, Netflix uses a selection process called Aesthetic Visual Analysis (AVA). AVA is a set of tools and algorithms that search Netflix videos for the best images and videos pulling them out to create thumbnails.
It is a five-step process:
1. Frame annotation: A programme analyses every static video frame of a video.
2. Image recognition: Algorithms use information gleaned from Frame Annotation to create metadata. The metadata showcases unique characteristics to each video frame.
3. Netflix groups them into 3 categories that are key to identifying good images:
Visual: Brightness, colour, contrast and motion blur.
Contextual: Face and object detection, motion and shot angles.
Compositional: Focus on visual principles in cinematography, photography and design (such as the rule of thirds).
4. Image ranking algorithm that sorts the metadata to pick out the specific shots that Netflix has determined the most attractive and clickable – ones that aren’t blurry, have varied imagery, feature major characters and don’t contain unauthorised branded content or sensitive.
5. The first step that isn’t automated – a creative team steps in to design the best thumbnail artwork and add programme and branded titling.
After all this – Netflix still does A/B testing again, again and again. Your thumbnails will change regularly based on your engagement with previous titles. For example, if you are a stand-up comedy fan and have binged all Kevin Hart’s specials, if you then search ‘Good Will Hunting’, you may get a thumbnail with Robin Williams, comedian and one of the movie’s main characters. More into romance? You may get cover art for ‘Good Will Hunting’ with the two leads kissing.
Netflix is obsessed with A/B testing everything; brands and agencies can learn a lot from this and becoming better at creating personalised experiences in an age where it is expected from your users.