topic (really, a question):
here's an honest question i've been stumbling over for 4 months: what do we get out of content recommenders? what's the benefit to us, as users? i'm not asking about whether they help us discover new stuff, products, content. i know they do that. and i certainly understand the product implication of a good content recommender for a company. they're built to increase loyalty, engagement, to decrease churn, to sell more products, to sell better products. they're built to ensure a better immediate-term outcome. but what i wonder is: how are they better than the alternatives? in an alternate universe, could something better be out there?
pre-discussion: what is a content recommender, and how does a basic one work?
here's a recommender that a lot of us are familiar with, and an example i'm going to keep referring to.
it's netflix! below, i highlight the recommender's output-- Named & Recommended Categories, and Recommended Titles:
ok, and... how do recommenders work?
recommenders can work LOTS of ways-- so many-- maybe even as many ways as there are engineers. but the bare bones that the recommenders Netflix builds, for instance, start with a list of users, a list of films, and then does some extrapolations predicting ratings of unseen movies based on seen movies and similar users. Visually, a little dataset might look like this:
so basically there are people in the rows, movies in the columns, and then the values filling the table would all be ratings. not everyone has seen every movie, so there are blanks. recommenders guess at what the future ratings in those blank spaces might be.
how do they do that?
the methods vary, and they really don't have to be complicated. in the example above, you could probably guess a rating from looking at it for a while. i would guess 4 stars. people use correlations, normalize individual users' ratings to the user's average, and other things.
once a dataset resembling this example is compiled, and using as many methods as you care to, the process for recommenders goes:
1) look at your data and make some predictions about ratings (that thing we did above, by eye)
2) figure out how to rank the ratings predictors you build (is a very confident prediction more important than a higher rating prediction that's less certain?)
3) figure out how to best show these predictions to the user (the obvious choice: if you predict a user will rate a film highly, show it to them)
these problems are fun to work on, and probably fun to test in the wild (i've implemented product recommenders on retail sites, but never worked on my own custom recommender project professionally).
moreover, especially for shopping sites, recommenders are fun to use when implemented correctly. they can help you discover something you really like, without needing to know tons of keywords or brand names. they can be amazing.
they can, though, also be pretty awful and frustrating. to anyone who's ever gotten trapped in an 'endless loop' of the same 4 red midi-length dresses, it's obvious (to me, after working in eCommerce for a few years) that implementation and data quality really matter for recommendation systems.
so my big wonder is, can we 'feel' whether the logic of a sophisticated content recommender, implemented over a large dataset, is helping us find new interesting stuff more than it's hurting us? because it's pretty rare for a company that relies on content recommenders to make the rules it's using explicit to the average consumer. both amazon and netflix do a pretty OK job of "because you XXX" , like this:
this kind of labeling is a really good thing, i think. context is good. but this isn't the kind of transparency i mean. Netflix, for instance, has a bunch of 'categories' that it creates on the fly-- "Action Movies with a Strong Female Lead," etc. many of the categories feel 'easy' to read. i get why Netflix thinks I'm a fan of strong female leads-- i probably overindex on TV that features women with speaking parts (low bar). but what i don't know is what other kinds of categories i'm not being shown. especially if ML techniques are used to create the categories, there could be millions of categories. i might see 10 names in a month. what kinds of other categories do people see? are there white supremacist netflix categories? I don't know.
that said, i'm not sure there's an easy way to even communicate "transparency" to users, since most users probably don't think about this kind of thing too often, or don't have the technical background to understand the sophisticated techniques used to implement recommender systems. netflix has a super cool blog about new techniques its engineers develop here, but it's not accessible for the average couch картошка (and i wear this label proudly).
ok so wait, go back. content recommenders can hurt us?
it feels weird to say that, since who cares? content recommenders feel like fun, gimmicky games to me most of the time, especially on retail sites where i'm just deciding between inventory the store has already stocked, and i'm not worried that 'picking the wrong dress' or 'not seeing the right dress' is somehow a big deal. i don't think it's, like, a world peace kind of issue. or at least, it doesn't look like one.
here's why i think content recommenders can end up hurting us: for services like Netflix, everything that isn't in the Search function is essentially the output of a recommendation. right down to the cover art. netflix has so many brilliant engineers working on new recommendation engine types-- interleaving being one. they're brilliant. that's cool. i hope they're having fun and i hope they're achieving their goals.
but ultimately, netflix's homepage is a zero-sum setup. so any specific item recommended by the content recommender also means something else was suppressed. a content recommender is, by definition, a content suppressor, too.
i think this is where i differ from data scientists i've met in the valley. maybe i spent too long being a traditional media research dork, but i value subject matter expertise in media. i'm not the biggest fan of the infinitely customizable experience. it sort of makes me feel... alone.
i value transparent (or even semi-transparent) editorial choices. networks decide what shows they're going to run, magazines choose which writers they're going to hire on staff. the media industry has leaders who filter out what does and does not belong to their brand. something about that is very much lost when 100% of your 'editorial' decision amounts to 'whatever the person is most likely to watch.' i don't know how to summarize that stance. it doesn't sound like something that's going to challenge me, or even keep up with changes in real time.
ok, that was kind of confusing. let me go back.
let me explain with a picture what i think is lost when we rely 100% on content recommenders, whose logic is generally 'guess-able', but is opaque to consumers, instead of people-based, more explicit 'recommender' systems (aka TV networks, newspapers, magazines making explicit and slo-mo calls about who they want to be).
here's an egregious oversimplification of what i'm afraid of. the basic concept is:
there is a positive feedback loop on the left, and a negative feedback loop on the right (I'm not a physicist, but this picture makes sense to me). traditional media, despite its many faults, was (for me) a positive feedback loop.
what's that mean, when it comes to media and entertainment?
a positive feedback loop, to me, is one where maybe i have to deal with the occasional new show that's stupid or sequel that's derivative, BUT i also get cool new shows that come from a bunch of different places. i expect to channel surf, and give different networks and different shows a try at various points, even if i haven't liked them in the past. i get to choose, in this scenario, from a lot of options that are somewhat predictable but not completely.
the idea of diverse media offerings, beyond just increasing reach for a network, is development of content that isn't an instant hit. i hated The Office the first time i saw it. so did a lot of america. it wasn't a hit until its fourth season. it's a vital principle of media, that over time, you 'invest' time in new ideas that aren't derivative of old ones. there's room for disruptors, room for new hits and flops and everything in between.
the big problem to me with the Netflix approach to content recommenders is the dosage. there aren't 'boundaries' around recommendations, the way there are on retail sites like Nordstrom, which fit product recommendations into neat little tiles at the bottom of a product page, or isolate them to a "for you" tab in a mostly-editorial site.
even in the early 2000s, when netflix was mailing DVDs to and from our homes, we didn't really need recommenders. our media environment was much more limited, and much more communal. we had TV, radio, magazines, newspapers and movie theaters. those functioned as recommendation (really, just marketing) engines. they weren't personalized, of course. discovering a niche TV show from australia (anyone remember's McLeod's Daughters?) or anywhere else was a social thing. you might read about it on a message board, or be recommended by a friend. 'recommendations' existed-- and they still do. but the algorithmic recommender has no sense of 'finiteness.' you could scroll forever. it makes the experience weirdly out of control.
to their credit, one of netflix's high-priority categories is "where you left off" or "your list," which gives you a little control over what you see when you log in. but 90% of what you see, or more, is generated by a recommender engine. because of this, to me, the 'editorial stance' of Netflix almost feels like it doesn't exist.
so, somehow, the same thing that i love when i see taking up, idk, 10% of nordstrom's visual space, in a product recommender, i get very squeamish about when i see taking up 90-100% of netflix's space, for the following reasons:
this isn't to say Netflix isn't working real hard on developing shows and testing the absolute hell out of them. but they prioritize the shows they themselves are producing. so they uprank those shows in their recommendation engines. which makes sense-- owning their content, shifting it to 'generic' Netflix-produced content (some of which is amazing, don't get me wrong) instead of ABC/FOX/Lionsgate etc. produced content, is better for their bottom line.
But for us cord-cutters, or just us lazy-binge-watchers, what are we missing? maybe we have Hulu, and maybe we have cable still, and that's great. but Netflix is producing a TON of content that's exclusive to Netflix, so it raises questions of bias in personalization, to me.
why are you so up in knots about this? seriously, who cares
ok so here's the story that got me thinking about this, and it's a little melodramatic but IT'S MY LIFE and you don't have to be here! but oh do please stay because i am trying to be reasonable. and the comments are open for a reason, so give a shout if you're hating this.
I LOVE Aziz Ansari. Ignoring the whole 'terrible sex' thing, i was a big fan of Master of None's first season. i watched it twice, and I made my parents watch Episode 1 at home while I cried and hugged them. that show genuinely brought me a new perspective on my parents and it allowed me to express feelings towards them. that is cool.
so obviously i was all about catching the second season when it came out, and netflix was onto me about this, because as i mentioned i watched season one a few times. the day season two was released, netflix sent my phone a push notification about it, through the Netflix app. this made me feel happy and excited and also made me feel like Netflix 1) got me and 2) was making a push to support Aziz and to support brown people doing TV stuff. which i'm 1000% for because I didn't meet or see any brown people in person or on TV until i moved to manhattan. at 17. AND I WENT TO HIGH SCHOOL IN REDMOND WASHINGTON. so. come on america. fund more productions by brown people. tf.
as it happened, the day the second season of Master of None came out, i was dog-sitting for a friend, and so i logged onto their Netflix in order to watch it. i'm not often in other people's homes, logging into their Netflix accounts. i expected it to be 'different' from my Netflix.
what i didn't expect, given that I'd received a push notification from Netflix about this show earlier in the day, was that Master of None wasn't being promoted on this friend's main page.
or below the fold. or anywhere.
once i realized it wasn't being featured as a general rule, i decided to 'look' for this title using my friend's login. i looked. and i looked. i spent 30 minutes looking.
i learned later that i wasn't using my friend's login-- i was using some login they'd given to someone i didn't know, a long time ago. so i don't know whose account i wrecked. but i gotta admit-- i was looking at both Netflix and my friend a little crosseyed until i asked my friend about it. it didn't even occur to me that this Season Two event wasn't the news of the day to Netflix until about 15 minutes into this search of my friend's Netflix feed.
but i realized through this experience, Netflix doesn't have news of the day. they're always launching content, always retiring content, always breaking out a new star and always quietly sweeping another one under the rug. in as many ways as they have viewers.
obviously, there are some types of content that they do prioritize: they're probably showing most users their own produced shows, like Dark and Altered Carbon. but maybe they're only showing them to people in my little nerd bubble. it's really easy for me to believe, though, that they're much more selective about who they're featuring standup comedian specials to. there are probably enough netflix-sponsored comedy specials to analyze by the comedian's skin color vs the user's skin color. and who knows, maybe cordcutters out in Nebraska who are reality TV aficionados never catch a whiff of any of it.
i don't know. because i don't know what Netflix is about, except for money. and that's a weird position for a media company.
so i get it, netflix. if i'm looking for a good movie, then i might find one slightly faster using the recommender system than i would have without a recommender system. good.
here's the problem: content recommendation is a zero-sum game. for every movie an algorithm upranks and decides to show me, it deranks something else and doesn't show it to me. a content recommender is also a content suppressor.
so why not think about tools netflix could build to give a better sense of control over what we see?