foreign we have our next bio break we will have another really exciting talk and I know we all need another coffee but uh stay tuned now and we will talk about responsible AI as I mentioned beforehand the roads to carbon neutrality and this time it's exciting because we have two speakers here and I would like to ask you to come on stage laughs to talk about tell you something about the general approach of City F to Ai and machine learning with a special use case of recommendations um so maybe you know it's Atif that's why the Deutsche fans in yesterday was a at a speaker's dinner and not everybody under 30 was wealth that there's a channel in Germany that sends linear TV so I just want to emphasize that so besides linear TV we obviously have a set of Media Tech that is our streaming service that is quite successful in Germany where we have a full program and I would like to emphasize that you see it here in the right side and
a screen that a lot of the content on the start page is generated by algorithms obviously so when we start with the stage uh teaser and also the next content cluster that is all run by algorithms right now um overall that generates roughly about 50 of the viewing volume that comes from the start page so very important um and what's also really interesting and that's important to us as a Public Service Media broadcaster is that we have 20 of all the content that is available within mediatek and a lot of content available are recommended and used each day and that is of course magnitudes more than some editors could do it in the manual fashion uh very interesting especially important is recommendations on smart TV on the big screen where you just don't want to yeah interact with the with the controller more than ever needed so very important on that um but there we don't want to stop we won't go into further and further personalize and automate this set of mediatique and that's
very important Point here we do this according to our Public Service Media remit and that is some kind of Regulation you had about this topic of the day and uh before we go into the details I would just would like to highlight that we build up a team for that that is the personalization automation team at cityf uh we built up that team since 2018 roughly um the people there have a very different background so we have editors in there we have also data Engineers data scientists um in here in different roles those people are running the service itself um right now it's a multi-client service so the red is also using that service also some kind of sustainability um and we also develop our own algorithms um and what's interesting the background of the different people I just mentioned we have also people with a background nuclear physics and who are really strong in statistics for example and these people are really determined and have really the Public Service Media at their hearts um
and they are facing some really interesting challenges I want to highlight right now um so when you start with the public service media Rama that is encoded in law and paragraphs so a lot of text and you just look down vertically to the algorithms where you have code and data it's obviously not so easy to tell if there's a really correlation or some kind of alignment and what that makes even more complicated is that you have different distinct layers of people and communication standards where you have a common sense for example talking about the psmr that is the domain of course of our boards and Regulators we have a lot of them that's good of course we have the editors who are in charge of doing the right thing on the website and they're talking about these peace and values like impartiality transparency and all that stuff and on the other hand on the lowest level here we have of course the data scientists the machine learning engineers and all people handling code and data
and they're thinking about something completely different they are talking about information retrieval metrics like meanwear it's a program not discounted going to gain hit rate um and that is really kind of a problem and back in 2018 when we started this personalization thing we had to advance some concept how to bridge things and that is what we would uh we we put up as a concept here is that we said okay we have to bridge that and so we have to develop our own metrics so the PSM Matrix um especially to address the effects of recommendations when it comes to the audience most important for us of course and the content Partners we have a lot of content Partners right now we have nearly all the Rd content within set of Media Tech right now and we put up them so they are based on some really established metrics like for example coverage so how much do we use at the catalog novelty how often do we have Niche content available that is really important
to us and of course diversity there was a t when we mentioned that is of course it's it's kind of family of metrics so diversity has a lot of different detailed definitions and the most important one is for us of course Serendipity so we want to bring users to use different content or different topics and that is something that is deeply within the mindset of the public service media especially so we set up that and something that is also now a DNA is transparency obviously um so we said okay we've got this concept now we have to talk to the people and bring that into the public we set up a website I grew up and said FD um where we explain all the use cases so we tell okay this part of the media Tech is automated or personalized we use this algorithms we have made that decision that is uh the kpi in the special use case here and we also deal really openly with our own limitation and weaknesses so for example
we've got a lot of bias in our data so we've got a lot of problems here but that's really important to address them openly and get back with feedback and improving on that so that is not the idea we we tell you what is right and wrong and we are the best ones it's more like okay that is our current approach if you have something we can approve just tell us um also this is I think a step before the next step we are playing right now so the next step would be for US Open sourcing all the code we have so the code basis and especially the algorithms why haven't we done it the other way around because these thing it's really important to have a concept A conceptual framework what we try to achieve with those algorithms and just handing all the algorithms is not so interesting at all um yes and of course transparency is the basis for an informed discussion we were hoping for a lot of feedback now I can
tell you we got since we launched that in November I think 10 emails so 10 people half of them said I'm clever enough I don't need algorithms I can search for myself okay that's fine that's also fine for us obviously you can just switch off personalization completely within centimetateague um but we also got some some really valuable feedback and interesting discussions here um what we also have there is um a bit more data and a bit more technology so one really interesting problem is when you learn from past data um that you have uh data bias data in so there's some hero content that's used much more than other content and there is a lot of papers around there where there where developed counter measures and this is a paper that was published last year on the ASMR access I would just put that paper and it has really nice approach because you just have to adjust um your Evolution function there's a term you can bring into your elevation function and then things go
better and we tested that and the right side you can see and that is also online available what uh what that result brought to us so if you can see here on the right side we have an A B testing running all the time and within the D bias Group C you can see there is a lot less popular content recommended that's really great that's the goal we try to achieve but also and that's also really interesting for us um that we have no decrease in viewing volume but we've seen increase in viewing volume so people really think it's a great thing that we don't have just really popular content because those recommendations are more tailored to this specific interest of the users and not just what nearly all or everybody really likes to see so also very interesting um now let's go pick deeper into the sustainability thing um carbon neutrality is our sustainable goal number one overall when it comes to recommendation algorithms we back in 2021 um had a talk earned a
paper at the rexus conference and we can't determ green recommendations that means where we can we save energy and the different parts of a recommender system and there are obviously some ways to do it something yeah we have just already done for for example using sheller models is really a means of using much less energy of course using less data is always interesting we've also deeply into online training so for example we are using within the stage teaser reinforcement learning where you just going to learn and probability distribution of your content an online fashion and you don't have this really high volume High GPU loaded retraining and batches from past data also in the stage teaser we have user segments so we first started in a lot of use cases with Hyper utilized recommendations or for a specific user with a very very specific viewing history he got us really special and individual recommendations but we found out that in some use cases it's good enough I interested in discussion what that means in detail
but it's good enough to just recommend and use a segment so you're also saving a lot of engine costs and that is for the training and the algorithm part but also within the influence of course caching really plays a big role also adaptive free computations really heavy computations you can do when you load and the gpus is not so high actually and also again there is the idea that maybe user segments can be more power efficient and lead to comparable results so this is more like a broad overview um the next part life will tell us tell you about some experience we run also right thanks a lot um Andrea so um yeah indeed um we have been um the big one all right yeah so I would like to tell you a bit more uh what we did concretely together uh with set AF in an experiment so we are saying we're a research-based startup we focus on green Ai and more precisely on personalized semantic search and recommendation so our DNA is also
transparency sustainability and also privacy and I will also go a bit deeper into how this approach works for us so first of all we actually focus on language models now of course we talk today a lot about language models we talk about mostly large language models right everybody knows now about GPT chatgpt Etc and those models of course are great when it comes to generation of text when it comes to processing not a lot of data actually right so we have really big models that are actually suited for not a lot of data so we actually take another approach we actually go from Big to small so that we can actually really process a lot of data and with a lot of data we mean millions of documents or even more and the second aspect is privacy and also in privacy we actually go also the other way around we go from Big to small so instead of just focusing on all of the user and base and all of the well content and and
transactions that these users process and looking at them at once we look at them individually so what we do is actually we build a known model a language model it's a small language model so we actually go from generalization to specialization we focus on search as mentioned and recommendations and we also focus on distilling this model very specifically to these use cases and making it as small and efficient as possible I will tell you later about about the actual metrics behind that um and well we even achieved with our system that we can run for example these language models these Transformer models uh on mobile phones instead of super computers that have to be developed now so it's really suited for processing a lot of data so here is a more concrete example of what this actually means so on the left side we compare ada2 from openai which is a comparable model so you can actually do comparable things so you can embed a lot of documents with it and you can for example
do a similarity search you can do asymmetric searches so you can do question answers on a lot of documents obviously but if you look at the actual computational power that's needed to process these models and in comparison for example to our model Xenia then the difference is gigantic really so we talk about 98 energy consumption less and just to have a few hints of what you can actually save for that you can shave more than 8 000 beds with bits with that or for a more berlin-based example you can actually run a massage chair for 14 hours straight um and uh so obviously this energy reduction also relates straight away into cost reduction right so obviously you save CO2 but on the other hand you save also cost and cost is of course really uh important when it comes to processing a large number of documents so this is a customer example from us where we have to process around 80 million documents and obviously you need to do this in a very efficient way
otherwise no one will actually use it in production and we hear this actually a lot when it comes to language models and Transformer models and natural language processing a lot are actually talking about it a lot are playing around with ChatGPT with gpt4 gpd3 Etc also bird models from Google but no one really uses them in production only very few and only for use cases where it doesn't come to a lot of data a lot of search requests and also in this example here we talk about millions of transactions and the costs are of course really important when it comes to production use cases so we are actually focusing on sustainability but then actually leading to production cases now let's come to 30f so what did we do here so set a f has um as Andreas introduced to you the mediatek so it's a classical streaming platform right everybody knows these streaming platforms you know Netflix you know Disney you know how they work right without recommendations without personalization there would almost not be
usable because the content base is just too large right so these platforms need to understand the users these platforms need to understand the content behind them and one of the main use cases on these platforms and I think pretty much everybody of us here has experienced this use case I would even say 100 of you is a lean back scenario so you watch something and recommends to you the next content it's a very very classical use case now of course set AF mediatek as a streaming platform also has to serve this use case right I mean you have to compete against the other streaming platforms you have to be modern you have to really also innovate but also you have to really use the same Technologies right now also here in this use case study F recommends the next video and as Andrea has mentioned already they do this already in a very well efficient way efficient in a way that they use rule-based systems for watching for example is serious right if you watch
a series well you likely want to watch the next episode when you finish the first one all right so you don't need AI for that but when it comes to for example let's say a movie a documentary whatever then actually they use their own models that they developed um and in this case Andreas already mentioned they're trying to be as data efficient as possible their only retrained incrementally uh they don't retrain all the time and of course by that also save cost and save energy however let's come to what you what else you can do for energy saving so as mentioned uh Saturday F developed a sequential model for this which is really highly adaptive to the user Behavior right so it uses the user interactions it uses the content and it's trained on this data right so it's a very specialized model now let's switch this out with a zero shot model so we did that and we used our system for this and we achieved a similar click rate although not have not
having seen the content before right so not having a knowledge about the user not really training on the users before and applying this completely zero shot and this is actually really impressive because this is a model that is generated and was trained on data that we actually just got from the internet right so we don't even know and of course we don't have just access to all of the settaf users right so we do this in a privacy preserving way we don't want to have this data and so this is a pretty impressive result but let's wait for the next result we actually measured also the energy consumption that's necessary and the computing power that's necessary to actually realize this use case and we saw that our system consumes 98 less energy than the sequential model that was trained from CTF so we think that this result speaks for itself and it should also aim at taking away the fear and we just talked about fear in the last talk right talked about fear of
applying this right and fear is not only about what can the irai do but we already see that companies fear the cost right because well AI where you don't know what it will cost you at the end right so we hear uh that a lot of these models are too big they're too complex Etc it will cost us Millions but actually we see here that we can actually really work on the algorithms themselves and bring just better results bring more resource efficient results and another remark to this I always hear that um all these kind of legal constraints in Europe all this kind of focusing on privacy right I mean we heard about gdpr Etc that this is like going against Innovation right and also another Point Europe doesn't have access to this large amount of data that for example China or the US has of course that's true but I really believe that actually this kind of constraint of not having enough data having all this regulation is actually Innovation friendly on the other
side as well because we have to really think about those systems we can't just like throw data on some simple decision trees and still produce good results we actually have to think about really deeply how can we produce better models how can we produce better systems it's also not only about the AI it's about the system behind it right and so we really believe that Europe can also go its own path and we can actually really develop own systems that are actually inside more clever because we just have to be more clever we have to work harder and I think that this actually can also Drive innovation so what are the next steps so we will go further and analyze the data we want to also change a few things and see whether we can do even better than this want to experiment with different hardware and algorithm setups and see what else we can do and we'll also prepare a paper and submit this and also want to of course see what other researchers
what the community thinks about these approaches thank you very much oh Andrea's life great successes you celebrate here from the energy consumption site we all know data is always in the center of everything and when we could exclude this privacy part I think we would grow so much faster and what I would be interested in is how do you see the user experiences with AI in 10 years oh well 10 years is really far away right yeah so currently we think the next big step for us is to fully automate and personalize the the media take of course there will always be uh points where the editors say okay there's an event going on for sports polls for example so we have that um but it that will be a conversational AI for example I really strongly written that or what would roll this on platforms will play so it's really hard to foresee I think with the next two or three years we see more personalization more Automation and we will see um of
course gender fa coming into play so for example there are also some use cases where you can use uh explainability of those of those generative models so I think yeah two or three years you have somebody there 10 years I'm really sorry yeah it's it's so far away right what do you think about AI chatbots or AI virtual avatars well I think I mean they just will belong to the Future right I mean of course people probably will just learn how to interact with them and of course we see like a big enthusiasm right now but uh I think it would will just become normal normality right so and um also maybe not so clean data what do you think yeah I think I think it's really interesting um to think about um the minimum amount of data we use for the use case at hand so I don't want to ask people for data when I kind of bring them back some value and it's really really an open complex question what the right
amount of data for our use cases is right now maybe in personalized medicine for example that's really another Point another discussion um but we have seen that we in most cases don't need this really highly individual persolation so I think I think looking for the minimum amount of data and the simplest algorithms is really way to go for us within that use case yeah have you thought about closing these gaps with AI also and I mean there is also the option to for cast maybe new data and the development of for example user experiences yes of course as synthetic data is really right now in in production we use that to get some distributions of specific data that also helps with some privacy issues of course you don't have to use data from the people outside there really interesting for us yeah that's that's quite normal I would say yes um and the other use cases um I'm not so sure about that yeah so we'll we'll move a bit slowly because you know we
are really in a constrained domain uh we have really a lot of Regulation and bringing that into production was really hard work so I think yeah we have to gather some feedback on that and try to explain to the people what we're doing yeah really interesting you want to mention anything yeah actually I want to say that um coming back Really to the cleverness and how you actually do this right I mean for example in our case um it's pretty interesting because we we just for privacy reasons also we just can't look at all the day um the users at the same time right and do cross cross user referencing and so on um which you can see of course in other platforms and uh two points that are important here first point um we also see uh studies where people actually don't really like this so much right it doesn't really help in the adoption so that for example let's say uh coming back to the mediatek just because the majority of people watches
football right now you recommend also football right and you maybe just because you look similar from your profile you get also some other recommendations from other users just because you did something right so looking at the users from an individual perspective and just recommending based on their interactions is actually also something that can help really in the reducing also the risk and reducing the fear of people and for example of course what arises then is the cold start problem right I mean you have to start somewhere and you have to really rapidly learn and this is of course where um for example what we do like with uh really kind of looking at the user like from an um mathematical view that you can really picture the user in the embedding space and really very very rapidly learn what the user might like and not like is something that actually can can help you also with the cold start problem and you just have to focus on solving this problem in a more clever way
right if you don't have the data at hand so one last sentence the future of user experiences is in one sentence be personalized I think okay and for you fully personalized of course okay so you're aligned right thank you thanks a lot