I so Welcome to my talk totally psyched to be here especially since this this stage already had and will have talks about how we might need to integrate knowledge or know-how from nature from living systems into into the artificial systems that we want to build in future. So I Yeah as I or somebody else said I'm Tim. I am professor of artificial and collective intelligence at Free University of Berlin. So just south of here come by visit me. I'll show you my lab and what we usually do is we look at biological model systems. So could be fish could be bees much simpler than humans but that is for a reason. So usually we would invest time and money and and thoughts into building like robotic interfaces into animal societies.
That's the collective part and sometimes we would use you know modern tools machine learning and robotics to learn about individual intelligence and today I'm going to focus a little bit more on on bees because they're so cute. And but they're also super cool in terms of what they can do and the way how they do it. So those guys are little smart robots themselves in a way. They live 6 to 7 weeks in in summer and they work their asses off their abdomens off and and you know, fly I over several kilometers of distance, collect food, and bring it back. And so, they're they're super robust. So, this is something we could learn. Talk will not focus on this, but they're also super smart.
So, the cognitive abilities that they show are remarkable, and this is something that we go after. So, my idea or my thesis here would be in order to build good and better AI, we need to understand the BI, the biological intelligence. So, B comes before A. All right. Good. So, as I said, so let's focus on what they can do. Just a short introduction to what bees can do. If you put yourself out there and watch a flower, you see a bee. This bee might be coming from very far away, right? So, they have a radius of 6 km. So, if you're standing in in the center of Berlin, you might find a bee that's coming from I don't know, Weissensee or Pankow, all right?
And if you imagine yourself coming from there, having I don't know, you know, walked the city, taken a stroll, I don't know, explored, maybe find a restaurant, would you be able like in a in a new city, in a in an unknown city, would you be able to find back home? I don't know. With I mean, without mobile phones and stuff. So, this is remarkable, right? So, they they can fly, find stuff, find back, and then find this place again and collect stuff. So, this is not like knowledge that they have, you know, inherited genetically. It's what they learn, right? The brain learns it. And that's cool, because the brain is much smaller than this dot.
I had to increase the size of the dot so you can see it in the last row there. It's super tiny. It's this. It's less than 1 cubic millimeter. So, in there, you got all the you know, motor control, sensory processing, and all the learning and everything crammed in this tiny uh little gooey material there. And even if we just compare two brains, you know, not not you know, leave out all the artificial systems that we have right now. In terms of energy and space efficiency, this is remarkable even compared to us. Okay. So, I hope hopefully made the point. Uh you be you believe me uh uh individually uh bees are super cool, super smart.
But, what they also do is like communicate to each other and solve stuff collectively. So, what we have is a network of many many nodes connecting through many different types of communication channels, just like like we do, right? So, we take our phone and call our friends or meet uh I don't know, with other friends and watch Game of Thrones. And uh so, we'd have different types of channels conveying different types and also amounts of information. Um and this multi-channel network um is something what that of course we have and we would like to understand, but we can't really under I mean I mean I can't understand because I don't have the data because Facebook doesn't give me their data or Google or all those big players.
Um and there are a couple of reasons why bees are even uh adding value to this analysis, but I'm going to going to show you uh this in a in the next slide. This is the bee dance. This is one specific form of communication that relates intelligence. So, what they learn in the field to the collective part of the intelligence. So, they communicate the direction and distance to their nestmates where to those then fly and find the target, right? So, this is one specific channel in the many many uh channels that they have. All right. Um So, and in order to understand the biological intelligence of a model organism. We not we just not uh do this for the sake of understanding bees.
We do that of course and I hope you understand this uh you know, to understand uh uh ourselves. The way brains work, the the way brains learn, represent information of the world. Uh the way we together as a society, as a network of individual uh communicators uh process information, right? So, if we knew how that worked, of course we could predict uh decisions in elections, for example, right? So, they they have something that is very similar to elections when they find uh try to find a new nest site. So, the the whole dynamics within the network are interesting. Uh there are no tools to look I mean, not yet, I guess, in uh in in the complexity that we would like to have.
Um and that's why we take bees and we we um we take bees for several reasons, not just because they're small and we can have it in the lab and not just because we don't have the data in humans because uh this might seem for you the on the only reason why we look at bees. No, there's several reasons we we look at bees because they give us more value. First of all, we can track all of them over their entire life. From the day they emerge from the from the brood comb until the day they die. That's six or seven weeks. That's well doable in one year. Uh overlapping generations, right? We could look at life cycles.
We could look at how they evolve, what is the average path through life, and what is with those people that don't follow the average path in life. And those questions. Uh so, look at those matrices. Those are interaction matrices. So, we look at who is interacting with whom. So, the number, so one of those dots in there tells you how many times or how long they they interact in a certain period of time, maybe 15 minutes or so. Uh and then we have those matrices, many many of those matrices, millions of those matrices over time, and that's the data that we would like to have in humans. We don't have it. Uh I mean, at least not my group.
And then, so one of the things that we did with machine learning, other than, you know, tracking and and and all those things that that I just showed you, is uh to reduce the dimen- dimensionality of the life of a bee, because this is 4 million points per matrix. Uh so, 4 million times a couple of million time points, that's huge. We have to reduce this. Um and um what you see is the circle of life of bees. I I I suspect that the circle of life of humans would look a little more complex, but still, I think the methods will be the same. Um What else? Well, those are feeding sites. So, those locations in the world are those places where bees make um um experiences that define how they act in the future.
So, whatever they en- encounter there at food sources, be it a uh a bee eating bird, which is bad, or a value valuable like sugar-rich uh nectar, that's good. That will define their future behavior, right? So, whatever we do uh in the way we are rewarded or punished defines our future behavior. So, whatever we detect and measure there informs our analysis later. And so, I hope you always have this parallel stream of of uh uh you know, the the human meta- um metaphor in your head. So, this needs to be done for humans to understand how we work. Um and we do this because we we we would like to understand this in humans, too.
So, um and another thing that that you cannot do with humans is uh putting them on a copter um and poke electrodes in their brain. And this is what we do with bees. Uh so, you know, the analogy would be poking electrodes in your brain or in my brain and then, you know, look for for a couple years uh whatever the brain says uh you know, when we interact, when we navigate, when we do stuff in in the real world. And um so, there's several reasons why we have to resort to bees and not to use humans, some practical, some uh ethical reasons. And um so, I I hope I made the point why we use bees.
Um okay, let's move forward. Um And like the nature of what we do is typically just applying whatever tools the machine learning/AI/whatever community provides. And but sometimes we reach uh the limits of what's possible with those tools. And one of those those limits uh is is uh what we did here. So, the task in this picture is find that the the markers that we glue on the backs of all the bees and then decode them. So, the first task is pretty simple. Uh you train a machine learning model uh convolutional uh neural network uh on data that you clicked. So, you you press the mouse uh uh you know, uh whenever it's in the center of one of those tags and then uh you have a huge data set of many instances of of a tag in this and this and this orientation, but it looks like that and then you can train a model and then you can locate them or detect and locate them.
But what about the decoding part? Well, uh we have four numbers that we'd like to extract. Three numbers are the rotation angles in space, and the fourth number is the 12-bit ID that they all carry on their backs because we need to know who they are, right? And so, how would you do that? Label this kind of data? Well, we tried that. We we fitted like a 3D model on top of this image and then click those bits. And it took like, I don't know, 2 minutes per tag, and I estimated this like to come up with a reasonably, you know, large data set would take us more than 6 person months, right? I would have to tell poor student to invest 6 months of his life just to give us the data.
And once we would change the labels or the sorry, the the design of the tag, he would be or she should do that again and again and again. No, that can't be true. So, that's the limit of deep learning, right? Once the the labels get more and more complex, it's more time you have to invest to get the labels, but also the the problem is more complex. So, you need a larger data set to describe the problem. So, that's that sucks, man. That's not cool. So, what should we do? Well, um so, we didn't invest the time to label the data. We invested double the time to come up with a solution that that's cool and that's this.
So, first of all, location or detection of those markers and and localizing those markers is done in a in the traditional way. And the second part is done by a neural network, too, but it is it's not trained by human labeled data data, it's trained by with generated data. So, synthetic data that just looks realistic. And so, there there is a solution called generative adversarial networks that work like this. You have two networks. One is generating hopefully realistic data, and one is saying, "Oh, that's that's fake." Uh or that's real. Uh and you have a data set full of samples that that are real, but they're not labeled. And so, uh in this um adversarial training that's called, um you know, the generator and the discriminator get better and better and better like faking, I don't know, uh money and the police uh finding out what, you know, how how this fake money looks like and adapting, you know.
Um we can we uh come we come up with realistically looking uh B-marker images, but the problem is and the community hasn't solved that problem yet and we kind of did for our problem is how do you control the labels? Because you can learn to generate nicely looking faces and all those things, but you have no control over the semantics of the uh of the underlying object. So, we came up with something that we call RenderGAN and that's Leon and I hope he's here. I haven't reached him. He he was supposed to be here. So, whenever you see this guy, uh uh you know, call him uh and and talk to him. Um so, the solution Leon proposed was to learn uh data augmentations from from the from the data.
So, all those image characteristics that uh imaging processes have, blur and reflections and I don't know, compression artifacts can be learned um in a pretty pretty general idea. And the cool thing is the community said, "Oh, that's cool. Although, it's like a pretty niche-y kind of problem, but they still uh you know, um cite this paper uh which I love because it wasn't really planned." And now, uh we we uh ported the the idea to faces, which is a pretty similar problem. Um as you can imagine, all 3D objects have at least those three angles in space you want to model and then the I the idea of the faces, all, you know, all those um things in there.
Um anyway, so this is something that wasn't planned in the in the beginning and as I will come out, I mean, it's not published yet. Please don't steal the idea. Um It's working quite nicely on faces. Other data sets are not as nice. But anyway, that's that's something that we should go go further into. Um So, let me let me oops let me um Oh, yeah. Let me go to a second example of how basic science that started going in this direction came up with something that's valuable for others. Um This example is a project in which we as I said before poke electrodes in the head of of bees which is totally allowed. Um and then we look at the activity of very specific neurons in very specific parts of the of the brain.
Um and um then we well we fly them over a field because we don't want to simulate the the 3D environment with all its, you know, features that the bee needs to think she's out there. We just use the real world and had to to um like miniaturize the the the lab environment to be able to fit it on a copter. And so and and so the the funding agencies said, "Well, they you're crazy. That it will never work." I mean, not not the funding agency but the reviewers said. So, it took us a quite a while to get the get the money and even though most of you I guess are not from within academia, you can relate to that, right?
So, you have a crazy idea but you have problems finding the money which is pretty common in I think Germany, I think. We're not that risk-taking kind of species here. Anyway, so that that that that was something we had to cope with. Once we found the money, which I'm glad I can cite the Klaus Tschira Stiftung for. It was possible to to to and I'm not saying I mean we were super smart to do actually be the only ones to to to to do this. I think anyone not anyone's there are many groups that could have done that. But we did it. So we we fly over the field, we record noisy data. It is noisy, yes, but there are ways to cope with that and then we come up with beautiful spikes of single individual neurons, which is great.
And this is by the way the the lab that we miniaturized. So this is typically like a rack standing like this. So that's cool. Um and I'm going to conclude. So is this the time that's yeah, that's running down? That's okay. All right. All right. Okay. Awesome. Thanks. Thanks. Yeah. Um Okay, I'm going to conclude with an example of research that I have problem I still have problems finding funding for. And so maybe you know someone who knows someone and then you know, but it's still related somehow to bees and that's why it it it fits in here. So think of self-driving cars. Everybody's talking about them. I can't see them. But I hope they will be there soon because then everything gets so much better.
Uh Um except if they're all electric, we have a problem because we have to charge them and battery technology hasn't really made leaps like we have seen you know, self-driving autonomy and stuff. Um we still have to plug it in. Or I mean, we could use induction charging, but still we have to, you know, wait. We have to wait until the car is charged. Takes uh even with supercharging 20 minutes or so to get a decent amount of energy, and it still degrades the battery, so you have to throw the thing away after 2 years or so. So, not friendly for the for the environment, but still a problem because the automobile isn't mobile anymore.
So, how do bees solve the problem? Well, um we look at foragers. Yeah, we look at foragers coming back from a food search trip. She didn't find anything, is very hungry. She doesn't go into the the colony to the to where the honey is stored. No, she asks a peer, "I'm hungry, give me something." And the the peer uh throws up. I know. Uh regurgitates um a drop of nectar it from her social stomach. It's called social stomach. It's just there It's It's the social stomach. It's just there to give others food. How crazy is that? And the other hungry previously hungry bee can drink the nectar and can fly off and take, you know, on her work.
And this is a tiny optimization in evolution, right? So, it just saves maybe a minute or so per per forager. But that minute was valuable, so this design with the social stomach and the vomiting and stuff worked out. They all have it, and they all do it. And so, why don't we do it? And so, think of you in 10 years or so, hopefully, you get into your car. Ah, damn, forgot to plug the cable in. Um but then your car still your car tells you, "Wait. Don't worry. Don't worry. I'll get you. Where Where do you want to go?" "To the office, please." "Ah, you're short of energy, but don't worry. I'll have I'll have you back."
So, the system, before even, you know, starting the car knows where it goes it should go, right? So, this is where I don't know, maybe where's the pointer here? This is it doesn't work on the thing. This is where you start, this is where you want to go and the system knows exactly when you will be at a certain point on that route and can find another participant that meets you at those points and time. It doesn't even have to be a long stretch like this, it could be just 20 seconds or a minute or so two participants meet on the road. One who has the energy and throws up and the other one that needs to drink that nectar.
All right? And of course we are electric so we need kind of different form of throwing up which is energy transfer via induction. There are technologies available. I have to resort to model cars as I have told you I do have to take models always to to show stuff cuz we're we're uh Anyone knows MacGyver, the TV series? That this is what this is what we do. Um so so yeah, you have autonomous cars, self-driving cars. They have sensors, they have they have huge compute power. They can do, you know, the this this road train thingy and then we we just bought like coils from Alibaba from China. Two days later we had we had this thing running.
So, it's it's doable in in the model and our simulations show it's quite doable in in the real world. And uh my time is up so let me conclude with this. Let's learn a little bit from the bees. 10 more Well, maybe for questions. Anyway, I'll just talk and then it's over at some point. Um So, let's let's like wrap it up. Okay. Bees, okay, they're smart, okay. We can look into their brains and use robots and stuff, okay, to learn about them. Um that's cool. Learn about the the social network, that's cool, too. Some things may come out if you risk giving us the money, we will give back to community stuff that we haven't planned to do.
Well, we do it anyway. So, we we uh so, that's that's the basic science is good part, okay? And there will be applications coming out even if we don't plan for them. Promise. Pinky promise. Second, uh funding agencies, especially in in academia, need to give us more money, man. Um, it's because the risk is in the public sector, and that's good for companies spinning out of those those labs. That's good, too. Uh right? We don't want to have all the risk in in those startups. Um that's the second point. And this the third point is we can just look at nature. It's just uh you know, one of those many examples. Uh and that's an old story, you all know that.
We could just look at it and be open to it and then try to implement it. And then, you know, everything comes together, risk taking and funding and so on to do this in the real world. And this may even lead to us, you know, uh transforming our perception of human humankind and humanity from this to this. Right? So, now we see the value in the others. I I don't feel that way all the time, I have to admit, but uh you know, this this this charging project might contribute to a little bit to this. Um and I think that was all I had to uh convey to you. I thank you for your attention, and I'm here for your uh questions.
Thank you. Don't run away. I have a mic. Don't run away. Ask him some questions, and you did not realize he made a lot of jokes. None of you were ever laughing. Yeah, but there was a lot of jokes in there. Like as an example I'm so selfish with with my jokes. Yeah, yeah, but uh maybe they're not good. I can train you on jokes, yeah. I think your jokes were very funny. Thank you. You can laugh inwards. Yeah, it's like in university when the lectures and stuff. Yeah. Um they're funny. Yeah, yeah. Very Leonard. Ah. Very funny dude. Yeah. Whatever. Um okay, so a question um to what's the social stomach? Is it possible that uh the social social stomach was initiated to prevent uh bees that are very hungry from overeating?
Or is it proven that they just save time? I would love to give you a long answer. I can't. I just don't know. I mean, how how do you know what was the actual reason? You don't know. You just You just see it's there and then you can interpret you know, it by kind of thinking about the life of a bee million years ago. Uh I don't know. Might have several other reasons than to just feed hungry bees. Yeah. Okay, thank you much uh very much. There you go. Careful. Well, first of all, thanks a lot for the say such a exciting presentation. So, you are analyzing a collective intelligence. And my impression is that in the case of insects a part of it involves a transformation of the environment.
So, I'm thinking of chemotaxis. So, how um the the the bees interact over time by by leaving chemical traces. So, my question is what can we learn from uh this kind of complex interactions? Uh so, first of all, I I I I don't know if I understood your comment correctly. The first part I did and thank you for that. But the other part I don't know. If you If you're referring to how ants you know um build up traces of pheromones so the others can find the path, it's not working over several kilometers. Uh and especially not in the air. Um So the way they navigate uh is to a large part uh through uh vision.
And I don't know what else. Nobody knows. the colony. And within the colony. So my point is I I I guess twofold. One is uh the the social network that they that they form is very complex. Uh it shares structural uh properties with our social network. Just look at us. We're pretty close. Um and it's not that every bee standing close to another bee is communicating over all possible channels. No, it's it's very There's a lot of structure in there. And we can use this model system to learn about or to actually develop methods enabling us to look at our networks. Um That That was my point and this is what I actually do right now.
So we use all the machine learning and AI techniques that we have, computer vision, uh and and all those things to get the data. And it's like many many levels of data like the the images, the detections, the decodings, the tracks, the behavior, and the interaction networks. So that it's it's all uh a huge pile of data. But I think it's worth doing it because bees are perfect model as I said. You know, think of uh farming with a thousand pigs. You cannot do this kind of research with a thousand pigs. You will never get a permit. It's limited to eight or 10 or so pigs. Right? So, if you want to understand how diseases spread, need a model.
And what better model is there as the beast? Right? I hope this kind of Okay.