Rise of AI Conference 2022 · Berlin · Meta Stage

The New AI

Florian Dohmann · Birds on Mars

At Rise of AI 2022 in Berlin, Florian Dohmann, founder of the Berlin-based agency Birds on Mars, argues for perspectives on AI beyond automation and efficiency, aiming to make it social, creative and climate-friendly. He opens with an enterprise example, a project with Deutsche Bahn predicting parking availability near German train stations, to illustrate that most effort in applied machine learning goes into data preparation and governance rather than the model itself, while the most user-facing part delivers the most value. He then describes several long-running side projects: the 'Artificial Muse,' a five-year collaboration with painter Roman Lipsky in which an AI trained on his digitized paintings generates new visual inspiration; a project bringing AI-based creative tools to people with mental and physical disabilities; an album with the band Mouse on Mars built on a technique he calls 'vector hacking' to semantically manipulate a voice-synthesis model; and Q Trees, a Berlin project using satellite, sensor and public data to predict which city trees most urgently need water as the climate warms.

Key takeaways

  • In Birds on Mars's applied AI projects, Florian Dohmann says most effort goes into data preparation and governance, while the user-facing part of a system creates the most value.
  • The 'Artificial Muse,' a five-year collaboration with painter Roman Lipsky, trains an AI on his digitized paintings to generate new visual inspiration he paints from.
  • Birds on Mars developed a 'vector hacking' technique to semantically manipulate a voice-synthesis model's feature space for the band Mouse on Mars's album 'Anarchic Artificial Intelligence.'
  • A Birds on Mars project brings AI-supported creative tools for painting and music to people with mental and physical disabilities.
  • Q Trees combines satellite shadow data, public data and ground sensors to predict which trees in Berlin most urgently need water as the climate warms.

Detailed notes

Applied enterprise AI: the Deutsche Bahn parking example

Dohmann introduces Birds on Mars as a Berlin company working at the intersection of human creativity, artificial intelligence and organizational identity, drawing on staff with backgrounds in sociology and philosophy as well as computer science. He illustrates 'applied AI' with a recent project for Deutsche Bahn that predicts how full parking areas near German train stations are, combining external and internal data. He notes that projects often start with the newest neural network architectures but end up using logistic regression or decision trees, and that most effort goes into data preparation, governance and process design, which creates little value on its own, while the smaller, user-facing part of a project, its user experience, generates most of the actual value.

The Artificial Muse: AI as inspiration for a painter

He describes the 'Artificial Muse,' a project started more than five years ago with painter Roman Lipsky, in which an AI extracts features such as composition, brushstroke, color and contrast from Lipsky's digitized paintings and recombines them into new visual inspiration, a kind of 'kaleidophone.' Lipsky studies the AI's output and paints new work, which is fed back into the model, so the system evolved over years from a simple back-and-forth into an interactive tool Lipsky can work with in real time. The work has since been exhibited internationally, including at the National Museum of China.

AI in the social sector and with musicians

A related project, the Kaleidophone, brings AI-supported creative tools, painting and music, to people with mental and physical disabilities, an area Dohmann says is economically important but underpaid and rarely connected to AI development. Separately, Birds on Mars collaborated with the band Mouse on Mars on the album 'Anarchic Artificial Intelligence,' using an encoder-decoder network to turn language into a playable synthesizer for an individual voice; the team developed a technique they call 'vector hacking' to semantically manipulate the model's feature space rather than simply clone sounds. The resulting tool was later released as a public product used for audiobooks and university teaching, and combined live with Roman Lipsky's painting in performances, including one where Lipsky navigates his own latent space of paintings by drumming.

Q Trees: AI for climate action

Dohmann presents Q Trees, a project funded by a ministry with Berlin's CityLAB and the city's urban greenery service, which combines satellite data, such as calculated shadows, public data and ground sensors to predict which trees in Berlin most urgently need watering as average temperatures rise and trees increasingly dry out. He also mentions Kiki, a children's education project, framing all these initiatives as attempts to make AI more social, inclusive, sustainable and playful, arguing this requires drawing on many different human perspectives, not only technical ones.

Quotes

AI is just applied maths, we talk about vectors and mathematics and vector spaces and feature spaces and latent spaces.

Florian Dohmann

The fancy AI part is often really not the part where you invest most of the time.

Florian Dohmann

It's in our hands and it's about doing it.

Florian Dohmann

Questions answered in this talk

What is the Artificial Muse project by Birds on Mars?

According to Florian Dohmann at Rise of AI 2022, the Artificial Muse is a five-year collaboration with painter Roman Lipsky in which an AI trained on his digitized paintings extracts features like composition, brushstroke and color and recombines them into new visual inspiration that Lipsky then paints from, with new paintings fed back into the model over time.

How does Birds on Mars use AI to support people with disabilities?

According to Florian Dohmann at Rise of AI 2022, the Kaleidophone project brings AI-supported creative tools for painting and music to people with mental and physical disabilities, aiming to let everyone express and 'jam' together in their own individual way.

What is the Q Trees project?

According to Florian Dohmann at Rise of AI 2022, Q Trees is a Berlin project, funded by a ministry with CityLAB Berlin and the city's greenery service, that combines satellite data such as calculated shadows, public data and ground sensors to predict which city trees most urgently need water as temperatures rise.

Mentioned

Artificial Muse, vector hacking, latent space, encoder-decoder network, Kaleidophone, climate action, generative AI

Keep exploring

More from Florian Dohmann

More from Rise of AI Conference 2022

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 3,700 words

Show the full transcript

our next speaker is a computer scientist and an artist at the same time and he's also a consultant and founder of birds on math a company that is doing um ai as an ai agency basically and you are consulting your org please come up jonas jonas dorman you're consulting companies on the the interface between art and i will say a few words about that uh in a minute okay so your speech is called the new ai yeah great thanks welcome thanks to be here hello yeah so first of all it's very nice to be here what i learned the last two years that it's that i cannot take it for granted uh to have sessions like that it's very very cool to be here again in person and have a bit of the human intelligence part uh more yeah so um my topic is the new ai how we make ai not only creative but also social and climate friendly and just a few words about birds so as you um as you introduced yeah

so we are a berlin-based company um our claim is connecting intelligences we are working at the intersection of human creativity um artificial intelligence and organizational identity so we are mainly working for corporate corporates in the field of strategic work architecture we call it intelligence architecture and also development but that is not the topic of today and i want to talk about some new perspectives to ai and want to give some examples but before i do that this is an extract of my colleagues i find that very important when we talk about ai and that's why i show it and i think it's very valuable to have a very diverse perspective to ai so it's not only made for the math and computer science people but for people with a diverse background so in our case people also have a background in the field of sociology or philosophy but also combining it with with economic skills and math and computer science and i think it's a very very very important that we all together work on

the future of artificial intelligence and i want to start with one example coming more from the corporate field here called enterprise ai you can also say applied ai so really machine learning comes into action and we've heard many great examples also in the other talks i want to start with one one recent project that we just launched together with deutsche bahn i think it's a very very classical machine learning case you have a prediction in this case predicting how full are the parking slots close to any train station in germany and this is i think one i chose that example because i think it's a very very good one for applied enterprise artificial intelligence or applied machine learning you collect data from the outside you combine it with your with your internal data and then you train your models and based on that trained model you can predict something and very often as some of you know who are very working who are working in the field of ai you start with neural networks and

with uh with the fanciest newest architecture of neural networks and then in the end you come to logistic regression or decision trees and all that things but i think it's very important that is also part of applied machine learning and how i experience it in the practical field that the fancy ai part is often really not the part where you invest most of the time usually you need a lot of pre and pros post processing of the data you need a lot of requirements like the right people the right skills the the um you need to learn more you have to define structures like data governance data platforms new processes and all these things but what is interesting when you look at it from the outside this part here on the right maybe is not the most effort you take but you can get the most value out of it because on the left side you take the most time but you don't have any value if you just create a data governance in the

end you have to use that for really value and i think to create value and um i think it's very important this looks like because looks like means how the ai looks like so how people are using it so it has a lot to do with user experience design for example so to make the to to make the ai really working in the end and what i think and that is the bridge now to the to the topic is that um because there is this huge value also for the corporates not only for the art world or for the fancy creative people who have the hacker spaces and want to just do something i think for for the for the economic for the economy overall i think because of this here we need many many new perspectives on the topic of ai which is much more than simple automation and optimization and raising the efficiency and looking at the value chain and think okay we can optimize that with a machine learning model save time

save costs etc i believe that we need this perspectives and that's why i'm here i want to give you some examples today where i think that these are good ones to show by examples because the human intelligence learns best by examples and also the artificial intelligence as we learned in the previous talks the machine learning systems love examples and i think humans too i want to give you show you a few projects that people in my team and also me personally and other people have worked and are still working on to bring new perspectives to the field of ai we have in our company a small unit called sol and sol is the name of the mars day and the mars day has 40 minutes more than the earth day previously it was talked about the future of work i believe also in the future of work it's good to spend time on stuff that not at the first moment has a return on investment or something like that but to spend time and this

is what all of us do every day a minimum of 40 minutes on topics where there is no client at the beginning and there is no funding at the beginning but where we think that it's valuable to bring new perspectives to the field of ai and all that started with a project called the artificial muse a project that i personally started over five years ago with a painter roman lipsky you see him here and we created a ai that is able to extract different features of his digitalized paintings like the composition the brush stroke the color the contrast and all these things um also features that where we do not really know what it is um but in the end it's a kind of new generation collider phone you can think of it that's why we call it artificial muse it's about seeing ai as an inspiration and not an autonomous creative something and how it is done roman lipsky can bring his digital art pieces through the network and then has this kind of

very clever collider phone kaleidoscope i will come to the collider phone later kaleidoscope the artificial muse that brings new colors new shapes new forms but which is also related to his work and this this mix was for him very valuable he's now for over five years working with that international exhibited um still enjoying the work with the muse and that's also important of course in the end it's engineering so it's not an ai that you once create and then it's done you have to work on that and this is the lessons learned from my side we now work on that for over five years and it was not this just one thing we do is showcase a proof of concept or whatever it's really like continuous hard work and it's super valuable for for us we learn a lot and also for roman lipsky as one of the um the person the painters who did that first internationally exhibiting internationally and what i want to show there is that what was also important roman and

me or what was valuable is what i said in the beginning these different perspectives in this case very very explicit i'm i'm having an um business and also computer science background and roman a painter for his whole life and these two perspectives you see us both here during a walk in the forest was super valuable also for creating our artificial muse and it developed over time so it started with um yeah you can say a pamper and brief vexil um so roman was getting the the images the digital image images of the artificial muse he started to study it and then started to paint something new the new stuff which was painted became again part of the model and his own artificial muse model was growing and over time it developed and we have now different versions and now currently and i will show a few examples in a minute um he's able to interactively work with the system really like giving something in getting immediately something out and being in this flowing fluid dialogue

and yeah we as a team working on that learned a lot from that also in the beginning we wouldn't have thought how that would lead to for example we we exhibited in the national museum of china when we started five years ago in a small studio in berlin schunerberg we just wanted to start and wanted to find out how latest development in ai we can use for um for using ai as an inspiration and in between coming to another project as i said i want to give some examples it's one project called the collider phone so not the kaleidoscope the collider phone you find more info on kaleidophone dot ai that is a project where we collaborate with a very very nice organization called banners in hamburg they have people with mental and physical disabilities to work creatively so to paint or also to make music and people for example with down syndrome or other mental or physical illnesses or so-called illnesses and which is very interesting as we started the collaboration with them why

because i thought that the social sector is super important to include it also in the development of ai it's the biggest one of the biggest fields for economy also and where by the way people get less paid than in other fields but it's super important for all of us when we get older we profit from that and also our young children profit from that but currently you have the ai discussion on the one side and the social sector on the other side and i think that we need to find also there new connections of dots that are super interesting for all of us where we can all profit from and we started this project called collider phone we got some money from from a ministry won some prices got some basic funding but nothing where you can pay or pay your salaries with but just have some starting funding and yeah here you see a small video there's also an artifacts uh an artitrax video about this collaboration but here you see a jam session

where we jammed with the guys from the kaleido phone from on the right side and with katarina we are currently finding out together how she who can who can express differently which is just a different way of expressing yourself our vision is to have a tool that enables everyone to jam together independent if you speak fluently like i try currently at the moment or katarina who has her own individual way of talking and this is just it's it's a process and it will again take time but it's super valuable and i just want to say that it's also and want to invite also to share this journey of connecting the social sector with um with artificial intelligence and talking about music we did a project with a band mouse on master we are called birds on mars they are called mauson mass that's why we we were connected and mars on mars very famous legends in the music industry they they wanted to to do something with ai and they asked us and then we

did it and in the end we had an uh a release it's called anarchic artificial intelligence aai you can find on spotify very cool album and also please buy an lp of course not only spotify and itunes etc and we had a very again a super unique collaboration that is still going on and i show you a small extract of what that sounds like because i i will tell afterwards how it worked but first listen please engineering systems and mastering them with noise engineering systems and mastering them with noise engineering systems among the machines we know different desire is the precondition of knowledge by the way the visuals are created by casey reese the inventor of the processing programming language they have worked often together mouse on mars and casey reese but today it's not about the visuals the idea behind that album was we wanted to use language we heard about texts and language models in the previous talks we wanted to create an ai that makes it possible to use the language as

a synthesizer to make your own individual voice playable and that was something that was the basic idea then we started to create prototypes and wanted to find again out we wanted to find new perspectives what else can we do together in the field of ai with people who are open for that but who are not machine learners from their background and after the album was released we also created a product out of that it was it is available for everyone so we don't want to earn money with that but we want to inspire and show people what can we do with ai and then many other artists came and used it for example for audio books or for university teaching and many more and we wanted um we wanted to to make a scalable ai product out of that which worked and it's currently still available but we are now in a kind of pause mode and uh thinking of how could be a 2.0 version of this product would look like and the interesting

part in this when we when we have the only deep dive the only machine learning deep dive in that talk here when we look a bit under the hood here um it was an a network architecture that was published in a paper just recently the basic concept is definitely not new there's this so-called encoder decoder concept not today the details but as you already have heard in other talks in the end it's just applied math so ai's just applied maths we talk about vectors and mathematics and vector spaces and feature spaces and latent spaces when we talk about generative ai and what was here interesting was the first the initial part when we when we were in this lab mode and what we wanted to find is a kind of hacking this anarchy moment the anarchy moment moment of the ai we called it vector hacking so modifications of the feature space that cre that makes it possible to generate not cloned sounds so at um in cache ai you can bring in sound and

text and the model learns to translate text into sound that is already there there are open source projects you can you can focus them you can start to program with that but what we wanted to do is we wanted to manipulate it in a way so semantically being able to semantically change your voice deep into your own voice and try to do something with it and we've we've found a way to do it and uh yeah the result was then used by mouse and must for the album please just listen to it then you then you hear what comes out of that and then we also combine two worlds and that is another impulse i want to give to break barriers uh with artificial intelligence so then we used krach with um with a portrait or a music a musician based in l.a and berlin and roman lipsky the painter and we did our first live gigs combining the music and the painting at one time to explore how it is possible to to dive

into this intersections the plus between ai and humans but also between music and painting in this case and we did some live sessions again and it was a lot of fun in this case it was in karlsruhe an intermediate jam session um and now the latest version is also funny we call it drum and space faber in veltron and the drum in space is now the possibility for roman lipsky that he can drum through his own latent space which is uh super funny roman the the initial point was roman he has a very huge atelier in kreuzberg i visited him there and roman told to me we are friends now after five years and a lot of beers and wines and a lot of creation and by the way there are many colleagues in my company also involved to make that all that happen it's definitely not just me i'm standing only here now representing all of them in a way um roman said i want to play drums i want to learn play drums

and this was the starting point then we thought if he wants to learn playing drums it would be very cool that he can drum through his own painting world and you can see it it's now possible you have this latent space in the we have a gpu server in his studio and the gpu server gets as an input the sound and is then able in real time to navigate through his own individual latent space and he can with drumming explore the different painting worlds and then find something interesting at a time and then this is a starting point for painting again this was now a bit about painting about art about music about the social sector one other thing which is for me very important when we talk about new perspectives and the new new in ai is among other sustainability and in this case climate action how it is called in the sdgs it's very cool not just climate or climate change it's called climate action and climate action is a starting point also

for a project that we are currently working funded by the ministry together with the city lab the technology shifting berlin and also the greenfleet of berlin and the starting point was that it's called q trees q trees dot ai is that the climate change does not only affect our world but also our local areas as you see it here it's the average temperature over the last years in berlin brandenburg and what we thought is how can we use ai to do something here and we started that project called q trees and the basic idea behind q trees is to quantify trees and quantify trees from the soil to the cloud using sensor data but also public data we are currently for example calculating the shadows in berlin based on satellite data because the shadows are very interesting features for the need of water of trees because the problem is that most of the trees dry and die sooner or later it's a very very hard problem in most of the cities and also in berlin

and currently you don't have a prioritization a good prioritization of which tree needs water most and we want we are currently working on training models using all that data and also calibrating that models by using sensor data to be able to predict which trees need water most and so it's not also about climate just one example it's about the the future of our next generation i believe it's very important to include also the kids we for example created with the dia kuni together a box called kiki for kinder and the kids started to love kiki kiki and i think it's just the beginning of all these things and all of us all the guys who are part of that should participate there and use it also in the schools and also before the schools and when you talk about ai at work include the children at home i think it's a lot about not only kids but gaming in general so find playful funny interesting ways of interacting with ai to make something good out

of that so hopefully ai will be social inspiring inclusive child-friendly open sustainable and much more it's in our hands and it's about doing it and by the way a lot of fun to connect these distant points of ai and create something really new and cool is it requires a lot of human creativity and intelligence and many of them to create ai how we want ai to be like and i think that is the most important part how we want ai to be like and the more perspectives we have the more potential we can realize thanks

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.