hi everyone I'm Sean Moore the CEO of true face true face is a computer vision company on the United States that focuses on pattern recognition in real-time video feeds so real-time video analysis and stills I'm here today to talk about facial recognition and the intelligent decisions that we can make based on this information it's important first to understand what facial recognition is it's a mathematical representation of a face meaning that everyone's face has a different representation based on that different vendor so for us for true face it's 512 different characteristics of the proportions of landmarks on your face that will be different across every single vendor in the space and facial recognition actually was brought up on about 1960 gabber the name of Woodrow Wilson Bledsoe decided he was going to hand-draw proportion analysis with side by side side by side faces so a pretty impressive thing to do in 1960 wasn't until 1991 where we actually started using computers to run these proportion analysis and then in 2002 the Super Bowl in Tampa Florida
was the first real commercial use of facial recognition by the United States this was something that turned out to be not such a good idea they were looking for potential criminals and ended up having quite a few false positives in which the city was forced to shut the program down after about three months after the Super Bowl its surfaced then again in 2010 when Facebook started researching if they could use facial recognition to auto tag photos as I'm sure many of you know now that they do this and they do this very well on 2014 they said that they could do it 97.6% which was the same as the human eye could could recognize and then in 2015 Google came out and said they've had a hundred percent accuracy and for anyone that knows what happened in 2015 and I'll get to it in a couple slides here Google was very mistaken and you know with those accuracy numbers you better be sure that you have a hundred percent on 2017 the iPhone X came
out and this was not the first handset that used facial recognition but this was the first handset that turned facial recognition into a consumer technology and the reason 20/20 is up there because there are 650 million cameras in the world right now are cameras on street corner cameras on computers cameras on phones in the next year year-and-a-half there will be 1 billion cameras which means that there's an incredible amount of infrastructure out there but the one thing that they lack is scalable architecture to ingest that video footage and to make sense of it so this just gives you an example of the transition of facial recognition from 1960 to now I was literally hand drawings in 1960 on the 1990s it was running on a computer probably something like a 56k then into a DSL so extremely slow but able to make these kept these very advanced calculations and now we're at a point in time where we can take facial recognition which is something that typically runs at 400 to 500 megabytes in size
shrink it down to 5 megabytes in size and get it to run at the edge locally on infrastructure like a Raspberry Pi or other development boards where we're seeing this used right now is in smart home products for video doorbells to recognize people as they approach and that runs entirely local on that unit which is a very very impressive thing this is a very recent example I was in Munich this past fall for Oktoberfest and they've got hundreds of cameras that watch over this event and I started asking around I started poking questions with the the individuals from the security team there about why they were not using facial recognition and they told me well we've got it covered we've got 27 super recognizers so 27 people sitting in a facility monitoring camera feeds and I did some more research and found out that six million people pastors pastor this event so how is that possible how do you have 27 people monitoring six million people it just is not effective and it's not efficient
so there are restrictions around facial recognition right now at that event but I think that next year or the following year you'll start to see this technology in operation because it's a far more effective way to identify these potential threats this is an example of what that could look like this is a video from our technology this is actually anonymous facial recognition so we're not storing any personal information we have a list of individuals for potential threats in a certain environment we do not have their names we don't have their ages their birthdays or social security numbers any information about them other than what their face looks like so this is an anonymized way in which you can run facial recognition thereby avoiding the privacy issue that people have when they assume that you're collecting information and storing it with their face this consistent of the public concerns of facial recognition the most important one is the personal information how is it being stored where is it being stored who has access to it are
you selling it and given the recent events with Facebook I think this is on a lot more people's minds and it ever was before it's a very very big issue when it comes to biometric storage and biometric information which leads into privacy infringement is being watched in a city or in a private environment concerned and infringement of your on your privacy and one of the things that I want you to contemplate is you as a person on the right to your face that is your face that is your biometric identification tool I is a CEO of a company that runs a facial recognition algorithm have a discreet way of identifying those metrics on your face so does my math that I take analyze your face the transition from your face to my database is technically now mine and so that's a very large moral question that we're currently undergoing with regulation in the United States is at the end of the day who owns that data because that representation is actually mine it has no
identifiable characteristics to you you cannot reverse engineer those numbers you can't take those numbers and apply it to Amazon's recognition those numbers are very specific to that vendor and so this privacy and fringe macness personal information are becoming a very very big topic in facial recognition and then you get into mass surveillance and racial bias are we under mass surveillance there are sort there are some countries that are under mass surveillance yes but for the most part the world is not under mass surveillance what it's under is a inability to detect threats in real time so it's feeding in information that our governments have of known people of interest and using that to identify if there's a potential threat at a stadium if there's a potential threat at an airport in you know I do not think that we will get into a mass surveillance state what I know about the United States government is the organization is lacking and that would be very difficult for them to do let alone know facial-recognition company that
I know in the United States right now or even in Europe would allow that to happen so you know I'm not as concerned about the mass surveillance that I am about bias or personal privacy or the privacy infringement and then into the bias so in 2015 this was the Google example that I mentioned they had claimed they had a hundred percent facematch accuracy and face matches a one to one face match it's how you unlock your phone so it's taking one template and matching it with that person in real time to see if that matches well Google got into a lot of trouble with gorillas being recognized as individuals if you don't know about this event it's worth looking up it's it's humorous but it's very sad and so Google stops sawing that software shortly after they claimed they had a fix it wasn't fixed and they got into quite a bit of trouble for it MIT Media labs came out of the study in 2018 saying that facial recognition should not be used by
governments because there is a bias in it and that has a propensity to then target specific ethnicities or people of color people that are not white Caucasian males and the reason being is that the Prada the publicly available datasets that some of these companies were training their algorithms on we're skewed 70 percent 80 percent white male so if your data in is skewed or bias your data out is also going to be skewed and bias and that leads us into 2019 which is where we are today you have a magnitude of open-source datasets they're all bias and so it's on that company to go out find and source proportionately balanced data so that you can provide an algorithm that does not have this data bias if we continue to use public data that's going to be the consequence one of the ways in which we do this is a new advancement I think I heard someone earlier talk about it in here it's called generative generative adversarial networks so when you create these datasets or
when you create datasets in general you can take them from three places you can take it from the public like I mentioned there's IMDB out there which has a public data set of all celebrities and movies you've got you know a variety of other I think Microsoft and IBM both have the data sets that are available to download and then you've got private data sets which you go to clients to get clients provide that data back to you you can train algorithms to make them specific to that use case or you can go out and collect your own and the third is the Gans the Nexus of all these things or the center of all these things is how you get your performance it's how you get your speed your accuracy in the reduction in bias so I'd like to ask all of you how many of you guys are how many people here think that one or two images are faking this a show of hands would be great how many think three or
four are fake how many think all are fake all are fake it's a pretty impressive thing to know that all these images are not real people not a single person here exists and this is the power of what again is this is an example of again working you basically take a ton of data a ton of faces you feed it into an algorithm this algorithm strips out those proportions and then recreates a face from those proportions so it's a an extremely impressive way of generating data not always an impressive its scalable and you can customize that data set so if you want to if you say I need people that are sixteen to twenty two years in age that look this way you know with a mustache and long hair this will output that so you have now a way to create the data that you need for that specific use case at scale for a very low cost and you eliminate the bias so what does this mean this means that facial recognition is
coming you know it's here right now but it's coming based on edge computing power the advancements and these cameras that are being deployed a 3d computing and just really taking what was traditionally a very very heavy or heavy footprint a very heavy box and allowing it to run in real time on this infrastructure that with custom data collection allows this technology to work in real time across the world and provide the feedback necessary to make decisions those decisions could be for access control that could be for threat detection that could be convenience and we'll get into a couple more of those now so specifically for travel the efficiency that is can bring to travel I'm sure many of you have been through an airport you're starting to see facial recognition for baggage drop for entry points into gates I was recently at LAX and flying to Heathrow I didn't have to show a passport or a boarding pass to enter my plane and what they have found is that a typical seven a trip typical
triple7 from LAX to Heathrow takes about 50 minutes to board 5-0 it takes 25 minutes to board with facial recognition it's an extremely impressive number there to reduce the travel time and frustration from 50 to 25 minutes the safety in schools and in stadiums in the United States we're having a very very big problem with schools as I'm sure some of you have heard the news there are many threats at schools these days from children bringing guns to schools to people that should not be there going into these schools so a facial recognition is a way to ensure that we have the proper people in those schools that the teachers are there that the students are there but that no one is unauthorized hasn't entered that building and then a fraud reduction the banks all over the world are heavily investing in facial recognition technology to reduce to reduce fraud that's called digital fraud that's ATM fraud that's desktop fraud and that's in retail fraud and then convenience I mean this is a no-brainer if
we could eliminate passwords and have facial recognition be used as your password for everything would that not make life a ton simpler and the question that's always asked here is well what if your path what if your your face or your identity and a facial recognition algorithm is compromised do I lose my face can anyone just go take that the short answer is that's not even close to how this works as I said earlier a facial recognition company creates a template of your face a mathematical representation of your face if that specific mathematical representation is compromised they delete that file they put it in the trash like they would on a computer and their reissue one it's going to be a new string of 512 different digits so it's as simple as just deleting that password and then security and automation these these go one and one here security being things like home protection knowing who's coming to your house we're seeing you know smart doorbells have been around now since about 2012 2013 they're
starting to deliver facial recognition as a way to gain access but also a way to prevent people from entering their home when they're not there what I mean by that is 80% of home invasion in the United States that burglar or robber or that person trying to attack actually knocks or rings the doorbell first to see if anyone's home and so there are innovative companies in the u.s. right now that are putting speakers just inside the door the other play music to make it sound like a dog is barking to say hello in a human voice it's a way to try and deter that that potential robber that potential threat from coming in and then automation is access control we work with clients now who have thousands of people entering a building every day and that takes roughly 45 minutes in the morning rush to enter with facial recognition we think we can get that down to about 15 minutes so it's it's a significant way to improve the efficiency of these buildings or of
this access control this is an example of that that Airport what I was talking about you simply walk up look at the camera and within three seconds you board the plane they're taking that information from when you registered for that flight to have your passport information on your on file and they have your passport picture so it's a very very simple process safety I mentioned this earlier it's the schools we have a shooting roughly every four or five days in the United States in a school so there are about 100 gun incidences in 2018 it's a staggering number it's a number that has to change for us but this is largely in part due to people that should not be on that campus at that time and people that gain access to that campus you know after hours they scope the campus out and so if we had this technology running on these schools we could you know effectively deter and eliminate these types of events from happening and the reason that we can do
this is you know there's a privacy risk around schools and recognizing children but as I mentioned earlier we can anonymize that identification and we can actually blur technology our blur faces in real time so if we know they're not a threat we make that data fleeting meaning it's eliminated before it's processed and it's not stored anywhere so we are not collecting biometrics of these students it's only if there is a threat from that person that we previously know about and then fraud reduction we're seeing this at ATMs all around the world now now imagine walking up to an ATM looking at it punching in your code it recognizing you and you being able to pull money out and walk away it's a significant reduction in time that you spend at the ATM but it's also a very very protected and secure way of accessing your account one in which we have trouble with today this would eliminate fraud from ATMs that would eliminate people using your credit card online that doesn't that doesn't have access
to it and so it's it's a really powerful way for banks to are reducing that shrinkage that they have from fraud and then convenience as I mentioned earlier this is the the password piece if you're able to log into all of your devices walk into a bank and you you use the same credential to gain access to everything it just makes our lives that much easier if you could walk into a store and pay with your face and walk out like Amazon go how many people would go shopping that way it makes things significantly easier in your life and it's more secure than the current the current form of payments then security so this is an actual image from a home doorbell this is identifying the person that's there so it's a known visitor we know who it is you can play audio back to them if you want to say hi welcome you know the door's unlocked so think about Airbnb or home sharing opportunities you can just pass this credential through that platform
and then they have access to your house you don't have to meet them there with the key or have someone set up with a key so it's a great way to verify who's visiting your house or if you've got a package left identifying that the package has been there for you know 10 minutes 15 minutes or if that package is taking you'll be able to see and detect who that package is taken by access control as I mentioned earlier it's a pretty simple thing to wrap your head around it's a way of authenticating you you know either single factor or multi factor so a lot of construction sites are using facial recognition now for their workers they've got hundreds of workers coming there on a daily basis they badge with one hand they look at the camera and the door opens of the badges to run a one-to-one match instead of a one to end match so as I mentioned earlier the face matches one-to-one it's how you unlock your phone one to n means
that we're looking for people in a crowd or we're looking for people amongst many so it's one face amongst n so a thousand ten thousand one million to give you an example the speed in which this technology works my company specifically can scan through a billion faces in one second to find an individual so with that power of computing this become becomes real-time decision-making the future of facial recognition as I see it right now 3d recognition will become very apparent in our lives the issue a 3d recognition is a sensors out there are all 2d at this stage so you've got six hundred and fifty million cameras with a billion being deployed the vast majority are 2d and so until infrastructure changes on a scale this won't be that meaningful you know I would say roughly five to ten years before we have infrastructure overhaul real time personal experiences is something I'm very excited about and we're working on it's probably about five years out but you can imagine an environment where you walk into
a store you walk into a mall and you're receiving personalized ads in real time based on things that you've shopped for this would obviously be an opt-in use case because you can't just run this technology on anyone and so having an opt-in approach is is something that's very important to one of those earlier points of privacy and infringement on on your rights so opting in is something like where you've got your Instagram feed you've got all of your friends pictures and you see an ad that's very relevant to you this is what we're working on with retailers right now so it's to have extremely hyper relative ads in those spaces if you've opted in based on who you are what you've purchased and what we think you might like automated processes I've touched on now so in the past two-three months that is becoming a very present form of facial recognition regulation and package solutions what I mean by that is using facial recognition with age recognition with object detection with license-plate recognition so taking
one solution that is typically in microservices or a compartmentalized solution combining it into one and making that run at the edge you have all those data points need to make to make a decision that's about a year out from happening in regulation I'm all for regulation but we just had a law passed in California yesterday or two days ago now that is banning the use of facial recognition by the local law enforcement and the ban is is actually not a ban what it is is it's a they now have to use facial recognition like a warrant for someone's arrest so they have to go get judge approval to use facial recognition on their cities now the police forces they are fighting back saying that we've caught X amount of individuals who have committed crimes with this technology why would you ban it but San Francisco being the city that it is is very focused on its law by law-abiding citizens and so they want to ensure that there is regulation there is a discussion around
privacy before they just let the local law enforcement go wild with this technology so I think it's a step in the right direction I think there should be federal federal federal e regulated or regulated by someone like the United Nations where we've got one consensus on how facial recognition can be used among a variety of industries because Defense Department's use it very differently than iPhones do and that they use it very differently than Facebook does so there's a variety of applications within facial recognition it's important to delineate or two vertical eyes those implementations when you do regulate this technology this is just an example of how this technology works with the age so age recognition right now is at a mean average error of two and a half years this footage is from the 1960s and we're still able to pull those numbers so it's incredibly powerful when you talk about the retail environment if you're buying alcohol and tobacco instead of having to show a form of identification we now know it than two
and a half years what your age is based on just your face so combining this with things like a motion analysis and facial recognition becomes a very powerful solution so it's important for me to leave you with one last thing in this technology and in computer vision in general is only adopted on a mass scale when it's used responsibly it's very very important that there are checks and balances when it comes to facial recognition and how it's deployed so if this technology is used responsibly we're very confident that we can provide a safer and smarter world for everyone thank you very much Sean I'm sure there are a lot of questions in this audience so raise up your head I see one over here Theresa and next to it also three already we have the time so we take them hi how do you prevent accidental consent to a scan for example if you're I don't know sleeping or something and someone holds a phone up to your face and you accidentally without knowing consent
to a payment how do you prevent that and what steps is your company taking absolutely so typically there's going to be two parts that equation there's going to be the vendor providing that solution and then us providing the technology to that vendor and what we do is we put liveness checks in there so depending on the sensor we can measure your heartbeat through your forehead and very simple sensors can do that now so it's a pretty impressive technology and then we measure things like blinking or face movements micro movements in your face over the course of 15 30 frames so there are ways to measure for that that specific application is something Apple has had trouble with and you know they may be prematurely launched at that platform I'm not sure I'm not in the mobile space so it's something that is absolutely of a concern for our specific clients you know we don't run into the sleeping issue it would be more of what happens if you're held up at an ATM and they
force you to put your face and can we recognize the stress and so that's something we are working on is distress signals okay next to him there was another question Suzanne said okay I saw a Christian bear yes I just like to say I'm very persuaded persuaded that the technology will be used but my concern from an ethical perspective is it's dependent upon the concept of responsible government if it's a government decision to implement it or not and there are responsible governments at time T and there are irresponsible governments at time t1 or TN or whatever so that's my concern and I would like to see personally some ethical regulation saying that there are cases where were it's valid and there are cases where it's not valid and it would be prohibited a kind of United Nations perspective on that and you know I've seen and heard and read about applications of facial technology to detect criminality potential it just seems weird you know the case I'm talking I do yes yes yeah I think
you know referencing China that's a it's a different beast there based on what their government has done they've invested billions of dollars in those companies that are providing facial recognition them and the Russian government and so they have vested interested in the top or the three billion dollar companies in China right now I think you're absolutely right I think there has to be some ethical regulation around this technology the companies in the United States wash being one of it one of them have released ethical statements about what we will and won't do and I would say the most important thing is that humanity comes first there's total transparency in humanity comes first mean that if a decision will affect the human's life a human has to make that final decision so it's meant to be a point of data as collecting evidence from a crime scene each each piece of evidence is a point of data that's how we view facial recognition in that regard so I entirely agree with you I think there needs
to be the checks and balances on the ethical level to avoid what is currently happening in China yeah okay the problem with that is there are situations in which you deploy your software locally and at the end of the day that agency that representative that company can prevent you from knowing how they're using it so they can be lying to you about how they're using it and you have no way to stop that I know that yeah so that's a very hard thing to monitor but I see your point and it's valid we have one more question over there I would like to challenge or they recognition age estimation metrics because especially with longevity and I think it's difficult for people to tell how old people look even when they go to a doctor sometimes your physi physiology you know that your physical age is not necessarily matching your biological age how can facial recognition really estimate that there are definitely outliers but what you have to understand is that it's all math at the
end of the day and if you get billions and billions and billions of points of input you start to collect closer and closer to the center on an x and y axis and so you know you do get two and a half years on a large majority of the population there are some instances which your age advances faster than your bar your body advances faster than your age and that's things do - like alcohol and tobacco so you know there is some built in measurement for those types of things but I can tell you right now that I'm confident over millions and millions of faces that it is two and a half years mean average error a Christian to Christians we're still here we have time can you differentiate twins and there are some sources that claim that for each one of those there are sin identical people in the world so how do you get around with that twins are difficult there is no basis for a like a government study based on twins
so there's a lot of metrics that we can quote for recognition one two one one two and aging from 30 years to 60 years there's nothing yet on twins so it's difficult for me to give you a number you know 90% confident on twins I can tell you we've personally tested I think 15 to 20 sets which is not significant statistically significant at all and we've never had a problem so we're able to identify every single set appropriately I think that is an issue but I think you have to look at in what environment does that become a concern and if it is a concern and we have other information on that person we all know they have a twin so it comes back to reviewing that you know I think that that's something that is difficult for facial recognition companies but from my own experience we've been able to figure it out not at scale okay one more question over here quite similar to that how easy is it to outsmart facial recognition technologies
you saw an example I think about a week ago with somebody wearing the certain panel maybe show was no longer recognizable yep that's well that's an adversarial attack so that's a way to infect the algorithm to see something that's not there another example of that is people were putting QR codes and stop signs and that was reading that as a green light so it's a significant risk when it comes to self-driving cars because they'll see that adversarial attack and go through that stop sign for facial recognition when we know about them we can detect them and so you know it's a game of cat and mouse and it always will be there's makeup out there that contours your face but if you've seen that makeup in real life it's way more obvious that you're trying to trick the system than if you just had regular makeup on or had a hat on so I think that's a game of cat and mouse that will never stop there will always be people trying to put that
technology out there as long as we know what an adversarial attack looks like we can stop that do you think that should be regulated it's gonna be very hard regulating the hackers is difficult I do think it should be regulated though yes yes it should be a crime yeah I agree there's one last question over there and then we head over to the next speaker I didn't understand the beat about put in it put in the face recognition hash into the trash bin at all so I presume that you have a 512 bits or something that's a hash of some matrix of a face that you have so the actual space is much smaller than 520 then if I as a developer for instance have access to the actual metrics that were used to generate a hash what prevents me from using general and Rosario networks to generate a face that would match those metrics and get the same hash and without changing the algorithm you would not be able to detect that I did
that so how would you fight it yeah I mean you you would have to have like on-site access to all of our source code to even get close to doing that yes so let's say I have that I said I'm a developer I told context which again sent around there are ways in which we know if you try to infect our system yes but it would be difficult if you are on our team and you had that full access to that if you wanted to reverse engineer you could try to do that there are ways to protect that there are encryptions that we can put in that completely obliterate those numbers so you know it's really not that likely of a scenario for us there's quite a bit of protection around everything that we do we work with some some clients that enforce that very strictly and put us through those tests so it's just not as big of an issue to us at this stage if you are working in our company and you
ended up being a spy from a you know a foreign agency we would have at least an idea that you were trying to infect our software so it's you know it's an internal mechanism that we have to put up okay guys yeah thanks