And so our goal was a natural looking Gate it was real it was surprisingly Hard to get that to work Um and we but we did build an early Machine Uh we called it pet man prototype it was The Prototype before the Pac-Man robot And it had a really nice looking Gate where you know it would stick the Leg out it would do heel strike first Before it rolled onto the toe so you Didn't land with a flat foot you Extended your leg a little bit Um but even then it was hard to get the Robot to walk where when you're walking That it fully extended its leg And getting that all to work well Took such a long time in fact I I Probably didn't really see the nice Natural walking that I expected out of Our humanoids until maybe last year And the team was developing on our newer Generation of Atlas you know some new Techniques For developing a walking control Algorithm and they got that natural Looking motion as sort of a byproduct of A just a different process that we're Applying to developing the control So that probably took 15 years 10 to 15 Years to sort of get that from from you Know The Petman prototype was probably in 2008 and what was it 2022 last year that
I think I saw a good walking on Atlas The following is a conversation with Robert plater CEO of Boston Dynamics a Legendary robotics company that over 30 Years has created some of the most Elegant dexterous and simply amazing Robots ever built including the humanoid Robot Atlas and the robot dog spot One or both of whom you've probably seen On the internet either dancing doing Backflips opening doors or uh throwing Around heavy objects Robert has led both the development of Boston Dynamics humanoid robots and Their physics-based simulation software He has been with the company from the Very beginning including its roots at MIT where he received his PhD in Aeronautical Engineering this was in 1994 at the legendary MIT leg lab he Wrote his PhD thesis on robot gymnastics As part of which he programmed a bipedal Robot to do the world's first 3D robotic Somersault Robert is a great engineer robot Assistant leader and Boston Dynamics to Me as a roboticist is a truly inspiring Company this conversation was a big Honor and pleasure and I hope to do a Lot of great work with these robots in The years to come This is the Lex Freedom podcast to Support it please check out our sponsors In the description and now dear friends
Here's Robert plater When did you first fall in love with Robotics Let's start with love and robots well Love is is relevant because I think the The fascination the Deep Fascination is Really about movement And uh I was visiting MIT looking for a place To get a PhD and I wanted to do some Laboratory work and one of my professors At in the Aero Department said go see This guy mark raber down in the basement Of the AI lab And so I walked down there and saw him He showed me his robots And he showed me this robot doing a Somersault And I just immediately went whoa you Know yeah robots can do that and because Of my own interest in in gymnastics There was like this immediate connection And um you know I was interested in I Was in an arrow Astro degree because you Know flight and movement was all so Fascinating to me and then it turned out That you know robotics had this big Challenge how do you how do you balance Uh how do you how do you build a legged Robot that can really get around And that just that was a Fascination and It still exists today you're still Working on perfecting Motion in robots What about the elegance and the beauty
Of the movement itself is is there Something Maybe grounded in your appreciation of Uh movement from your gymnastics days Did you Was there something you just Fundamentally appreciate about the Elegance and beauty of movement you know We had this concept in in gymnastics of Um Letting your body do what it wanted to Do when you get really good at Gymnastics Part of what you're doing is putting Your your body into a position where the Physics and the body's inertia and Momentum will kind of push you in the Right direction in a very natural and Organic way And the thing that Mark was doing you Know in the basement of that laboratory Was trying to figure out how to build Machines to take advantage of those Ideas how do you build something so that The physics of the machine just kind of Inherently wants to do what it wants to Do and he was building these springy Pogo stick type you know his first cut At Lego Locomotion was a pogo stick Where it's bouncing and there's a spring Mass A system that's oscillating has its own Sort of natural frequency there and sort Of figuring out how to augment those
Natural physics With also intent how do you then control That but not overpower it it's that Coordination that I think creates real Potential we could call it Beauty yeah You could call it I don't know Synergy That people have different words for it But I think that that was inherent from The beginning that was clear to me that That's part of what Mark was trying to Do he asked me to do that in my research Work so Um you know that's where I got going so Part of the thing that I think I'm Calling elegance and Beauty in this case Which was there even with the pogo stick Is maybe the efficiency so letting the Body do what it wants to do Trying to discover the efficient Movement it's definitely more efficient It also becomes easier to control in its Own way because the physics are solving Some of the problem itself it's not like You have to do all this calculation and Overpower the physics the physics Naturally inherently want to do the Right thing There can even be you know a feedback Mechanisms stabilizing mechanisms That occur simply by virtue of the Physics of the body and it's you know Not all Not all in the computer or not even all In your mind as a person and I there's
Something interesting in that melding You were with Mark for many many many Years but you were there in this kind of Legendary space Of leg lab and a my team in the basement All great things happen in the basement Is there some memories uh is there some Money from that time That you have because it's so it's such Cutting Edge work In in in robotics and artificial Intelligence The memories the distinctive lessons I Would say I learned in that in that time Period And um and that I think Mark was a great Teacher of Was it's okay to pursue your interest Your curiosity do something because you Love it Um you'll do it a lot better if you love It Um That that is a lasting lesson that I Think uh we apply at the company still Um and really is a core value so the Interesting thing is I got to um Uh with people like Ross Cedric and Um and others like the students that Work at those robotics labs are like Some of the happiest people I've ever Met I don't know what that is I mean a lot Of PhD students a lot of them are kind
Of broken by the wear and tear of the Process uh but roboticists are while They work extremely hard and work a long Hours There's a There's a happiness there the only other Group of people I met like that are People that Skydive a lot like for some Reason there's a deep fulfilling Happiness maybe from like a long period Of struggle to get a thing to work and It works and there's a magic to it I Don't know exactly because it's so Fundamentally Hands-On and you're Bringing a thing to life I don't know What it is but they're happy We see you know our our attrition at the Company is really low people come and They love the pursuit And I think part of that is that there's Perhaps an external connection to it It's a little bit easier to connect when You have a robot that's moving around in The world and part of your goal is to Make it move around in the world You can identify with that and and this Is on a this is one of the unique things About the kinds of robots we're building Is this physical interaction Lets you perhaps identify with it so I Think that is a source of happiness I Don't think it's Unique to robotics I Think anybody also who is just pursuing Something they love
It's easier to work hard at it and be Good at it and Um Not everybody gets to find that I I do Feel lucky in that way and I think we're Lucky as an organization that we've been Able to build a business around this and That keeps people engaged So if it's all right let's link on mark For a little bit longer Mark raybert so He's a legend Uh he's a legendary engineer Roboto says What what have you learned about life About Robotics and Mark through all the Many years you worked with him I think The most important lesson which was you Know have the courage of your Convictions and and do what you think is Interesting Um Be willing to try to find big big Problems to go after and at the time you Know like at Locomotion Um especially in a dynamic machine Nobody had solved it and that felt like A Multi-decade problem to go after And so you know have the courage to go After that because you're interested Don't worry if it's going to make money You know that that's been a theme so That that's really uh probably the most Uh important lesson I think that uh I Got from Mark how crazy is the effort of
Doing legged Robotics at that time especially You know Mark got some stuff to work uh Starting from the simple ideas oh so Maybe the other another important idea That has really become a value of the Company is try to simplify a thing to The Core Essence And and while you know Mark was showing Videos of animals running across the Savannah or uh uh climbing mountains What he started with was a pogo stick Because he was trying to reduce the Problem to something that was manageable And and getting the pogo stick to Balance had in it The fundamental problems that if we Solve those you could eventually Extrapolate to something that galloped Like a horse And so look for those simplifying Principles Um how tough is the job of simplifying a Robot so I I'd say in the early days the The thing that made Boston The researchers at Boston Dynamics Special Is that we we worked on under figuring Out what that that Central principle was And then building software or machines Around that principle and that was not Easy in the early days and and it it Took Um real expertise in understanding the
Dynamics of motion and feedback control Principles and how to build and with Computers at the time how to build a Feedback control algorithm that was Simple enough that it could run in real Time at a thousand Hertz And actually get that machine to work Um and that was not something everybody Was doing you know at that time Now the world's changing now and I I I Think the approach is to controlling Robots are going to change Um but uh and they're going to become More broadly yet Um available Um but at the time there weren't many Groups who could really sort of work at That principled level With both the software and And make the hardware work And I'll and I'll say one other thing About your sort of talking about what Are the special things the other thing Was it's okay it's good to break stuff You know Um you know use the robots break them Repair them Um you know fix and repeat test fix and Repeat and that and that's also a core Principle that has become part of the Company And it lets you be Fearless in your work Too often if you are working with a very Expensive robot maybe one that you
Bought from somebody else or that you Don't know how to fix then you treat it With kit gloves and you can't actually Make progress you have to be able to Break something and so I think that's a Been a a principle as well so just the Link on that psychologically how do you Deal with that because I remember I had Uh Uh I built a RC car With that some Uh it had some custom stuff like compute On it and all that kind of stuff cameras And uh because I didn't sleep much the Code I wrote has an issue where it Didn't stop the car and then the car got Confused and at full speed at like 20 25 Miles an hour slammed into a wall And I just remember sitting there alone In the deep sadness Um Sort of Full of regret I think almost anger Um Uh but also like sadness because you Think about well these robots especially For autonomous vehicles like like you Should be taking safety very seriously Even in these kinds of things but just No good feelings and made me more afraid Probably to do this kind of experiments In the future perhaps the right way to Have seen that is positively Like it's it's too it depends if you
Could have built that car or or just Gotten another one right that would have Been the approach Um I remember Um When I got to grad school Um you know I got some training about uh Operating a lathe and a mill up in the Machine shop and I could start to make My own parts and I remember breaking Some piece of equipment in the lab and Then realizing Because I maybe this was a unique part And I couldn't go buy it and I realized Oh I can just go make it That was an enabling feeling yeah then You're not afraid yeah it might take Time it might take more work than you Thought it was going to be required to Get this thing done But you can just go make it and that's Freeing in a way that nothing else is You mentioned uh the the feedback Control the Dynamics sorry for the Romantic question but is in the early Days and even now is the Dynamics Probably more appropriate for the early Days is it more art or science There's a lot of science around it And and trying to develop you know Scientific principles That let you extrapolate from like one Legged machine to another You know develop a core set of
Principles like like a spring Mass Bouncing system and then figure out how To apply that from a one-legged machine To a two or a four-legged machine those Principles are really important and and We're definitely a core part of our work There's also You know when we started to pursue Humanoid robots Um there was so much complexity in that Machine That You know one of the benefits of the Humanoid form is you have some intuition About how it should look while it's Moving And that's a little bit of an art I Think and now I'd say or maybe it's just Tapping into a knowledge that you have Deep in your body and then trying to Express that in the machine but that's An intuition that's a little bit more on The art side maybe it it predates your Knowledge you know before you have the Knowledge of how to control it you try To work through the Art Channel and Humanoids sort of make that available to You if it had been a different shape Maybe we wouldn't have had the same Intuition about it yeah so you're Knowledge about moving through the world Is not made explicit to you So you just that's why it's art and it Might yeah it might be hard to actually
Articulate exactly you know there's Something about Um and being a competitive uh athlete There's something about Seeing a movement you know a coach one Of their greatest strengths a coach has Is being able to see you know some Little change in what the athlete is Doing and then being able to articulate That to the athlete you know and then Maybe even trying to say and you should Try to feel this Um so there's something just in scene And again you know sometimes it's hard To articulate what it is you're seeing But there's a Receiving the motion at a rate that is Again sometimes hard to put into words Yeah I Wonder How it is possible to achieve sort of Truly elegant movement you have a movie Like ex machina I'm not sure if you've Seen it but the main actress in that who Plays the AI robot I think is a Ballerina I mean just a natural Elegance and the I don't know eloquence Of movement It's it's it looks efficient and easy And just it looks right It looks it looks right is sort of the Key yeah and then you you look at uh Especially early robots I mean they They're so cautious in in the way they
Move that It's not it's not the caution that looks Wrong it's it's something about the Movement that looks wrong that feels Like it's very inefficient unnecessarily So and it's hard to put that into words Exactly we think that and part of the Reason why people are attracted to the Machines we build Is because the inherent dynamics of Movement are are closer to right Um because we we try to use you know Walking Gates or we build a machine Around this gate where you're trying to Work with the Dynamics of the machine Instead of To stop them you know some of the early Walking machines You know you're essentially you're Really trying hard to not let them fall Over and so you're always stopping the Tipping motion you know And sort of the insight Of dynamic stability and a lighted Machine is to go with it you know let The Tipping happen you know let yourself Fall but then catch her catch yourself With that next foot and there's Something about getting those physics to Be expressed in the machine That people interpret as Lifelike or or elegant or just natural Looking and so I think if you get the Physics right
It also ends up being more efficient Likely There's a benefit that it probably ends Up being More stable in the long run you know it Could it could walk stably over a wider Uh rain range of conditions And it's uh and it's more beautiful and Attractive at the same time so how hard Is it to get the humanoid robot Atlas To do some of the things that's recently Been doing let's forget the flips and All of that let's just look at the Running Maybe you can correct me but there's Something about running I mean that's Not careful at all that's you're falling Forward You're jumping forward and they're Falling so how hard is it to get that Right our first humanoid we needed to Deliver natural looking walking you know We took a contract uh from the army they Wanted a robot that could walk naturally They wanted to put a suit on the robot And be able to test it in a gas Environment and so they wanted that the Motion to be natural And so our goal was a natural looking Gate it was real it was surprisingly Hard to get that to work And we but we did build an early machine We called it pet man prototype it was The Prototype before the Pac-Man robot
And it had a really nice looking Gate where you know it would stick the Leg out it would do heel strike first Before it rolled onto the toe so you Didn't land with a flat foot you Extended your leg a little bit But even then it was hard to get the Robot to walk where when you're walking That it fully extended its leg and Essentially landed on an extended leg And if you watch closely how you walk You probably land on an extended leg but Then you immediately flex your knee as You start to make that contact And getting that all to work well Took such a long time in fact I I Probably didn't really see the nice Natural walking that I expected out of Our humanoids until maybe last year And the team was developing on our newer Generation of Atlas you know some new Techniques Um uh for developing a walking control Algorithm and they got that natural Looking motion as sort of a byproduct of A just a different process they were Applying to developing the control So that probably took 15 years 10 to 15 Years to sort of get that from from you Know the Petman prototype was probably In 2008 and what was it 2022 last year That I think I saw a good walking on Atlas if you could just like Linger on It what are some challenges of getting
Good walking so is it uh Is this is this partially like a Hardware like actuator problem is it the Control is it the artistic element of Just observing the whole system Operating in different conditions Together I mean is there some kind of Interesting Quirks or challenges you can speak to Like the heel strike yeah so one of the Things that makes the like this straight Leg uh a challenge is you're sort of up Against a singularity a mathematical Single Singularity where you know when Your leg is fully extended it can't go Further the other direction right There's only you can only move in One Direction and that makes all of the Calculations around how to produce Twerks at that joint or positions makes It more complicated and so having all The mathematics so it can deal with These singular configurations is one of Many challenges uh that we face and I'd Say in in the You know in those earlier days again we Were working with these really Simplified models So we're trying to boil all the physics Of the complex human body into a simpler Subsystem that we can more easily Describe in mathematics and sometimes Those simpler subsystems don't have all Of that complexity of the straight leg
Built into them and so what's happened More recently is we're able to apply Techniques that let us take the full Physics of the robot into account and And deal with some of those strange Situations like this like the straight Leg so is there a fundamental challenge Here that it's uh maybe you can correct Me but is it under actuated are you Falling under actuated is the right word Right you can't you can't uh push the Robot in any direction you want to right And so that that is one of the hard Problems of of uh like at Locomotion and You have to do that for natural movement It's not necessarily required for Natural movement it's just required You know we don't have you know a Gravity force that you can hook yourself Onto to apply uh an external force in The direction you want at all times Right the only the only external forces Are being mediated through your feet and How they get mediated depend on how you Place your feet and uh you know you Can't just uh you know God's hand can't Reach down and give and push in any Direction you want you know so is there Uh is there some extra challenge to the Fact that Alice is such a big robot There is the humanoid form is um Attractive in many ways but it's also a Challenge in many ways Um
You have this big upper body that has a Lot of mass and inertia Um and throwing that inertia around Increases the complexity of maintaining Balance and as soon as you pick up Something heavy in your arms you've made That problem even harder And so uh in the early work in the leg Lab and in the early days at the company And we were pursuing these quadruped Robots which had a a kind of built-in Simplification you had this big rigid Body and then really light legs so when You swing the legs The leg motion didn't impact the body Motion very much All the mass and inertia was in the body But when you have the humanoid that Doesn't work you have big heavy legs you Swing the legs it affects everything Else And so Dealing with all of that interaction Does make the humanoid a much more Complicated platform And I also saw that at least recently You've been doing more explicit modeling Of the stuff you pick up yeah which is Very real Um really interesting so you have to What model the shape The weight distribution [Music] I don't know what like you have to under
Like include that as part of the Modeling as part of the planning because Okay so for people who don't know Uh so Atlas at least in like a recent Video like throws a heavy bag throws a Bunch of stuff So what what's involved in uh picking up A thing a heavy thing Uh and when that thing is a bunch of Different non-standard things I think it Also picked up like a barbell And to be able to throw in some cases What are some interesting challenges There So we were definitely trying to show That the robot and the techniques were Applying to the robe uh to Atlas let us Deal with heavy things in the world Because if the robot's going to be Useful it's actually got to move stuff Around yeah and that and that needs to Be significant stuff that's an Appreciable portion of the the body Weight of the robot And we also think this differentiates us From the other humanoid robot activities That you're seeing out there mostly They're not picking stuff up yet And not heavy stuff anyway Um but just like you or me you know you Need to anticipate that moment you know You're reaching out to pick something up And as soon as you pick it up your Center of mass is going to shift
And if you're gonna you know turn in a Circle you have to take that inertia Into account and if you're gonna throw a Thing you know you've got all of that Has to be sort of included in in the Model of what you're trying to do so the Robot needs to have some idea or Expectation of what that weight is and Then and sort of predict you know think A couple of seconds ahead how do I Manage my now my my body plus this big Heavy thing together to get and and Still maintain balance right and so um I I uh that's a big change for us and I Think the tools we've built are really Allowing that to happen Um quickly now some of those motions That you saw in that most recent video We were able to create in a matter of Days it used to be it took six months to Do anything new you know on your robot And and now we're starting to develop The tools that let us do that in a Matter of days and so we think that's Really exciting it means that the Ability to create new behaviors for the Robot is going to be um A quicker process so being able to Explicitly model New things that it might need to pick up New type of thing and you know to some Degree you don't you don't want to have To pay too much attention to each Specific thing right
Um there's sort of a generalization here Yeah Um obviously when you grab a thing you Have to conform your your hand your end Effector to the surface of that shape But once it's in your hands it's Probably just the mass and inertia that Matter and the the shape may not be as Important yeah and so you know for some In some ways you want to pay attention To that detailed shape and in others you Want to generalize it and say uh well All I really care about is the center of Mass of this thing especially if I'm Going to throw it up on that scaffolding And it's easier if the body is rigid What if it's there's some doesn't it Throw like a sandbag type thing that Tool bag you know you've had loose had Loose stuff in it yeah so it it managed That there are harder things that we Haven't done yet you know we could have Had a big jointed thing or I don't know A bunch of loose wire or rope what about Carrying another robot how about that Yeah we haven't we haven't done that yet I guess we did a little bit of uh we did A a little skit around Christmas where We had two spots holding up another spot That was trying to put you know a bow on A tree so I guess we're doing that in a Small way Okay that's pretty good uh let me ask The all-important question uh do you
Know how much Atlas can curl goodbye Have you I mean you know this for us humans That's really one of the most Fundamental questions you can ask Another human being a bench it probably Can't curl as much as we can yet but a Metric that I think is interesting is um You know another way of looking at that Strength Is you know the box jump so if how high Of a box can you jump onto question and Uh Alice I don't know the exact height It was probably a meter high or Something like that it was a pretty Pretty tall jump that Atlas was able to Manage when we last tried to do this and And I have video of my chief technical Officer Doing the same jump and he really Struggled you know the human but the Human getting all the way on top of this Box but then you know Atlas was able to Do it Um we're now thinking about the next Generation of Atlas and we're probably Going to be in the realm of a person Can't do it you know with this with the Next Generation you know the robots the Actuators are going to get stronger Where there really is the case that at Least some of these joints some of these Motions will be stronger and to Understand how high it can jump you
Probably had to do quite a bit of Testing oh yeah and there's lots of Videos of it trying and failing and That's you know that's all you know we Don't always release those those videos But they're a lot of fun to look at Uh so we'll talk a little bit about that Uh but if can you talk to the jumping Because you talked about the walking it Took a long time many many years to get The walking to be natural but there's Also really natural looking Uh robust resilient jumping how hard is It to do the jumping Well again this stuff has really evolved Rapidly in the last few years you know The first time we did a somersault Um you know there's a lot of kind of Manual iteration What is the trajectory you know how hard Do you throw it in fact in these early Days uh I actually would when I'd see Early experiments that the team was Doing I might make suggestions about how To change the technique again kind of Borrowing from my own intuition about How backflips work Um But frankly they don't need that anymore So in the early days you had to iterate Kind of in almost a manual way trying to Change these trajectories of the arms or The legs to try to get you know a Successful backflip to happen
But more recently we're running These model predictive uh control Techniques where we're able to the robot Essentially can think in advance for the Next second or two about how its motion Is going to transpire and you can you Know solve for optimal trajectories to Get from A to B so this is happening in A much more natural way and we're really Seeing an acceleration happen in the Development of these behaviors again Partly due to these Optimization techniques uh sometimes Learning techniques So it's there's it's hard in that There's a lot of mathematics in behind It but we're figuring that out so you Can do model predictive control for Uh I mean I don't even understand what That looks like when the entire robot is In the air flying and doing a back yeah I mean but but that's the cool part Right so you know yeah you know the Physics we we can calculate physics Pretty well using you know Newton's laws About how it's going to evolve over time And the road you know this this the sick Trick which was a front somersault with A half twist is a good example right You saw the robot on various versions of That trick I've seen it land in Different configurations and it still Manages to stabilize itself and so you Know what this model predictive control
Means is again the in real time the Robot is projecting ahead you know a Second into the future and sort of Exploring options and if I if I move my Arm a little bit more this way how is That going to affect the outcome and so It can do these calculations many of Them you know uh and and basically solve For you know given where I am now maybe I took off a little bit screwy from how I had planned I can adjust so you're Adjusting in there just on the fly so The the model predictive control lets You adjust on the Fly And of course I think this is what you Know people adapt as well we when when We do it even a gymnastics trick we try To set it up so it's as close to the Same every time but we figured out how To do some adjustment on the Fly and now We're starting to figure out that the Robots can do this adjustment on the fly As well using these techniques in the Air And so I mean it just feels from a Robotics perspective just surreal well That's sort of the you talked about Under actuated right so when you're when You're in the air there's something There's some things you can't change Right you can't change the momentum While it's in the air because you can't Apply an external force or Torque and so The momentum isn't going to change so
How do you work within the constraint of That fixed momentum to still get from A To B where you want to be that's really Unfortunate You're in the air I mean you become a Drone for a brief moment of time no You're not even a drone because you Can't can't ever you can't hover you're Gonna you're gonna impact soon be ready Yeah are you considered like a hover Type thing or no no it's too much weight I mean it's just it's just incredible uh And just even to have the guts to try Backflip with such a large body That's wild but like uh we definitely Broke a few robots trying but that but That's where the build it break it fix It you know uh strategy comes in you Gotta be willing to break and what ends Up happening is you end up by breaking The robot repeatedly you find the weak Points and then you end up redesigning It so it doesn't break so easily next Time you know through the breaking Process you learn a lot like a lot of Lessons and you keep improving not just How to make the backflip work but Everything and how to build the machine Better yeah yeah I mean is there something about just the Guts that Come up with an idea of saying you know What let's try to make it do a backflip Well I think the courage to do a
Backflip in the first place and and to Not worry too much about the ridicule of Somebody saying why the heck are you Doing backflips with robots because a Lot of people have asked that you know Why why why are you doing this why go to The moon in this decade and do the other Things JFK [Laughter] It's not because it's easy because it's Hard yeah exactly Don't ask questions okay so the uh the Jumping I mean it's just there's a lot Of incredible stuff if we can just Rewind a little bit to uh the DARPA Robotics challenge in 2015 I think which Was for people who are familiar with the DARPA challenges It uh was first with autonomous vehicles And there's a lot of interesting Challenges around that and the DARPA Robotics challenge is when uh humanoid Robots were tasked to do all kinds of Uh you know manipulation walking driving Your car all these kinds of challenges With if I remember correctly sort of Some slight capability to communicate With humans but the communication was Very poor so it basically has to be Almost entirely autonomous you can have Periods where the communication was Entirely interrupted and the robot had To be able to proceed yeah but you could Provide some high level guidance to the
Robot basically load low bandwidth Communications to steer it I watched That challenge with kind of tears in my Eyes eating popcorn But I wasn't personally losing uh you Know hundreds of thousands millions of Dollars and many years of incredible Hard work by some of the most brilliant Roboticists in the world so that was why The tragic that's why tears came so Anyway what what have you uh just Looking back to that time what have you Learned from that experience I mean maybe if you could describe what It was uh sort of the setup for people Who haven't seen it well so there was a Contest where a bunch of different Um robots were asked to do a series of Tasks uh some of those that you Mentioned drive a vehicle get out open a Door go identify a vowel shutter valve Use a tool to maybe cut a hole in Um A surface and then crawl over some Stairs and maybe some rough Terrain So it was The idea was have a general purpose Robot that could do lots of different Things Um it had to be mobility and Manipulation on board perception And there was a contest which DARPA Likes at the time was running sort of Follow on to the
The Grand Challenge which was let's Let's try to push vehicle autonomy along Right they they encourage people to Build autonomous cars so they're trying To basically push an industry forward And Um Uh we were asked our role in this was to Build Um a humanoid at the time it was our Sort of first generation Atlas robot And we built maybe 10 of them I don't Remember the exact number And DARPA distributed those to various Teams Um that sort of won a contest showed That they could you know program these Robots and then use them to compete Against each other and then other robots Were introduced as well some teams built Their own robots Carnegie Um melon for example built their own Robot and uh and all these robots Competed to see who could sort of get Through this this maze or the fastest And again I think the purpose was to Kind of push the whole industry forward We provided the robot and some baseline Software but we didn't we didn't Actually compete as a participant uh Where we were trying to uh you know Drive the robot through this maze we Were just trying to support the other Teams
It was humbling because it was it was Really a hard task and honestly the Robots the tears were because mostly the Robots didn't do it you know they fell Down You know repeatedly Um it was hard to get through this Contest uh some did and and you know They were rewarded and won But it was humbling because of just how Hard these tasks weren't all that hard a Person could have done it very easily But it was really hard to get the robots To do it you know the general nature of It the variety of it the variety and Also that I don't know if the tasks were Sort of the task in themselves help us Understand what is difficult and what is Not I don't know if that was obvious Before the contest was designed so you Kind of tried to figure that out and I Think Atlas is really a general robot Platform and it's perhaps not best Suited for the specific tasks of that Contest like for just for example Probably the hardest task is not the Driving of the car but getting in and Out of the car and Atlas probably you Know if you were to design a robot that Can get into the car easily and get out Easily you probably would not make Atlas That particular car yeah the robot was a Little bit big to get in and out of that Car right it doesn't fit yeah this is
The curse of a general purpose robot That they're not perfect at any one Thing But they might be able to do a wide Variety of things and and that is That is the goal at the end of the day You know I think we all want to build General purpose robots that can be used For lots of different activities But it's hard and Um and the wisdom in in building Successful robots up until this point Have been go build a robot for a Specific task and it'll do it very well And as long as you control that Environment it'll operate perfectly but But robots need to be able to deal with Uncertainty if they're going to be Useful to us in the future They need to be able to deal with Unexpected uh situations and that's sort Of the goal of a general purpose or Multi-purpose robot And that's just darn hard and so some of You know there's these curious little Failures like I remember one of the a Robot you know the first the first time You start to try to push on the world With a robot you you forget that the World pushes back and and will push you Over if you're not ready for it and the Robot you know reached to grab the door Handle I think it missed the grasp of The door handle was expecting that its
Hand was on the door handle and so when It tried to turn the knob it just threw Itself over it didn't realize oh I had Missed the door handle I didn't have I Didn't I was expecting a force back from The door it wasn't there and then I lost My balance so these little simple things That you and I would take totally for Granted and deal with the robots don't Know how to deal with yet and so you Have to start to deal with all of those Uh circumstances well I think a lot of Us experience this in uh even when sober But drunk too uh sort of you pick up a Thing and expect it to be What is it heavy and it turns out to be Light yeah oh yeah and then so the same And I'm sure if your depth perception For whatever reason is screwed up if You're if you're drunk or some other Reason and then you think you're putting Your hand on the table and you miss it I Mean it's the same kind of situation Yeah but There's why you need to be able to Predict forward just a little bit and so That's where this model predictive Control stuff comes in predict forward What you think is going to happen And then if and if that does happen You're in good shape if something else Happens you better start predicting Again so if we like we re-uh regenerate A plan yeah when you don't I mean that
Um that also requires a very uh fast Feedback loop of updating What your prediction how it matches to The actual real world Yeah those things have to run pretty Quickly what's the challenge of running Things pretty quickly a thousand Hertz Of acting and sensing Quickly you know there's a few different Layers of that you you want at the Lowest level you like to run things Typically at around a thousand Hertz Which means that you know at each joint Of the robot you're measuring position Or force and then trying to control your Actuator whether it's a hydraulic or Electric motor trying to control the Force coming out of that actuator and You want to do that really fast Something like a thousand Hertz and that Means you can't have too much Calculation going on at that joint Um but that's pretty manageable these Days and it's fairly common And then there's another layer that You're probably calculating you know Maybe at 100 Hertz maybe 10 times slower Which is now starting to look at the Overall body motion and thinking about The the larger physics of of the uh of The robot Um and then there's yet another loop That's probably happening a little bit Slower which is where you start to bring
You know your perception and your vision And things like that and so you need to Run all of these Loops sort of Simultaneously you do have to manage Your your computer time so that you can Squeeze in all the calculations you need In real time in a very consistent way And the amount of calculation we can do Is increasing as computers get better Which means we can start to do more Sophisticated calculations I can have a More complex model Doing my forward prediction And and that might allow me to do even Better predictions as I as I get better And better and and it used to be again We had You know 10 years ago We had to have pretty simple models That we were running you know at those Fast rates because the computers weren't As capable about calculating forward With a sophisticated model but as as Computation gets better we can we can do More of that what about the actual Pipeline of software engineering how Easy it is to keep updating Atlas like Duke continuous development on it so how Many computers Are on there is there a nice pipeline It's an important part of building a Team around it which means you know you Need to also have a software tools Simulation tools you know so
Um we have always made strong use of Physics-based simulation tools to do uh Some of this calculation basically Tested in simulation before you put it On the robot but you also want the same Code that you're running in simulation To be the same code you're running on The hardware and so even getting to the Point where it was the same code going From one to the other We probably didn't really get that Working until you know a few years Several years ago Um but that was a you know that was a Bit of a milestone and so you want to Work certainly work these pipelines so That you can make it as easy as possible And have a bunch of people working in Parallel especially when you know we Only have you know four of the atlas Robots the modern Atlas robots at the Company And you know we probably have you know 40 developers there all trying to gain Access to it and so you need to share Resources and use some of these uh some Of the software pipeline well that's a Really exciting step to be able to run The exact same code and simulation as on The actual robot uh how hard is it to do Uh realistic simulation physics-based Simulation of of Atlas such that I mean The dream is like if it works in Simulation works perfectly in reality
How hard is it to sort of keep workout Closing that Gap the root of some of our Physics-based simulation tools really Started at MIT and Um we built some some good physics-based Modeling tools there The early days of the company we were Trying to develop those tools as a Commercial product so we continued to Develop them it wasn't a particularly Successful commercial product but we Ended up with some nice physics-based Simulation tools so that when we started Doing legged robotics again we had a Really nice tool to work with And the things we paid attention to were Were things that weren't necessarily Handled very well in the commercial Tools you could buy off the shelf like Like interaction with the world like Foot ground contact so trying to model Those contact Um events well In a way that captured the important Parts of the interaction Was a really important element uh to get Right and to also do in a way that was Computationally feasible And could run fast because if you if Your simulation runs too slow you know Then your developers are sitting around Waiting for stuff to run and compile so It's always about efficient uh a fast Operation as well so that's been a big
Part of it you know I think developing Those tools in parallel to the Development of the the platform and Trying to scale them has has really been Essential I'd say to us being able to Assemble a team of people that could do This yeah how to simulate contact Periods so flick ground contact but sort Of for manipulation Because don't you want to model all Kinds of surfaces yeah so it will be Even more complex with manipulation Because there's a lot more going on you Know and you need to capture I don't Know things slipping and moving you know In in your in your hand Um it's a level of complexity that I Think goes above foot ground contact When you really start doing dexterous Manipulation so there's challenges ahead Still so how far are we away from me Being able to walk with Atlas in the Sand along the beach and Us both drinking a beer Yeah maybe we can out of a kid maybe Atlas could spill his beer because he's Got nowhere to put it Alice could walk on the sand uh so can It yeah uh yeah I mean you know have we Really had him out on the beach you know We take them outside often you know Rocks Hills that sort of thing even just Around our lab in Waltham we probably Haven't been on the sand but I'm a salt
Surface I don't doubt that we could deal With it yeah we we might have to spend a Little bit of time to sort of make that Work but we did take uh we we had a had To take Big dog to Thailand years ago and uh we Did this great video of the robot Walking in the sand walking into the Ocean up to I don't know its belly or Something like that and then turning Around and walking out all while playing Some cool beach music yeah great show But then you know we didn't really clean The robot off and the salt water was Really hard on it so you know we put it In a box shipped it back by the time it Came back we had some problems with salt It's the salt water it's not like old Stuff it's not like sand getting into The components or something like this But I'm sure if if this is a big Priority you can make it like waterproof Right that just wasn't our our goal at The time well it's a personal goal of Mine to it walk along the beach but it's A human problem too you get sand Everywhere it's it's just a jam mess So soft surfaces are okay so I mean can We just uh link on the the robotics Challenge there's a there's a pile of uh Like Rubble they had to walk over is That's Um how difficult is that task In the early days of developing big dog
The loose Rock was the epitome of the Hard walking surface because you step Down and then the Rock and you had these Little Point feet on the robot And the rock and roll and and you have To deal with that last minute you know Change in your foot placement yes so you You step on the thing and that thing Responds to you stepping on it yeah and And it moves where your point of support Is and so it's really that that became Kind of the essence of the test and so That was the beginning of us starting to Build Rock piles in our parking lots and And we would actually build boxes full Of rocks and bring them into the lab and Then we would have the robots walking Across these boxes of rocks because that Became the essential test So you mentioned big dog can you can we Maybe take a stroll through the history Of Boston Dynamics uh so what and who's Big Dog by the way is who Do you try not to anthropomorphize the Robots do you try not to To try to remember that they're this is Like the division I have because I for Me it's impossible For me there's a magic to the to the Being that is a robot it is not human But it is The same Magic Uh the living being has when it moves About the world is there in the robot so
Um I don't know what question I'm asking But uh should I say what or who I guess Who is Big Dog what is big dog Well I'll say to address the medic Question We don't try to draw hard lines around It being an it or a him or a her Um it's okay right people I think part Of the magic of these kinds of machines Is by nature of their organic movement Of the of their Dynamics We tend to want to identify with them we Tend to look at them and sort of Attribute Maybe feeling to that because we've only Seen things that move like this that Were alive And so Um this is an opportunity it means that You could have Feelings for a machine and you know People have feelings for their cars you Know they get attracted to them attached To them so that's inherently could be a Good thing as long as we manage what That interaction is So we don't put strong boundaries around This and ultimately think it's a benefit But it's also can be a bit of a curse Because I think people look at these Machines And they attribute a level of Intelligence that the machines don't Have why because again they've seen
Things move like this that we're living Beings Which are intelligent And so they want to attribute Intelligence to the robots that isn't Appropriate yet even though they move Like an intelligent being but you try to Acknowledge that the Anthropomorphization is there and try to First of all acknowledge that it's there And have a little fun with it you know Our most recent video It's just kind of fun you know to to Look at the robot we started off the the Video with Atlas Um kind of looking around for where the Bag of tools was because the guy up on The scaffolding says send me some tools Atlas has to kind of look around and see Where they are and there's a little Personality there that is fun it's Entertaining it makes our jobs Interesting and I think in the long run Can enhance interaction between humans And robots that in a way that isn't Available to machines that don't move That way this is something to me Personally is very interesting I've been Um I happen to have a lot of legged robots I hope uh to have a lot of spots in my Possession Um I'm interested in celebrating Robotics and celebrating companies and I
Also don't want to companies that do Incredible stuff like Boston Dynamics And I there's a You know I'm a little crazy and you say You don't want to you you want to align You want to help the company because I Ultimately want a company that busted an Image to succeed and part of that will Talk about you know success kind of Requires making money And so the kind of stuff I'm Particularly interested in may not be The thing that makes money in the short Term I can make an argument they will in The long term but the kind of stuff I've Been playing with is a robust way of uh Having the quadruped though the robot Dogs communicate emotion with their body Movement the same kind of stuff you do With it with a dog but not not Hard-coded uh but in a robust way and be Able to communicate excitement or fear Boredom all this kind of stuff and I Think as a base layer Of function of behavior to add on top of A robot I think that's a really powerful Way uh to make the robot more usable for Humans for whatever application it's Going to be really important and um it's A thing we're we're beginning to pay Attention to Um we really want to start a Differentiator for the company has Always been we really want the robot to
Work we want it to be useful uh Making it work at first meant the the Legged Locomotion really works it can Really get around and it doesn't fall Down and Um but beyond that and now it needs to Be a useful tool and our customers are For example Factory owners people who Are running a process manufacturing Facility and the robot needs to be able To get through this complex facility in A reliable way you know taking taking Measurements We need for people who are operating Those robots to understand what the Robots are doing If the robot gets into needs help or or You know is in trouble or something it Needs to be able to communicate And a physical indication of some sort Uh to so that a person looked at the Robot and goes oh I know what that's That robot's doing the robot's going to Go take measurements of my vacuum pump With its thermal camera You know you want to be able to indicate That and or even just the robots Um about to turn you know in front of You and maybe indicate that it's going To turn and so you sort of see and can Anticipate its motion so these this kind Of communication is going to become more And more important it wasn't sort of our Starting point
Um you know but now the the robots are Really out in the world and you know we Have about a thousand of them out with With customers right now This layer of physical indication I Think is going to become more and more Important we'll talk about where it goes Because there's a lot of interesting Possibilities but if you can return back To the origins of Boston and Dynamics With so that the more research the r d Side before we talk about how to build Robots at scale so big dog so um who's Big Dog so the company started in 1992. And in um probably uh uh 2003 I believe is when we uh took a contract From darpto basically 10 years 11 years Um we weren't doing robotics we did a Little bit of Robotics with Sony they Had uh IBO the their IBO robot we were Developing some software for that that Kind of got us a little bit involved With robotics again then there's this Opportunity to do a DARPA contract where They wanted to build A robot dog and uh we we won a contract To build that and so that was the Genesis of big dog And it was a quadruped and it was the First time we built a robot that had Everything on board that you could Actually take the robot out into the Wild and operate it so it had onboard Power plant it had onboard computers it
Had Hydraulic actuators that needed to be Cooled so we had cooling systems built In Everything integrated into the robot And uh that was a pretty rough start Right it was it was 10 years that we Were not a robotics company we were a Simulation company and then we had to Build a robot in about a year so that Was a little bit of a rough transition Um Can you just comment on the roughness of That transition because uh big dog I Mean this is just big Uh quadruped four legs robot rebuilt a Few different versions of them but the First one you're the very earliest ones You know it didn't work very well and we Would take them out and it was hard to Get you know uh You know a go-kart engine driving a Hydraulic that's not what it was And and you know uh having that all work Uh while trying to get you know the Robot to stabilize itself and so what Was the power plan what was the engine It seemed like uh my vague direct Collection I don't know it felt very loud and Aggressive and uh kind of thrown Together that's what it kind of oh it Absolutely was right we weren't trying To design the best robot Hardware at the
Time And uh we wanted to buy an off-the-shelf Engine and so many of the early versions Of big dog had literally go-kart engines Or something like that usually yeah like A yeah gas powered two-stroke engine And the reason why it was two-stroke is Two-stroke engines are lighter weight But they're also and we we generally Didn't put Mufflers on them because We're trying to save the weight and we Didn't care about the noise and so these Things were horribly loud Um but we're trying to manage weight Because managing weight in a legged Robot is always important because it has To carry everything That said that thing was big well I've Seen the videos yeah I mean the the Early versions you know stood about I Don't know belly High chest High Um you know they probably weighed maybe A couple of hundred pounds but you know It over the course of probably five Years We are able to get that robot To really manage a remarkable level of Rough terrain so you know we started out With just walking on the flat and then We started walking on rocks and then Inclines and then mud and slippery mud And you know by the end of that program We were convinced that legged Locomotion In a robot could actually work because
You know going into it we didn't we Didn't know that we had built quadrupeds At at MIT But they were they used a giant Hydraulic pump you know in the lab they Used a giant computer that was in the Lab they were always Tethered to the lab This was the first time something that Was sort of self-contained you know Walked around in the world And balance but and and the purpose was To prove to ourselves that the legged Locomotion could really work and so Um big dog really cut that open for us And it was the beginning of what became A whole series of robots so once once we Showed to darpan that you could make it Like a robot that could work There was a period at DARPA where Robotics got really hot and there was Lots of different programs And uh you know we were able to build Other robots we built other quadrupeds To hand like LS3 Designed to carry heavy loads we built Cheetah which was designed to explore What are the limits to how fast you can Run You know we began to build sort of a Portfolio Of machines and software that let us Build not just one robot but a whole Family everyone to push the limits in All kinds of directions yeah and to
Discover those principles you know you Asked earlier about the Art and Science Of a leg and locomotion We it were able to develop principles of Legged Locomotion so that we knew how to Build A small legged robot or a big one so Like like length you know was now a Parameter that we could play with Payload was a parameter we could play With so we built the LS3 which was an 800 pound robot designed to carry a 400 Pound payload And we we learned the design rules Basically developed the design rules how Do you scale different robot systems to You know their terrain to their walking Speed to their payload So when uh was spot born 2000 and uh 12 or so so again almost 10 Years into sort of a run with DARPA Where we built a bunch of different Quadrupeds we had a sort of a different Thread where we started building Humanoids Um we we saw that probably an end was Coming where the government was going to Kind of back off from a lot of Robotics Investment And uh in order to maintain progress we Just deduced that well we probably need To sell ourselves to somebody who wants To continue to invest in this this area And that was Google
And so Um at Google we would meet regularly With Larry Page and Larry just started Asking us you know what's your product Going to be and you know the logical Thing the thing that we had the most History with that we wanted to continue Developing was a quadruped but we knew It needed to be smaller we knew it Couldn't have a gas engine we thought it Probably couldn't be hydraulically Actuated So that began the process of exploring If we could migrate to a smaller Electrically actuated Um robot and that was really the Genesis Of spot So not a gas engine and the actuators Are electric yes so can you maybe Comment on what it's like Um my Google working with Larry Page Having those meetings and thinking of What will a robot look like That could be Built that scale what like starting to Think about a product Larry always liked the the toothbrush Test he wanted products that you used Every day Um What they really wanted was You know a consumer level product Something that would work in your house We didn't think that was the right next
Thing to do because to be a consumer Level product cost is going to be very Important Probably need to cost a few thousand Dollars and we were we were building These machines that cost hundreds of Thousands of dollars maybe a million Dollars to build of course we were only Building a two but But we didn't see how to get all the way To this consumer level product in a Short amount in a short amount of time And he suggested that we we make the Robots really inexpensive And part of our philosophy has always Been Build the best hardware you can Make make the machine operate well So that you're trying to solve You know discover the the hard problem That you don't know about don't don't Make it harder by by building a crappy Machine basically you build the best Machine you can there's plenty of hard Problems to solve that are going to have To do with you know under actuated Systems and balance And so we wanted to build these high Quality machines still and we thought That was important for us to continue Learning about really what was the Important parts of the make robots work Um and so there was a little bit of a Philosophical difference there that we
And and so ultimately that's why we're Building robots for the industrial Sector now because the industry can Afford a more expensive machine because You know their productivity depends on Keeping their Factory going and so if Spot costs you know a hundred thousand Dollars or more that's not such a big Expense to them whereas at the consumer Level no one's going to buy a robot like That and I think we might eventually get To a consumer level product that will be That cheap but I think the path to Getting there needs to go through these Really nice machines so we can then Learn how to simplify So what can you say to the almost the Engineering challenge of bringing down Costs Of a robot so that presumably when you Try to build the robot at scale that Also comes into play when you're trying To make money on a robot even in the Industrial setting but how interesting How challenging Of of a thing is that in particular Probably new to an r d company yeah I'm Glad you brought that last part up the Transition from an r d company to a Commercial company That's the thing you worry about you Know because you've got these Engineers Who love hard problems who want to Figure out how to make robots work and
You don't know if you have Engineers That want to work on the quality and Reliability and costs that is ultimately Required Um and indeed you know we have brought On a lot of new people who are inspired By those problems But but the big takeaway lesson for me Is We have good people we have Engineers Who want to solve problems and and the Quality and cost and manufacturability Is just another kind of problem And because they're so invested in what We're doing they're interested in and We'll go work on on those problems as Well and so I think we're managing that Transition very well in fact I'm really Pleased that I mean uh it's a huge undertaking by the Way right so even having To get reliability to where it needs to Be we have to have fleets of robots that We're just operating 24 7 in our offices To go find those rare failures and Eliminate them it's just a totally Different kind of activity than the Research activity where you get it to Work you know the one robot you have to Work in a repeatable way you know at the At the high stakes demo it's just very Different Um but I think we're making remarkable Progress against it so one of the cool
Things that got a chance to visit Boston Dynamics and I mean One of the things that's really cool is To see a large number of robots moving About Because I think one of the things you Notice in the research environment is The MIT for example I don't think anyone Ever has a working robot for prolonged Exactly so like most robots are just Sitting there in a sad State of despair waiting to be born Brought to life for a brief moment of Time but just to have I just I just Remember there's like a there's a spot Robot just I I had like a cowboy hat on It was just walking randomly for Whatever reason I don't even know but There's a kind of a sense of sentience To it because it doesn't seem like Anybody was supervising it but it was Just doing it I'm gonna stop way short Of the sentience Um it is the case that if you come to Our office you know today and walk Around the hallways Um you're gonna see a dozen robots just Kind of walking around yes all the time And that's really a reliability test for Us so we have these robots programmed to Do autonomous missions get up off their Charging dock walk around the building Collect data at a few different places And go sit back down and we want that to
Be a very reliable process because That's what somebody who's running a a Brewery a factory that's what they need The robot to do so we have to we have to Dog food our own robot we have to test It in that way And um so on a weekly basis we have Robots that are accruing something like Um 1500 or maybe 2 000 kilometers of Walking And uh you know over a thousand hours of Operation every week and that's Something that almost I don't think Anybody else in the world can do because Hey you have to have a fleet of robots To just accrue that much information You have to be willing to dedicate it to That test and uh so that's but that's Essential that's how you get the Reliability that's how you get it what About some of the cost cutting from the From the manufacturer side what what Have you learned from the manufacturer Side of the transition from r d and We're still we're still learning a lot There Um we're learning how to cast Parts in Solo instead of Mill it all out of you Know Billet aluminum Um we're learning how to get plastic Molded parts for and we're learning About how to control that process so That you can build the same robot twice In a row there's a lot to learn there
And we're only part way through that Process Um we've we've set up a manufacturing Facility in Waltham it's about a mile From our headquarters and we're doing Final assembly and test of both spots And stretches you know at that factory And um And it's hard because to be honest we're Still iterating on the design of the Robot as we find failures from these Reliability tests we need to go engineer Changes and those changes need to now be Propagated to the manufacturing line and That's a hard process especially when You want to move as fast as we do And that's been challenging and it makes It you know the folks who are working Supply chain who are trying to get the Cheapest parts for us kind of requires That you buy a lot of them to make them Cheap and then we go change the design From underneath them and they're like What are you doing and so you know Getting everybody on the same page here That it yep we still need to move fast But we also need to try to figure out How to reduce costs that's one of the Challenges of of this migration we're Going through and over the past few Years challenges to the supply chain I Mean imagine you've been a part of a Bunch of stressful meetings yeah things Got more expensive and harder to get and
Yeah so it's it's all been a challenge Is there still room for simplification Oh yeah much more and you know these are Really just the first generation of These machines we're already thinking About what the next generation of spots Going to look like Spot was built as a platform so you Could put almost any sensor on it and we Provided data Communications mechanical Connections uh Power connections And but for example in the applications That we're excited about where you're You're monitoring these factories for Their health There's probably a simpler machine that We could build that's really focused on That use case and that's the difference Between the general purpose machine or The platform versus the purpose-built Machine and so even though even in the Factory we'd still like the robot to do Lots of different tasks if it's if we Really knew on day one that we're going To be operating in a factory with these Three sensors in it we would have it all Integrated in a package that would be Easier more less expensive and more Reliable so we're contemplating building You know a next generation of that Machine so she mentioned that there's a Spot For people who somehow are not familiar Uh so it's a yellow robotic dog
And Um has been featured in many Dance Videos it also has gained an arm So what can you say about the arm that Spot has about the challenges of this Design And the manufacture of it we think the Future of mobile robots is mobile Manipulation that's where You know in the past 10 years It was getting Mobility to work getting The legged look emotional work if you Ask what's the hard problem in the next 10 years It's getting a mobile robot to do useful Manipulation for you And so we wanted spot to have an arm to Experiment with those problems Um and the arm is Um Almost as complex as the robot itself You know and uh it's a it's an Attachable payload Um it has you know several Motors and Actuators and sensors it has a camera in The end of its hand so you know you can Sort of see something The and the robot will control the Motion of its hand to go pick it up Autonomously so in the same way the Robot walks and balances managing its Own foot placement to stay balanced we Want manipulation to be mostly Autonomous where the robot you indicate
Okay go grab that bottle and then the Robot will just go do it using the Camera in its hand and then sort of Closing in on that um the grasp But it's it's a whole nother complex Robot on top of a complex legged robot And so and of course we made it the hand Look a little like a head you know Because again we want it to be sort of Identifiable in the last year a lot of Our sales have been people who already Have a robot now buying an arm to add to That robot oh interesting and so the the Armor is for sale oh yeah oh yeah it's An option what's the what's the Interface like to work with the arm like Is it pretty so are they designed Primarily I guess just ask that question In general about robots from Boston Dynamics is it designed to be Easily and efficiently operated remotely By a human being or is there also the Capability to push towards autonomy we Want both Uh in the next version of the software That we release uh which will be version 3.3 we're going to offer the ability of If you have a autonomous mission for the Robot we're going to include the option That it can go through a door which Means it's going to have to have an arm And it's gonna have to use that arm to Open the door And so that'll be an autonomous
Manipulation task that just you can Program easily uh with the robot Strictly through you know we have a Tablet interface And so on the tablet you know you sort Of see the The View that spot sees you Say there's the door handle you know the Hinges are on the left and it opens in The rest is up to you take care oh so it Just takes care of everything yeah so we We want in for a task like opening doors You can automate most of that and we've Automated a few other tasks we had a Customer Who had a high-powered breaker switch Essentially it's an electric utility Ontario power generation And they have to when they're going to Disconnect you know their power supply Right that could be a gas generator Could be a nuclear power plant you know From the grid you have to disconnect This breaker switch as you can imagine There's you know hundreds or thousands Of amps and volts involved in this Breaker switch And it's a dangerous event because Occasionally you'll get what's called an Arc flash as you just do this disconnect The power the Sparks jump across and People die doing this And so uh Ontario power generation used Our spot and and the arm through the Interface to to operate this this
Disconnect Um that's right in an interactive way And they showed it to us And we were so excited about it and said You know I bet we can automate that task And so we we got some examples of that Breaker switch and I believe in the next Generation of the software now we're Going to deliver back to Ontario power Generation they're going to be able to Just point the robot At that breaker so we have to indicate That's the switch there's sort of two Actions you have to do you have to flip Up this little cover press a button then Get a ratchet stick it in to a socket And literally unscrew this giant breaker Switch so there's a bunch of different Tasks and we basically automated them so That the human says okay there's the Switch go do that part That right there is the socket where You're going to put your tool and you're Going to open it up and so you can Remotely sort of indicate this on the a Tablet and then the robot just does Everything in between and it does Everything all the coordinated movement Of all the different actuators that Include the body and it maintains its Balance it it walks itself you know into Position so it's within reach and the Arm is in a position where it can do the Whole task so it manages uh the whole
Body so how does one become a big enough Customer to request features because I Personally want a robot that gets me Beer I mean that has to be like one of the Most requests I suppose in the Industrial setting that's uh a Non-alcoholic beverage Um of picking up objects and bringing The objects to you we love working with Customers who have challenging problems Like this and and this one in particular Because we felt like What they were doing a it was a safety Feature B we saw that the robot could do It because they they tele operated it The first time probably took him an hour To do it the first time right but the Robot was clearly capable and we thought Oh this is a great problem for us to Work on to figure out how to automate a Manipulation task and so we took it on Not not because we were going to make a Bunch of money from it and selling the Robot back to them but because it Motivated us to go solve what we saw as The next logical step But many of our customers in fact uh we We try to our bigger customers who Typically ones who are going to run a Utility or a factory or something like That We take that kind of direction from them And if they're especially if they're
Going to buy 10 or 20 or 30 robots and They say I really needed to do this well That's exactly the right kind of problem That we want to be working on and and so Note to self buy 10 spots And aggressively pushed for beer Manipulation I think it's fair to say it's Notoriously difficult to make a lot of Money as a robotics company How can you make money as a robotics Company can you speak to that it seems That a lot of Robotics companies fail Um it's difficult to build robots it's Difficult to build robots at a low Enough cost where customers even the Industrial setting want to purchase them And it's difficult to build robots that Are useful sufficiently useful so what Can you speak to and Boston Dynamics has Been Uh successful for many years of finding A way to make money well in the early Days of course you know the money we Made was from doing contract r d work And we made money But you know we weren't growing and we Weren't selling a product And then we went through several owners Who You know had a vision of not only Developing advanced technology but Eventually developing products and so Both you know Google and SoftBank and
Now Hyundai You know had that vision and were Willing to you know provide that in Investment Um now our discipline is that we need to Go find applications that are broad Enough that you could imagine selling Thousands of robots because it doesn't Work if you don't sell thousands or tens Of thousands of robots if you only sell Hundreds You will commercially fail and that's Where most of the small robot companies Have died Um And and that's a challenge because You know a you need to field the robots They need to start to become reli Reliable and as we as we've said that Takes time and investment to get there And so it really does take Visionary Investment to get there but we we Believe that we are going to make money In this uh industrial monitoring space Because You know if uh if a chip Fab if the line Goes down because a vacuum pump failed Someplace that can be in a very Expensive process it can be a million Dollars a day in Lost production maybe You have to throw away some of the Product along the way And so the robot if you can prevent that By inspecting the factory every single
Day maybe every hour if you have to There's real return on investment there But there needs to be a critical mass of This task and and we're focusing on a Few That we believe are Ubiquitous in the industrial production Environment and that's using a thermal Camera to keep things from overheating Using an acoustic imager to find Compressed air leaks using visual Cameras to read gauges Measuring vibration these are standard Things that you do to prevent unintended Shutdown of a factory And this this takes place in a beer Factory we're working with a B inbev it Takes place in chip Fabs you know we're Working with global foundries uh it Takes place in electric utilities and Nuclear power plants and so the same Robot Can be applied in all of these Industries And and as I said we have about actually It's 1100 spots out now to really get uh You know profitability we need to be at A thousand a year maybe maybe 1500 a Year you know for that sort of part of The business so it still needs to grow Um but but we're on a good path so I Think that's totally achievable so the Application should require the crossing That thousand robot barrier it really
Should yeah I want to mention you know Our second robot uh stretch yeah tell me About stretch what's stretch who stretch Stretch started differently than spot You know spot we built because we had Decades of experience building Quadrupeds we just we had it in our Blood we had to build a quadruped Product but we had to go figure out what The application was and we actually Discovered this this Factory Patrol Application uh basically preventative Maintenance by seeing what our customers Did with it Stretch is very different we started Knowing that there was warehouses all Over the world there's shipping Containers moving all around the world Full of boxes that are mostly being Moved by hand By some estimates we think there's a Trillion boxes cardboard boxes shipped Around the world each year and a lot of It's done manually It became clear early on That there was an opportunity for a Mobile robot in here to move boxes Around And the commercial experience has been Very different between stretch and with Spot As soon as we started talking to People or potential customers about what Stretch was going to be used for they
Immediately started saying oh I'll buy I'll buy that Roma you know in fact I'm Going to put in an order for for 20 Right now we just started shipping the Robot in January After you know several years of Development of this year this year so Our first deliveries of stretch to Customers were DHL and Maersk in January We're delivering a gap right now and we Have about seven or eight other Customers all who've already agreed in Advance to buy between 10 and 20 robots And so we've already got commitments for You know a couple hundred of these Robots This one's gonna go right it's so Obvious that there's a need and we're Not just going to unload trucks we're Going to do any box moving task in the Warehouse and so it too will be a Multi-purpose robot and we'll eventually Have it doing palletizing or De-palletizing or loading trucks or Unloading trucks There's definitely thousands of robots There's probably tens of thousands of Robots of this in in the future so it's Going to be profitable can you describe What stretch looks like it looks like a Big strong uh robot arm on a mobile base The base is about the size of a pallet And we want it to be the size of a Palette because that's what lives in
Warehouses right pallets of goods Sitting everywhere so we needed to be Able to fit in that space it's not a Legged mobile not a legged robot so it Was our first It was actually Um A bit of a Uh commitment from us a challenge for us To build a non-balancing robot To do the much easier problem and to put To do away well because it wasn't you Know it wasn't going to have this Balance problem and in fact the very First version of the logistics robot we Build was a balancing robot and that's Called handle and there's that thing was Epic oh it's a beautiful machine it's an Incredible machine so it was uh I mean it looks epic it looks like a out Of a uh I mean out of the sci-fi movie Of some sorts I mean just can you Actually just Linger on the like the Design of that thing because that's Another leap into something you probably Haven't done it's a different kind of Balancing yeah so let me I I'd love I Love talking about the history of how a Handle came about because it connects All of our robots actually So Um I'm going to start with Atlas when we When we had Atlas getting fairly far Along we wanted to understand I was
Telling you earlier the challenge of the Human form is you have this Mass up high And balancing that inertia that mass up High is its own unique Challenge and so We started trying to get atlas to Balance standing on one foot like on a Balance beam using its arms like this And yeah you can do this I'm sure I can Do this right like if you're walking a Tightrope How do you do that balance So that's sort of you know controlling The inertia controlling the momentum of The robot We were starting to figure that out on Atlas And so our first concept of handle which Was a robot that was going to be on two Wheels so it had to balance But it was going to have a big long arm So it could reach a box at the top of a Truck And it was gonna it needed yet another Counterbalance a big tail to help it Balance while it was using its arm So the reason why this robot sort of Looks epic it some people said it looked Like an ostrich or maybe an ostrich Moving around Was the wheels the Le it has legs so it Can extend its legs So it's wheels on legs we always wanted To build wheels on legs it had a tail And it had this arm and they're all
Moving simultaneously and in Coordination to maintain balance because We had figured out the mathematics of Doing this momentum control how to Maintain that balance and so part of the Reason why we built this two-legged Robot was we had figured this thing out We wanted to see it in this kind of Machine and we thought maybe this kind Of machine would be good in a warehouse And so we built it and it's a beautiful Machine it moves in a graceful way like Nothing else we've built but it wasn't The right machine for a logistics Application we decided it was too slow And couldn't pick boxes fast enough Basically and it was doing beautifully With Elegance but it just wasn't Efficient enough uh so we let it go yeah But I think we'll come back to that Machine eventually the fact that it's Possible the fact that he showed that You could do so many things at the same Time in coordination that's so Beautifully there's something there yeah That was a demonstration of what is Possible Basically we made a hard decision and This was really kind of a hard-nosed Business decision it was it was it Indicated us Not doing it just for the beauty of the Mathematics or the Curiosity but no we Actually need to build a business that
That could make money in the long run And so we ended up building stretch Which has a big heavy base with a giant Battery in the base of it that allows it To run for two two shifts 16 hours worth Of operation And that big battery is sort of helps it Stay balanced right so you can move a 50 Pound box around with its arm and not Tip over um It's omnidirectional it can move in any Direction so it has a nice suspension Built into it so it can deal with You know gaps or things on the floor and Roll over it but it's a that it's not a Balancing robot it's a mobile robot arm That can work to carry it or pick or Place a box up to 50 pounds anywhere in The warehouse From take a box from point A to point B Anywhere yeah palletize de-palletize We're starting with unloading trucks Because there's so many trucks and Containers that where goods are shipped And it's a brutal job you know in the Summer it can be 120 degrees inside that Container people don't want to do that Job Um and it's back breaking labor right Again these can be up to 50 pound boxes Um and so We feel like this is a productivity Enhancer and for the people who used to Do that job unloading trucks
They're actually operating the robot now And so by building robots that are easy To control And it doesn't take an advanced degree To manage You can become a robot operator and so As we we've introduced these robots to Both DHL and mariskin Gap the warehouse Workers who were doing that that manual Labor are now the robot operators and so We see this as ultimately a benefit to Them as well Can you say how much stretch cost Um not yet uh but I will say that uh we When we engage with our customers They'll be able to see a return on Investment in typically two years okay So that's something you're constantly Thinking about how yeah and I suppose You have to do the same kind of thinking With spot so it seems like we've Stretched the application is Like directly obvious yeah to slam dunk Yeah and so you can you have a little More flexibility well I think we know The target we know what we're going After yeah and with spot it took us a While to figure out what we were going After well let me return to that Question about uh Maybe the conversation you were having A while ago with Larry Page maybe Looking to the longer future Of uh social robotics of using spot to
Connect with human beings perhaps in the Home do you see a future there if we Were to sort of Hypothesize or dream about a future Where a spot like robots are in the home As pets a social robot we definitely Think about it and and we would like to Get there Uh we think the pathway to getting there Is you know likely through these Industrial applications And then Mass manufacturing you know Let's figure out what how to melt how to Build the robots how to make the Software so they can really do a broad Set of skills that's going to take Real investment to get there performance First right the principle of the company Has always been really make the robots Do useful stuff And so you know the the Social robot companies that tried to Start Someplace Else by just making a Cute interaction mostly they haven't Survived And so we think the utility Really needs to come first and that Means you have to Heart solve some of These hard problems and so to get there We're going to go through the design and Software development in industrial and Then that's eventually going to let you Reach a scale that could then be Addressed to a commercial a consumer
Level Market And so yeah maybe we'll be able to build A smaller spot with an arm that could Really go get your beer for you But there's things we need to figure out Still how to safely really safely if You're going to be interacting with Children you better be safe and right Now we we count on a little bit of Standoff distance between the robot and People so that you don't pinch a finger You know in the robot So you've got a lot of things you need To go solve before you jump to that Consumer level product well there's a Kind of trade-off in safety because it Feels like in the home You can fall Like you're you don't have to be as good At like you're allowed to fail in Different ways in more ways as long as It's safe for the humans so it just Feels like an easier problem to solve Because it feels like in the factory You're not allowed to fail That may be true Um but I also think the variety of Things a consumer level robot would be Expected to do will also be quite broad Yeah and they're going to want to get The beer and know the difference between The beer and a Coca-Cola or my snack And or you know they're all going to Want to clean up the dishes
Uh you know from the table without Breaking them Those are pretty complex tasks and so There's there's still work to be done There so to push back on that here's What application I think they'll be very Interesting I think the application of Being a pet a friend so like no tasks Foreign Just be cute because I not cute not cute Like that dog is more a dog is more than Just cute a dog is a friend is a Companion there's something about just Having interacted with them and maybe Because I'm hanging out alone with robot Dogs a little too much but like there's A There's a connection there and it feels Like that connection is not Should not be disregarded No it should Not be disregarded Robots that can somehow communicate Through their physical gestures are You're going to be more attached to in The long run do you remember IBO the Sony IBO yeah they sold over a hundred Thousand of those maybe 150 000. You know what probably wasn't considered A Successful product for them They suspended that eventually and then They brought it back so he brought it Back And people definitely you know treated
This as a as a pet as a as a companion Um and I think that will come around Again Um will you get away without having any Other utility Maybe in a world where we can really Talk to our simple little pet because You know chat GPT or some other Generative AI has made it possible for You to really talk and what seems like a Meaningful way maybe that'll open the Social robot Up again Um That's probably not a path we're going To go down because again we're so Focused on performance and utility We can add those other things also but We really want to start from that Foundation of utility I think yeah but I I also want to predict that you're wrong On that so which is that the very path You're taking which is creating a great Robot platform will very easily take a Leap to adding uh Capability maybe gpt5 and there's just So many open source Alternatives you Could just plop that on top of spot and Because you have this robust platform And you're figuring out how to mass Manufacture it and how to drive the cost Down and how to make it you know Reliable all those kinds of things it'll Be a natural transition to where just
Adding tragic Beauty all right I do Think that Being able to verbally Converse or even Converse through through gestures you Know part of part of these learning Models is that you know you can now Look At video and image imagery and Associate You know intent with that those will all Help in the communication between robots And people for sure and that's going to Happen obviously more quickly than any Of us were expecting I mean what else do You want from life Should be here And then just talk about this the State of the world I mean there's a deep loneliness within All of us and I think Uh a beer and a good chat solves so much Of it or take takes us a long way to Solving uh it'll be interesting to see Um you know when When a generative AI can give you that Warm feeling that you connected You know and that oh yeah you remember Me you're my friend you know we have a History You know that history matters right Memory of joint like memory of yeah Having witnessed I mean that's what Friendship that's what connection that's What love is in in many cases some of The deepest friendships you have is Having gone through a difficult time
Together and having a shared memory of An amazing time or a difficult time And kind of That memory creating this like Foundation based on which you can then Experience the world together the silly The mundane stuff of day-to-day is Somehow built on a foundation of having Gone through some in the past and The the current systems are not Personalized in that way but I think That's a technical problem not uh some Kind of fundamental limitation so I'll Combine that with an embodied robot like Spot which already has magic in its Movement I think uh it's a very Interesting possibility of what where That takes us but of course you have to Build that on top of a company that's Making money With real application with real Customers and with robots that are safe And at work and reliable and uh in Manufactured scale And I think we're in a unique position In that uh because of our investors Primarily Hyundai but also SoftBank Still owns 20 of us Um they don't they're not totally Fixated on Driving us to profitability as soon as Possible that's not the goal the goal Really is a longer term vision of Creating
You know what does mobility mean in the Future what are how is this mobile robot Technology going to influence Um us can we and can we shape that and They want both and so I we are as a Company are trying to strike that Balance between let's build a business That makes money I've been describing that to my own team As Uh self-destination if I want to if I Want to drive my own ship we need to Have a business that's profitable in the End otherwise somebody else is going to Drive the ship for us So that's really important But we're going to retain The aspiration that we're going to build The next generation of Technology at the Same time and the real trick will be if We can do both Uh speaking of ships Uh let me ask you about a competitor And somebody's become a friend So Elon Musk Contessa have announced Have been in the early days of building A humanoid robot how does that change The landscape of Of your work so so there's sort of from The outside perspective it seems like Well from a fan as a fan of Robotics it Just seems exciting all right very Exciting right when when Elon speaks People listen
And so uh it suddenly brought a bright Light onto the work that we've been Doing you know for over a decade And Um and I think that's only going to help And in fact what we've seen is that uh In addition to Tesla uh we're seeing a Proliferation of uh robotic companies Arise including humanoid yes oh well Yeah so and um interestingly many of Them uh as they're you know raising Money for example will claim whether or Not they have a former Boston Dynamics Employee on their staff as a criteria Yeah that's true that's uh I I I would Do that as a company yeah for sure yeah So it shows you're legit yeah so you Know what it's bring it has brung a Tremendous validation to what we're Doing And excitement uh competitive juices are Flowing you know the whole thing so uh It's all good Eli is also Kind of Stated That uh You know maybe he implied that the Problem is solvable in your term which Is a low-cost humanoid robot that's able To do that's a relatively General use Case robot So um I think Elon is known for sort of Setting these kinds of incredible
Ambitious goals Um maybe missing deadlines but actually Pushing not just the particular team he Leads but the entire world To like accomplishing those do you see You see Boston Dynamics in the near Future being pushed in that kind of way Like this excitement of competition kind Of Um Pushing Atlas maybe to uh do more cool Stuff trying to drive the cost of Atlas Down perhaps or um I mean I guess I want To I want to ask if there's Some kind of exciting Uh energy in Boston Dynamics uh due to This a little bit of competition oh yeah Definitely Um When we released our most recent video Of Atlas you know I think you'd seen it The scaffolding and throwing the box of Tools around and then doing the flip at The end yeah we were trying to show the World that not only can we do this Parkour Mobility thing but we can pick Up and move heavy things Because uh if you're going to work in a Manufacturing environment that's what You got to be able to do and for the Reasons I explained to you earlier it's Not trivial to do so you know changing The center of mass uh uh you know by Picking up a 50 pound
Black you know for a robot that weighs 150 pounds That's a lot to accommodate so we're Trying to show that we can do that and Um so it's totally been energizing you Know we see The next phase of Atlas being more Dexterous hands that can manipulate and Grab more things that we're going to Start by moving big things around that Are heavy and that affect balance and Why is that well really tiny dexterous Things probably are going to be hard for A while yet and you know maybe you could Go build a special purpose Robot arm you know for for stuffing you Know chips into Electronics boards but We don't really want to do really fine Work like that I think more coursework Where you're using two hands to pick up And balance an unwieldy thing maybe in a Manufacturing environment maybe in a Construction environment those are the Things that we think robots are going to Be able to do with the level of Dexterity that they're going to have in The next few years and that's the that's Where we're headed and and I think and You know Elon has seen the same thing Right he's talking about using the Robots in a manufacturing environment we Think there's something very interesting There about having this two-armed robot Because when you have two arms you can
Transfer a thing from one hand to the Other you can turn it around you know You can you can reorient it in a way That you can't do it if you just have One hand on it and so there's a lot that Extra arm brings to the table so I think In terms of mission You you mentioned Boston and Amex really Wants to see what are the what's the Limits of what's possible and so the Cost comes second Or it's a component but first figure out What are the limitations I think with Elon he's really dragging the cost down Is there some inspiration some lessons You see there Um of the challenge of driving the cost Down especially with Atlas with a Humanoid robot well I think the thing That he's certainly been learning by Building car factories is what that Looks like in scaling Um by scaling you can get efficiencies That drive costs down very well and uh And the smart thing that you know they Have in their favor is that you know They know how to manufacture they know How to build electric motors they know How to build uh you know computers and Vision systems so there's a lot of Overlap between modern Automotive companies And robots But hey
We have a modern robotic I mean that Automotive company behind us as well So uh bring it on who's doing pretty Well right the electric vehicles from Hyundai are doing pretty well I love it uh so how much so we've talked About some of the low level controls Some of the incredible stuff that's Going on and basic perception But how much do you see in currently in The future of Boston Dynamics sort of More higher level machine learning Applications do you see customers adding On those capabilities or do you see Boston Dynamics doing that in-house Some kinds of things we really believe Ought are probably going to be More broadly available maybe even Commoditized You know using a machine learning like a Vision algorithm so a robot can Recognize something in the environment That ought to be something you can just Download like I'm going to a new Environment I have a new kind of door Handle or piece of equipment I want to Inspect you ought to be able to just Download that and I think people besides Boston Dynamics will provide that and We've actually built an API that lets People add these uh Vision algorithms to Spot And we're currently working with some Partners who are providing that
Um levitas is an example of a small Provider who's giving us software for Reading gauges Um and actually another partner in Europe reply is doing the same thing So we see that we see It ultimately uh An ecosystem of providers doing stuff Like that I and I I think ultimately You might even be able to do the same Thing with behaviors so this technology Will also be brought to bear on Controlling the robot the the Motions of The robot and you know we're using Learning reinforcement learning to Develop Algorithms for both Locomotion and Manipulation And ultimately this is going to mean you Can you can add new behaviors to a robot You know quickly And uh that could potentially be done Outside of Boston Dynamics right now That's all internal to us I think I Think you need to understand At a deep level you know Um the robot control to do that but Eventually that could be outside but It's certainly a place where these these Approaches are going to be brought to Bear in robotics so reinforcement Learning is part of the process today so You do use reinforcement learning yes Foreign So there's uh increasing levels of
Learning with these robots yes And that's for both for uh for Locomotion for manipulation and For perception yes Well what do you think in general about The all the exciting advancements of uh Transformer uh neural networks Most Beautifully uh Illustrated through the Large language models like gpt4 Like everybody else we're all you know I'm I'm surprised at uh how much uh how Far they've come Um I'm a little bit nervous about the There's anxiety around them obviously For I think good reasons right Disinformation is a a curse that's a an Unintended consequence of social media That could be Exacerbated with these tools so if you Use them to deploy this information it Could be a real risk Um but I also think that the risks Associated with these kinds of models Don't have a whole lot to do with the Way we're going to use them in our Robots if I'm using a robot I'm building a robot to do you know a Manual task of some sort Um I can judge very easily is it doing The task I asked it to is it doing it Correctly there's sort of a a built-in Mechanism for judging is that is it Doing the right thing did it
Successfully do the task yeah physical Reality is a good verifier it's a good Verifier that's exactly it whereas if You're asking for yeah I don't know You're trying to ask a theoretical Question in chat GPT It could be true or it may not be true And it's hard to have that verifier what What is that truth that you're comparing Against whereas in physical reality you Know the truth And this is an important difference and Um so I'm not I think there is reason to be a little Bit concerned about Um you know how these tools large Language models could be used but I'm Not very worried about how they're going To be used Well how learning algorithms in general Are going to be used on robotics it's It's really a different application That has different ways of verifying What's going on well the nice thing About language models is that I Ultimately see I'm really excited about the possibility Of having conversations with spot yeah Uh there's no I would say negative Consequences to that but just increasing The bandwidth in the variety of ways you Can communicate with this uh particular Robot so you could communicate visually You can communicate through some
Interface and to be able to communicate Verbally again with the beer and so on Um I think that's really exciting to Make that much much easier we have this Partner levitas that's adding the vision Algorithms for daydreading for us they Just just this week I saw a demo where They hooked up you know a language tool To spot and they're talking a spot to Give example yeah can you tell me about The Boston Dynamics AI Institute what is It and what is its mission So it's a separate organization uh the Boston Amex artificial intelligence Institute And it's led by Mark Rayburn the founder Of Boston Dynamics and the former CEO And my old advisor at MIT Mark has Always loved the research the pure Research without the confinement or Demands of commercialization And uh he wanted to continue to you know Pursue that unadulterated research And uh uh so uh suggested uh to Hyundai That that he set up this institute and They agree that it's worth additional Investment to kind of continue push Pushing this Forefront And we expect to be working together Where you know Boston Dynamics is again Both commercialize and do research but The sort of time Horizon of the research We're going to do is you know in the Next let's say five years you know what
Can we do in the next five years let's Work on those problems And I think the goal of the AI Institute Is to work even further out Um certainly you know the analogy of of Legged Locomotion again when we started That that was a multi-decade problem and And so I think Mark wants to have the Freedom Uh to pursue really hard Over the Horizon problems and that's that'll be The goal of the Institute So we mentioned uh some of the dangers Of uh some of the concerns about large Language models that said uh you know There's been a long-running fear of These embodied Robots uh why do you think people are Afraid of uh Lincoln robots yeah I Wanted to show you this this So this this is a Wall Street Journal And this is all about chat GPT right but Look at the picture yeah it's a humanoid Robot That's saying I will say that looks Scary and it says I'm going to replace You yeah and so the humanoid robot is Sort of is the embodiment of this chat GPT tool that there's reason to be a Little bit nervous about how it gets Deployed yeah so I'm nervous about that Connection Um It's unfortunate that they chose to use
A robot as that embodiment for as you And I just said there's big differences In in this But uh people are afraid because we've Been Taught to be afraid for over a hundred Years so you know the word robot was Developed by a playwright named Carol Chapek in 1921 to check a playwright for Awesome's Universal robots and in that First depiction of a robot the robots Took over The end of the story And you know people love to be afraid And so we've been entertained by these Stories for a hundred years But I And I think that's as much why people Are afraid as anything else as we've Been sort of taught that this is the Logical progression through fiction Um I think it's fiction I think uh what people more and more Will realize just like you said That the threat like say you have a Super intelligent AI embodied in a robot That's much less threatening because It's visible it's verifiable it's right There in physical reality and we humans Know how to deal with physical reality I Think it's much scarier when you have Arbitrary scaling of Intelligent AI systems in the digital
Space That they could uh pretend to be human So robot spot is not going to be pretend It could pretend it's human all at once You can tell you you could put Chad gbt On top of it but you're gonna know it's Not human because you have a contact With physical reality and you're going To know whether or not it's doing what You asked it to do yeah like it's not Gonna like if it like I mean I'm sure You can start just like a dog lies to You it's like I didn't I wasn't part of Tearing up that couch try to lie that like you know it Wasn't me that spilled that thing but It's you're going to kind of figure it Out eventually it's but if it happens Multiple times you know Uh but I think that Humanity has figured Out how to make machines safe yeah and There's you know there's regulatory Environments and certification uh Protocols that we've developed in order To figure out how to make machines safe We don't know we and don't have that Experience with software That can be propagated worldwide in an Instant And so I think we needed to develop Those protocols and those tools and so Uh That's work to be done but I don't think The fear of that and that work should
Necessarily impede our ability to now Get robots out because again I think I Think we can judge when a robot's being Safe So and again just like in that image There's a fear that robots will take our Jobs I just um I took a ride I was in San Francisco I know way more Vehicles Than autonomous vehicle and uh I've done It several times they're they're doing Credible work over there uh but Uh people flicked it off the car so I Mean that's a long story of what the Psychology of that is it could be maybe Big Tech or what I I don't know exactly What they're flicking off yeah but there Is an element of like these robots are Taking our jobs or or Irreversibly transforming Society such That it will have economic impact and The little guy will be uh would lose a Lot would lose their well-being is there Something to be said about Um The fear that robots will take our jobs You know at every Um significant technological Transformation uh there's been fear of You know an automation anxiety yes that Uh it's it's going to have a broader Impact than than we expected And there there will be uh You know jobs will change Um sometime in the future we're going to
Look back at people who manually Unloaded these boxes from trailers and We're going to say why did we ever do That manually but there's a lot of People who are doing that job today That it could be impacted Um but I think the reality is as I said Before we're going to build a Technologies those very same people can Operate it and so I think there's a Pathway to upskilling and operating just Like look we used to farm with hand Tools and now we farm with machines and Nobody has really regretted that Transformation And I think the same can be said for a Lot of manual labor that we're doing Today And on top of that you know look we're We're entering a new world where where Demographics are going to have a strong Impact on economic growth in the you Know the advanced uh the first world is Losing population quickly Um in Europe They're worried about hiring enough People just to keep the logistics supply Chain going And you know part of this is the Response to covid and everybody's sort Of thinking back What they really want to do with their Life but these jobs are getting harder And harder to fill and I just I'm
Hearing that over and over again So I think frankly this is the right Technology at the right time Um where we're gonna need some of this Work to be done and we're going to want Tools to enhance that productivity and The scary impact I think again uh gbt Comes to the rescue in terms of being Much more terrifying Um The scary scary impact of basically so I I'm a I guess a software person inside a Program a lot and the fact that people Like me could be easily replaced Uh by gbt That's going to have a Well and a lot you know anyone who deals With texts and writing a draft proposal Might be easily done with a chat GPT now Consultants Journalists yeah Um everybody is sweating but on the Other hand you also want it to be right And they don't know how to make it right Yet it but it might make a good starting Point for you to iterate boy do I have To talk to you about modern journalism That's another conversation altogether But yes uh more right than the average Uh Um the the mean journalist yes Um You spearheaded the NT weaponization Letter
Uh Boston Dynamics has can you Describe uh what that letter States and The general topic of the use of robots In war We Authored a letter and then got several Leading robotics companies around the World including you know unitary and China and Um agility here in the United States and Um animal and in Europe And you know some others to co-sign a Letter that said we won't put weapons on Our robots And part of the motivation there is you Know as these robots start to become Commercially available You can see videos online of people Who've gotten a robot and strapped a gun On it and shown that they can you know Operate the gun remotely while driving The robot around And so having a robot that has this Level of Mobility And that can easily be configured in a Way that could harm somebody from a Remote operators Justifiably a scary thing And so we felt like it was important to Draw a bright line there and say we're Not going to allow this For Um You know reasons that we think
Ultimately it's better for the whole Industry if if it grows in a way where Uh the robots are ultimately going to Help us all and make our lives more Fulfilled and productive but by goodness You're going to have to trust the Technology to let it in and if And if you think the robot is going to Harm you that's gonna that's gonna hurt And impede the growth of that industry So we thought it was important to to Draw a bright line And uh and then publicize that and and Our plan is to you know begin to engage With Lawmakers and Regulators let's figure Out what the rules are going to be Around the use of this technology Um and use our position as leaders in This industry and Technology To help force that issue And so we are in fact I have a policy You know director at my company whose Job it is to engage with the public to Enjoy engage with interested parties and Including Regulators to sort of begin These discussions He has a really important topic and it's An important topic for people that worry About the impact of robots on our Society with autonomous weapon systems So I'm glad you're sort of leading the Way in this uh You are the CEO of Boston Dynamics uh
What's it take to be SEO of a robotics Company she started as a humble engineer Uh PhD Um just looking at your journey What does it take to go from being From building the thing To Leading a company what are some of The big challenges for you Uh courage I would I would put front and Center for multiple reasons I talked Earlier about the courage to tackle hard Problems So I think there's courage required not Just of me but of of all of the people Who work at Boston Dynamics Um I also think we have a lot of really Smart people we have people who are way Smarter than I am And I it takes a kind of courage to be Willing to lead them And to trust that You have something to offer to somebody Who probably is maybe a better engineer Than I am Um Adaptability you know part of the it's Been a great career for me I never would Have guessed I'd stayed in one place for 30 years Um and the job has always changed Um I didn't I didn't really aspire to be CEO from the very beginning but it was The natural progression of things there Was always a there always needed to be
Some level of management that was needed And so You know when I saw Something that needed to be done that Wasn't being done I just stepped in to Go do it and oftentimes because we were Full of such uh strong Engineers Oftentimes that was in the management Direction or it was in the business Development direction or or Organizational hiring geez I was not I Was the main person hiring at Boston Dynamics for probably 20 years so I was The head of HR basically So I you know just willingness to sort Of tackle any piece of the business that That needs it and then and be willing to Shift is there something you could say To what it takes to hire a great team What uh What's a good interview process how do You know uh the guy or Gallery are going To make a great member of uh of a Engineering team that's doing some of The hardest work in the world You know we developed an interview Process that I was quite fond of it's a Little bit of a hard interview process Because the best interviews you ask Somebody about what they're interested In and what they're good at And if if they can describe to you Something that they worked on and you Saw they really did the work they solved
The problems And you saw their passion for it Um and you could ask but but what makes That hard is you have to ask a probing Question about it you have to be smart Enough about what they're telling you Their expert at to ask a good question And so it takes a pretty talented team To do that But if you can do that that's how you Tap into ah this person cares about Their work they really did the work They're excited about it that's the kind Of person I want at my company You know at Google they taught us about Their interview process And it was a little bit different Um You know We we evolved the process at Boston Dynamics where it didn't matter if you Were an engineer or you were an uh Administrative assistant or a financial Person or a technician You gave us a presentation you came in And you gave us a presentation you had To stand up and talk in front of us And I just thought that was great to tap Into those things I didn't subscribed to You at Google they taught us and I think I understand why you're right they're Hiring tens of thousands of people they Need a more standardized process so they Would sort of err on the other side
Where they would ask you a standard Question I'm going to ask you a Programming question and I'm just going To ask you to write code in front of me That's a terrifying you know application Process yeah it does let you compare Candidates really well but it doesn't Necessarily let you tap in to who they Are yeah right because you're asking Them to answer your question instead of You asking them about what they're Interested in Um but frankly that process is hard to Scale and even at Boston Dynamics We're not doing that with everybody Anymore we're but we are still doing That with you know the technical people But we've because we too now need to Sort of increase our rate of hiring uh Not everybody's giving a presentation Anymore but you're still ultimately Trying to find that uh basic seed of Passion yeah into the world you know did They really do it did they did they find Something interesting or curious you Know Um and do they care about it I think uh Somebody admires Um Jim Keller and he He likes details So one of the ways you could if you get A person to talk about what they're Interested in how many details like how Much of the Whiteboard can you fill out
Yeah well I think you figure out did They really do the work if they know Some of the details yes and if they have To wash over the details well then they Didn't do it especially with engineering The work is in the details yeah I have to go there briefly Just to get your kind of thoughts in the Long-term future of robotics There's been discussions on the GPT side On the large language model side of Whether there's Consciousness inside These language models And I think there's fear but I think There's also excitement or at least the Wide world of opportunity and Possibility in embodied robots having Something like Let's start with emotion Love towards other human beings And uh perhaps the display real or fake Of consciousness Is this something you think about in Terms of long long term future because As you as we've talked about people do Anthropomorphize these robots uh it's Difficult not to project some level of I Use the word sentience some level of Sovereignty identity all the things we Think is human that's what Anthropomorphization is as we project Humanness onto mobile especially legged Robots Um is that something almost from a
Science fiction perspective you think About or do you try to avoid ever You try to avoid the topic of Consciousness altogether I'm certainly not an expert in it and I Don't spend a lot of time thinking about This right Um and I do think it's fairly remote for The machines that that we're dealing With Um Our robots you're right the people Anthem warfies they they read into the Robot's intelligence and emotion that Isn't there because they see physical Gestures That are similar to things they might Even see in people or animals Um I don't know much about how these Large language models really work I I Believe it's a kind of statistical Averaging of the most common responses You know to a series of words right it's It's sort of um A very elaborate uh word completion Um and I'm dubious that that has Anything to do with consciousness Um and I I even wonder if that model of Sort of simulating Consciousness by by Stringing words together that are Statistically associated with one Another Um whether or not that kind of knowledge If you want to call that knowledge
Um would be the kind of knowledge that Allowed a sentient being to grow or Evolve it feels to me like there's There's something about Truth or emotions That's just a very different kind of Knowledge that is that is absolute like A the interesting thing about truth is It's absolute and it doesn't matter how Frequently it's represented in the world Wide web it If you know it to be true it can only be It may only be there once but by God That's true And I think emotions are a little bit Like that too you know something you Know and And I just think that's a different kind Of knowledge than the way these large Language models uh derive sort of Simulated it does seem that intelligence Things that are true Um very well might be statistically well Represented on the on the internet Because the internet is made up of Humans So I tend to suspect that large language Models are going to be able to simulate Consciousness very effectively and I Actually believe that current gpt4 when Fine-tuned correctly would be able to do Just that and that's going to be a lot Of very complicated ethical questions That have to be dealt with that have
Nothing to do with robotics they have And everything to do with there needs to Be some process of Labeling I think what is true Because because there is also Disinformation available on the web and These models are going to consider that Kind of information as well And again you can't average something That's true and something that's untrue And get something that's moderately true It's either right or it's wrong and so How how is that process and and this is Obviously something that The purveyors of these Bard and chat gbt That I'm sure this is what they're Working on well if you interact on some Controversial topics with these models They're actually refreshingly nuanced They present because there's there's Well you realize there's no one truth You know uh what caused The war in Ukraine right any Geopolitical conflict you can ask any Kind of question especially the ones That are politically uh a a tense Divisive and so on GPT is very good at Presenting here's the like here's the App it presents the different hypotheses It presents calmly so the the amount of Evidence for each one it's very it's Really refreshing it makes you realize That truth is nuanced and it does that Well and I think with consciousness
It would very uh accurately say Well it Sure as hell feels like I'm one of you Humans but where's my body I don't Understand like you're going to be Confused the cool thing about GPT Is it seems to be easily confused in the Way we are like you wake up in a new Room and you ask where am I it seems to Be able to do that extremely well it'll Tell you one thing like a fact about When a war started and when you correct It say well this isn't it's not Consistent it'll be confused it'd be Yeah you're right It'll have that same element childlike Element with humility of trying to Figure out its way in the world and I Think That's a really tricky area to uh to Sort of figure out with us humans of What we want Um to allow AI systems to say to us Because then if there's elements of Sentience that are being on display you Can then start to manipulate human Emotion all that kind of stuff but I Think That's that's something that's a really Serious and aggressive discussion that Needs to be had On the software side Um I think again embodiment uh Robotics are actually saving us from the Arbitrary scaling of software systems
Versus uh creating more problems but That said I I really believe in the that Connection between human and robot There's magic there And I think uh there's also I think a Lot of money to be made there and Boston Dynamics is leading the world in Um the most elegant movement Done by robots So I I can't wait thank you to uh what Maybe other people that built on top of Uh Boston Dynamics robots or boss or Boston Dynamics by itself so you had uh One wild career one place and one Set of problems but incredibly Successful can you give advice to Young Folks today in high school maybe in College looking out into this future Where So much Robotics and AI seems to be Defining the trajectory of human Civilization can you give them advice on How to have a career they can be proud Of Or how to have a life they can be proud Of Well I would say uh you know follow your Heart and your interest you know what Again this was an organizing principle I Think behind The light Lab at MIT that that turned Into A value at Boston Dynamics which was Follow your curiosity
Love what you're doing Um you'll have a lot more fun and you'll Be a lot better at it as a result Um I think it's hard to plan you know don't Get too hung up on planning too far Ahead find things that you like doing And then see where it takes you you can Always change direction you will find Things that you know that wasn't a good Move I'm gonna back up and go do Something else So when people are trying to plan a Career I always feel like yeah there's a Few happy mistakes that happen along the Way and just live with that it you know But just But make choices then so Avail Yourselves to these interesting Opportunities like when I happen to run Into mark down in the lab the basement Of the AI lab But be willing to make it A decision and then pivot if you see Something exciting to go at you know Because if you're out and about enough You'll you'll find things like that that Get you excited so there was a feeling When you first met Mark and saw the Robots that there's something Interesting oh boy I gotta go do this There is no doubt What do you think in the 100 years What do you think Boston Dynamics is
Doing what do you think is the role even Bigger what do you think is the role of Robots in society do you think we'll be Seeing Billions of robots Everywhere Do you think about that long-term vision Well I do think that um I think the robots will be ubiquitous And they will be out amongst us Um And they'll be Certainly doing you know some of the Hard labor that we do today I don't think people don't want to work People want to work people need to work To I think feel productive we we don't Want to offload all of the work to the Robots because I'm not sure if people Would know what to do with themselves And I think just self-satisfaction and Feeling productive is such an ingrained Part of being human that we need to keep Doing this work so we're definitely Going to have to work in a complementary Fashion and I hope that the robots and The computers don't end up being able to Do all the creative work right because Because that's the part that's you know That's the rewarding the creative part Of solving a problem is the thing that Gives you That serotonin Rush that you never Forget you know or that Adrenaline Rush
That you never forget and so you know People need to be able to do that Creative work and and just feel Productive and sometimes that you can Feel productive over fairly simple work It's just well done you know and then You can see the result of So I yeah you know there is a I don't know there's a a cartoon was it Wally where they had this big ship and All the people were just Overweight lying on their best chairs Kind of sliding around on the deck of The of the movie because they didn't do Anything anymore yeah well we definitely Don't want to be there you know we need To work in some complimentary fashion Where we keep all of our faculties and Our physical health and we're doing some Labor right but in a complimentary Fashion somehow and I think a lot of That has to do with the interaction the Collaboration with robots and with AI Systems I'm hoping there's a lot of Interesting possibilities I think that Could be really cool right if you can if You can work in a comp internet Interaction and really be be helpful Robots you You know you can ask a robot to do a job You wouldn't ask a person to do and that Would be a real asset you wouldn't feel Guilty about it you know you'd say just Do it it's a machine I and I don't have
To have qualms about that you know the Ones that are machines I also hope to See a future And it is hope I do have optimism about That future where some of the robots are Pets have an emotional connection to us Humans and uh because one of the Problems that humans have to solve is This kind of a general loneliness um The more love you have in your life the More friends you have in your life I Think that makes a more enriching life Helps you grow and I don't fundamentally See why some of those friends can't be Robots there's an interesting Long-running study maybe it's an Harvard They just nice report article written About it recently they've been studying This group of a few thousand people now For 70 or 80 years and the conclusion is That Companionship and friendship are the Things that make for a better and Happier life And Um So I agree with you and I think that Could happen with a machine that is Probably you know simulating Intelligence I'm not convinced there Will ever be true intelligence in these Machines Sentience but they could simulate it and They could collect your history and they
Could you know I guess it remains to be Seen whether they can establish that Real deep you know when you sit with a Friend and they remember something about You and bring that up and you feel that Connection it remains to be seen if a Machine's going to be able to do that For you well I have to say it's inklings Of that already started happening for me Some of my best friends are robots Uh and I have you to thank for leading The way in in the accessibility and the Ease of use of such robots and the Elegance of their movement uh Robert You're an incredible person Boston Dynamics is an incredible company I've Just been a fan for many many years for Everything you stand for for everything You do in the world if you're interested In great engineering robotics go join Them build cool stuff I'll forever Celebrate the work you're doing and it's Just a big honor that he was sit with me Today and talk it means a lot so thank You so much he's doing great work thank You Lex I'm honored to be here and uh I Appreciate it was fun Thanks for listening to this Conversation with Robert later to Support this podcast please check out Our sponsors in the description and now Let me leave you some words from Alan Turing in 1950 defining what is now Termed the Turing test
A computer would deserve to be called Intelligent if it could deceive a human Into believing that it was human Thank you for listening and hope to see You next time