can you tour boston dynamics

• Watch Videos
• Meet Our Expert Guests
• Inside An Architect’s Office
• Showroom Feature: Focus on Fulton
• Shop the Marketplace
• In-Depth Product Profiles
• Chair Reviews
• Product Roundups
• Expert Insights
• Tips & Trends

Sign Up For Our Newsletter

• Advertise With Us

Popular Articles

The future face of facade design, designing for neurodiversity and inclusion, schuylkill yards’ west tower by cetraruddy, embracing sustainability in the workplace through furniture upcycling, chair of the month.

Bergmeyer created a new headquarters that tells the unique story of the Boston Dynamics family; robots included.

A robotics laboratory is a unique kind of place. At its core is a near-magical process: the invention, testing, and creation of the animated machines that will shape the 21st century. Boston Dynamics – a world-renowned robotics engineering firm – wanted everyone who entered their new headquarters to be not just visually but also viscerally connected to this creative process; a process that very much paralleled the design approach Bergmeyer brought to this assignment. To make these connections work and capture the company’s innovative, open, and dynamic culture, we had to draw deeply from our history of using design to tell stories.

Creating a corporate headquarters that would support Boston Dynamics’ rapidly growing engineering, research and development, and customer support teams while not losing sight of what is at the core of their culture – collaboration, wellness, and connectivity – was one of the primary objectives. The laboratories also needed to be adaptable to their ever-changing environment and durable enough to withstand the activity of the engineers and robots.

Spanning three levels, the 180,000 SF facility was built within an existing space. Leveraging the building’s sectional qualities, the mezzanine was extended to create cantilevered meeting rooms overlooking the robotics labs. Multiple laboratories were designed to support their people and creations, from fabrication, engineering, and bringing their sophisticated robots to market. Open stairs and generous exterior and interior glazing allow diagonal views and natural sunlight to crisscross into every corner of the facility.

Employee health and well-being were also a high priority for Boston Dynamics. The headquarters provides physical activity opportunities, views to the outdoors, and daylighting throughout the entire workplace. Uniquely people-centric spaces such as the lobby’s “robot gallery,” the multi-leveled Town Hall space, the kitchens and dining spaces, fitness rooms, and mothers and quiet rooms were designed with hospitality in mind to give the workplace an invitingly casual and playful feel.

The result is a home that tells the unique story of the Boston Dynamics family; robots included.

When was the project completed?  

Move in began August 2020

How many SF per person?

180,000 SF total (approximately 400 SF / per person)

How many employees work here?

80 Employees (not always the same employees) are currently on site at a time due to pandemic restrictions, but 200+ employees will work here with planned growth to 400.

Describe the work space type.

A mix of private offices, open plan workstations and private and open meeting spaces of varying sizes.

What kind of meeting spaces are provided?

There are varying size meeting rooms for a little as two people and up to 50 people. The unenclosed Think Tanks allow for informal gathering and discussions. There is also the Town Hall space that allows for scalable meetings from two people to the entire office which is anticipated to grow to 400.

What other kinds of support or amenity spaces are provided?

There are many amenities in the space including the main Food Servery located in Town Hall, strategically located Oasis Kitchenettes, Mother’s Room, Quiet Room, Private Fitness Room, Library and Outdoor Space.

Has the project achieved any special certifications?

We are anticipating LEED certification.

What is the projects location and proximity to public transportation and/or other amenities?

The site is not close to public transportation (Alewife, MA stop is six miles away), so in addition to the private servery and fitness room, there are also common food hall and fitness room for all building tenants on site.

How is the space changing/adapting as a result of the COVID-19 pandemic?

Boston Dynamics is adhering to the Commonwealth of Massachusetts’ guidelines for the pandemic and currently has only 80 of their 200+ employees on site at one time.

Was the C-suite involved in the project planning and design process? If so, how?

Yes. Along with other key team members, all executive team members were part of information gathering interviews conducted by Bergmeyer to learn more about Boston Dynamics’ history, needs for the increasing needs for the capability to host investors, immediate and future plans for growth and plans around the newly developing retail arm of the company.

What kind of programming or visioning activities were used to create the space?

In addition to the information gathering interviews that took place, Bergmeyer prepared boards of inspiration images and concepts hosted collaborative, working design sessions at our offices to gather feedback on the images and concepts, encouraging Boston Dynamics’ team to participate by “selecting” what resonated with them by placing color-coded stickers on images on the boards.

Were any pre-planning surveys conducted to get employee input?

Yes. All employees in the company took place in a survey developed by Bergmeyer to get feedback on ex. how long they had been with the company, how they used their current space, how they would better utilize a new space, and what success in the new space meant to them.

Were there any other kinds of employee engagement activities?

An all-employee event was planned by Boston Dynamics to intro everyone and to make the announcement at the new site prior to construction. Renderings in the locations where they were from and floor plans were in the space to encourage discussion and because Bergmeyer was also invited, it gave employees and opportunity to ask questions and give feedback.

Was there any emphasis or requirements on programming for health and wellbeing initiatives for employees?

Yes, this was a primary goal of Boston Dynamics. The facility provides physical activity opportunities, views to the outdoors, and daylighting throughout the entire workplace.

For specific examples, please describe the product, how it was used, and if it solved any specific problem.

Custom, steel sheets with randomized perforations used for wall material in the Lobby and Town Hall created using programs outside of Revit. Specifically, Rhino and Grasshopper.

What products or service solutions are making the biggest impact in your space?

We used DIRTT wall systems for all the private offices and meeting rooms. The walls allowed for a smoother construction schedule and will afford Boston Dynamics flexibility down the road if they choose to reconfigure or combine rooms.

What kind of branding elements were incorporated into the design?

The Boston Dynamics team wanted to make a clear announcement that this was their new home, and the team incorporated large Boston Dynamics branded signage on the building facing 128, on the building facing the parking area, and inside the lobby.

What is the most unique feature of the space?

Multi-tiered, multi-function Town Hall area that utilizes a mix of soft seating and tables on the tiers for comfort and smaller breakout discussions that provides an alternative to the stadium seating typically used in many workplaces today. The tiers extend an entire floor and encourages walking up the stairs at the center of it to get from one level to the next.

Are there any furnishings or spaces specifically included to promote wellness/wellbeing?

The entire space encourages walking and stair climbing, with natural light flooding throughout the entire space which provides visual and physical access to the outdoors.

If the company relocated to a new space, what was the most difficult aspect of the change for the employees?

The most difficult aspect was changing how certain spaces were used and utilizing the built-in organization that was established vs. the free for all that was prominent in the old space.

How did the company communicate the changes and moves?

As mentioned above, an all-employee event was planned by Boston Dynamics to intro everyone and to make the announcement at the new site prior to construction.

Who else contributed significantly to this project?

Bergmeyer – Architect, Project Lead

NV5 – MEPFP, IT, AV

Hartranft Lighting Design – Lighting Designers

Goldstein Milano – Structural Engineers

James Black Acoustics – Acoustical Consultant

Creative Office Pavilion – Furniture Supplier

Colburn Guyette – Foodservice Consultant

Anton Grassl Architectural Photography – Photo credit

• Boston Dynamics
• collaboration
• connectivity
• new project
• project profile
• robotics laboratory

More Articles

Purpose driven design – central square foundation by lyth design, a peek inside zoom’s hq by anacapa architecture, leave a reply cancel reply.

Save my name, email, and website in this browser for the next time I comment.

Latest Articles

The designer’s role amidst a climate crisis: how can we make a difference, is working near home the new working from home.

Boston Dynamics has turned its Spot quadruped, typically used for inspections, into a robot tour guide. The company integrated the robot with ChatGPT and other AI models as a proof of concept for the potential robotics applications of foundational models. 

In the last year, we’ve seen huge advances in the abilities of Generative AI, and much of those advances have been fueled by the rise of large Foundation Models (FMs). FMs are large AI systems that are trained on a massive dataset.

These FMs typically have millions of billions of parameters and were trained by scraping raw data from the public interested. All of this data gives them the ability to develop Emergent Behaviors, or the ability to perform tasks outside of what they were directly trained on, allowing them to be adapted for a variety of applications and act as a foundation for other algorithms. 

The Boston Dynamics team spent the summer putting together some proof-of-concept demos using FMs for robotic applications. The team then expanded on these demos during an internal hackathon. The company was particularly interested in a demo of Spot making decisions in real-time based on the output of FMs. 

Large language models (LLMs), like ChatGPT, are basically very capable autocomplete algorithms, with the ability to take in a stream of text and predict the next bit of text. The Boston Dynamics team was interested in LLMs’ ability to roleplay, replicate culture and nuance, form plans, and maintain coherence over time. The team was also inspired by recently released Visual Question Answering (VQA) models that can caption images and answer simple questions about them. 

A robotic tour guide seemed like the perfect demo to test these concepts. The robot would walk around, look at objects in the environment, and then use a VQA or captioning model to describe them. The robot would also use an LLM to elaborate on these descriptions, answer questions from the tour audience, and plan what actions to take next. 

In this scenario, the LLM acts as an improv actor, according to the Boston Dynamics team. The engineer provides it a broad strokes scrip and the LLM fills in the blanks on the fly. The team wanted to play into the strengths of the LLM, so they weren’t looking for a perfectly factual tour. Instead, they were looking for entertainment, interactivity, and nuance. 

Turning Spot into a tour guide

The hardware setup for the Spot tour guide. 1. Spot EAP 2; 2. Reseaker V2; 3. Bluetooth Speaker; 4. Spot Arm and gripper camera. | Source: Boston Dynamics

The demo that the team planned required Spot to be able to speak to a group and hear questions and prompts from them. Boston Dynamics 3D printed a vibration-resistant mount for a Respeaker V2 speaker. They attached this to Spot’s EAP 2 payload using a USB. 

Spot is controlled using an offboard computer, either a desktop PC or a laptop, which uses Spot’s SDK to communicate. The team added a simple Spot SDK service to communicate audio with the EAP 2 payload. 

Now that Spot had the ability to handle audio, the team needed to give it conversation skills. They started with OpenAI’s ChaptGPT API on gpt-3.5, and then upgraded to gpt-4 when it became available. Additionally, the team did tests on smaller open-source LLMs. 

The team took inspiration from research at Microsoft and prompted GPT by making it appear as though it was writing the next line in a Python script. They then provided English documentation to the LLM in the form of comments and evaluated the output of the LLM as though it were Python code. 

The Boston Dynamics team also gave the LLM access to its SDK, a map of the tour site with 1-line descriptions of each location, and the ability to say phrases or ask questions. They did this by integrating a VQA and speech-to-text software. 

They fed the robot’s gripper camera and front body camera into BLIP-2, and ran it in either visual question answering mode or image captioning mode. This runs about once a second, and the results are fed directly into the prompt. 

To give Spot the ability to hear, the team fed microphone data in chunks to OpenAI’s whisper to convert it into English text. Spot waits for a wake-up word, like “Hey, Spot” before putting that text into the prompt, and it suppresses audio when it its speaking itself. 

Because ChatGPT generates text-based responses, the team needed to run these through a text-to-speech tool so the robot could respond to the audience. The team tried a number of off-the-shelf text-to-speech methods, but they settled on using the cloud service ElevenLabs. To help reduce latency, they also streamed the text to the platform as “phrases” in parallel and then played back the generated audio. 

The team also wanted Spot to have more natural-looking body language. So they used a feature in the Spot 3.3 update that allows the robot to detect and track moving objects to guess where the nearest person was, and then had the robot turn its arm toward that person. 

Using a lowpass filter on the generated speech, the team was able to have the gripper mimic speech, sort of like the mouth of a puppet. This illusion was enhanced when the team added costumes or googly eyes to the gripper. 

How did Spot perform? 

The team gave Spot’s arm a hat and googly eyes to make it more appealing. | Source: Boston Dynamics

The team noticed new behavior emerging quickly from the robot’s very simple action space. They asked the robot, “Who is Marc Raibert?” The robot didn’t know the answer and told the team that it would go to the IT help desk and ask, which it wasn’t programmed to do. The team also asked Spot who its parents were, and it went to where the older versions of Spot, the Spot V1 and Big Dog, were displayed in the office. 

These behaviors show the power of statistical association between the concepts of “help desk” and “asking a question,” and “parents” with “old.” They don’t suggest the LLM is conscious or intelligent in a human sense, according to the team. 

The LLM also proved to be good at staying in character, even as the team gave it more absurd personalities to try out. 

While the LLM performed well, it did frequently make things up during the tour. For example, it kept telling the team that Stretch, Boston Dynamics’ logistics robot , is for yoga. 

Moving forward, the team plans to continue exploring the intersection of artificial intelligence and robotics. To them, robotics provides a good way to “ground” large foundation models in the real world. Meanwhile, these models also help provide cultural context, general commonsense knowledge, and flexibility that could be useful for many robotic tasks. 

About The Author

Brianna Wessling

Brianna Wessling is an Associate Editor, Robotics, WTWH Media. She joined WTWH Media in November 2021, and is a recent graduate from the University of Kansas. She can be reached at [email protected]

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed .

Related Articles Read More >

Formant launches analytics feature to optimize fleet performance

InOrbit introduces RobOps Copilot for mobile robot fleet insights

WiBotic unveils new high power wireless charging solution

U.S. manufacturers invested heavily in robotics in 2023, finds IFR

Search the robot report.

• Batteries / Power Supplies
• Cameras / Imaging / Vision
• Controllers
• End Effectors
• Microprocessors / SoCs
• Motion Control
• Soft Robotics
• Software / Simulation
• Artificial Intelligence
• Human Robot Interaction / Haptics
• Mobility / Navigation
• Collaborative Robots
• Self-Driving Vehicles
• Unmanned Maritime Systems
• Investments
• Mergers & Acquisitions
• Agriculture
• Manufacturing
• RBR50 Winners 2024
• RBR50 Winners 2023
• RBR50 Winners 2022
• RBR50 Winners 2021
• Realizing a Data-Driven Warehouse
• Supporting the Growth of AGVs and AMRs in the Warehouse
• Digital Issues
• Automated Warehouse
• Collaborative Robotics Trends
• Mobile Robot Guide
• Search Robotics Database
• Webinars / Digital Events
• RoboBusiness
• Robotics Summit & Expo
• DeviceTalks
• R&D 100
• Robotics Weeks

Watch CBS News

Boston Dynamics: Inside the workshop where robots of the future are being built

By Anderson Cooper

August 8, 2021 / 7:05 PM EDT / CBS News

Boston Dynamics is a cutting-edge robotics company that's spent decades behind closed doors making robots that move in ways we've only seen in science fiction films. They occasionally release videos on YouTube of their life-like machines spinning, somersaulting or sprinting, which are greeted with fascination and fear. As we first told you this past spring, we'd been trying without any luck to get into Boston Dynamics' workshop for years, and in March, they finally agreed to let us in. After working out strict COVID protocols we went to Massachusetts to see how they make robots do the unimaginable. 

From the outside, Boston Dynamics headquarters looks pretty normal. Inside, however. it's anything but. If Willy Wonka made robots, his workshop might look something like this. There are robots in corridors, offices and kennels. They trot and dance and whirl and the 200-or-so human roboticists, who build and often break them, barely bat an eye. 

That is Atlas, the most human-looking robot they've ever made.

It's nearly 5 feet tall, 175 pounds, nd is programmed to run, leap and spin like an automated acrobat. 

Marc Raibert, the founder and chairman of Boston Dynamics doesn't like to play favorites, but definitely has a soft spot for Atlas. 

Marc Raibert: So here's a little bit of a jump.

Anderson Cooper: I mean, that's incredible. (LAUGH) 

Atlas isn't doing all this on its own. Technician Bryan Hollingsworth is steering it with this remote control. But the robot's software allows it to make other key decisions autonomously.  

Marc Raibert: So really the robot is

Anderson Cooper: That's incredible--

Marc Raibert: You know, doing all its own balance, all its own control. Bryan's just steering it, telling it what speed and direction. Its computers are-- adjusting how the legs are placed and what forces it's applying--

Marc Raibert: In order to keep it-- balanced.

Atlas balances with the help of sensors, as well as a gyroscope and three on-board computers. It was definitely built to be pushed around.

Marc Raibert: Good, push it a little bit more. It's just trying to keep its balance. Just like you will, if I push you. And you can push it in any direction, you can push it from the side. (LAUGH) 

Making machines that can stay upright on their own and move through the world with the ease of an animal or human has been an obsession of Marc Raiberts' for 40 years. 

Anderson Cooper: The space of time you've been working in is nothing compared to the time it's taken for animals and humans to develop.

Marc Raibert: Some people look at me and say, "Oh, Raibert, you've been stuck on this problem for 40 years." Animals are amazingly good, and people, at-- at what they do. You know, we're so agile. We're so versatile. We really haven't achieved what humans can do yet. But I think-- I think we can.

Raibert isn't making it easy for himself, he's given most of his robots legs.

Anderson Cooper: Why focus on, on legs? I would think wheels would be easier. 

Marc Raibert: Yeah, wheels and tracks are great if you have a prepared surface like a road or even a dirt road. But people and animals can go anywhere on earth-- using their legs. And, so, that, you know, that was the inspiration. 

Some of the first contraptions he built in the early 1980s bounced around on what looked like pogo sticks. They appeared in this documentary when Raibert was a pioneering professor of robotics and computer science at Carnegie Mellon. He founded Boston Dynamics in 1992, and with CEO Robert Playter has been working for decades to perfect how robots move. 

They developed this robot, called Big Dog, for the military as well as a larger pack mule that could carry 400 pounds on its back. Experimenting with speed, they got this cheetah-like robot to run nearly 30 miles an hour.

None of these made it out of the prototype phase. But they did lead to this. It's called Spot. Boston Dynamics made it not knowing exactly how it would be used.

But the inspiration for it isn't hard to figure out.

Hannah Rossi: So Spot is a omni-directional robot. So I can go forwards and backwards.

Anderson Cooper: This is crazy. (LAUGH)

Robert Playter: This is the real benefit of legs. Legs give you that capability. 

That's Robert Playter, the CEO, and Hannah Rossi, a technician who works on Spot. 

Hannah Rossi: I'm not doing anything special to let it walk over those rocks. There you go.

The controls are easier to use than you might expect.

Anderson Cooper: Does it have to come in, straight on?

Hannah Rossi: You don't have to be perfect about it drive it close to wherever you want to go and the robot will do the rest.

Anderson Cooper: Wow. In some ways it's like driving a very sophisticated remote control car. What makes it different?

Robert Playter: Spot is really smart about its own locomotion. It deals with all the details about how to place my feet, what gait to use, how to manage my body so that all you have to tell it is the direction they go to.

And in some cases, you don't even have to do that. When signaled, Spot can take itself off its charging station and go for a walk on its own -- as long as it's pre-programmed with the route.

It uses five 3D cameras to map its surroundings and avoid obstacles. 

Atlas has a similar technology, while we were talking in front of Atlas, this is how it saw us.  

Marc Raibert: This is inside Atlas's brain. And it shows its perception system. So, what looks like a flashlight is really the data that's coming back from its cameras. And it-- you see the white-- rectangles, that means it's identifying a place that it could step. And then once it identifies it, it attaches those footsteps to it, and it says, "Okay, I'm gonna try and step there." And then it adjusts its mechanics so that it actually hits those places when it's-- running. 

All of that happens in a matter of milliseconds.

Marc Raibert: And so it's gonna use that vision to adjust itself as it goes running over these blocks.

Atlas cost tens of millions of dollars to develop, but it's not for sale. It's used purely for research and development.

But Spot is on the market. Around 500 are out in the world. They sell for about $75,000 apiece, accessories cost extra. Some spots work at utility companies using mounted cameras to check on equipment. Others monitor construction sites and several police departments have tried them out to assist with investigations.

Anderson Cooper: Let's talk about the the fear factor, When you post a video of Atlas or Spot doing something, a ton of people are amazed by it and think it's great. And there's a lot of people who think this is terrifying. 

Robert Playter: The rogue robot story is a powerful story. And it's been told for 100 years. But it's fiction. Robots don't have agency. They don't make up their own minds about what their tasks are. They operate within a narrow bound of their programming.

Anderson Cooper: It is easy to project human qualities onto these machines. 

Robert Playter: I think people do attribute to our robots much more than they should. Because you know, they haven't seen machines move like this before. And so they-- they want to project intelligence and emotion onto that in ways that are fiction.

In other words, these robots still have a long way to go.

Anderson Cooper: I mean, it's not C3PO. It-- it's not-- a thinking--

Marc Raibert: Yeah. So let me tell you--

Anderson Cooper: Okay.

Marc Raibert: About that. There's a cognitive intelligence and an athletic intelligence. You know, cognitive intelligence is making plans, making decisions-- reasoning, and things like that.

Anderson Cooper: It's not doing that?

Marc Raibert: It's mostly doing athletic intelligence--

Anderson Cooper: Okay--

Marc Raibert: Which is managing its body, its posture, its energetics. If you told it to travel in a circle in the room it can go through the sequence of steps. But if you ask it to-- go find me a soda, it's-- it's not doing anything like that.

Just picking an item off the floor can sometimes be a struggle for Spot. Enabling it to open a door has taken years of programming and practice and a human has to tell it where the hinges are.  

Kevin Blankespoor: Each time we add some new capability-- and we feel like we've got it to a decent point, that's when you push it to failure to figure out, you know, how good of a job you've really done.

Kevin Blankespoor is one of the lead engineers here, but at times, he prefers a very low-tech approach to testing robots. 

Anderson Cooper: You're pretty tough on robots.

Kevin Blankespoor: We think of that as-- as just another way to push them out of the comfort zone. 

Failure is a big part of the process. When trying something new, robots, like humans, don't get it right every time. There might be dozens of crashes for every one success.

Anderson Cooper: How often do you break a robot? (LAUGH)

Marc Raibert: We break them all the time. I mean, it's part of our culture. We have a motto, "Build it, break it, fix it." 

To do that, Boston Dynamics has recruited roboticists with diverse backgrounds - there's plenty of Ph.D's, but also bike builders, and race car mechanics. Bill Washburn is part of that pit crew. 

Anderson Cooper: They all look pretty dinged up. 

Bill Washburn: Yeah.

Anderson Cooper: How often do these need to get repaired?

Bill Washburn: The biggest-- kinda failures for me are, like, the bottom part of the robot breaks off of the top part of the robot. (CHUCKLE) And it's like--

Anderson Cooper: That seems like a big-- big failure. (CHUCKLE)

Bill Washburn: And the hydraulic hoses are the only thing holding it together.

Recently, Raibert and his team decided to push their robots in a way they never had before.

Marc Raibert: We spent at least six months, maybe eight, just preparing for what we were gonna do. And then we started to get the technical teams working on the behavior. 

The behavior was dancing. All their robots got in on the act. The movements were cutting edge, but the music and the Mashed Potato were definitely oldschool. 

Anderson Cooper: There are some people who see that and say, "That can't be real."

Marc Raibert: Nothing's more gratifying than hearing that. 

Anderson Cooper: What's the point in proving that the robot can do the Mashed Potato?

Marc Raibert: This process of, you know, doing new things with the robots lets you generate new tools, new approaches, new understanding of the problem-- that takes you forward. But, man, isn't it just fun? 

Anderson Cooper: But, I mean, it's-- it costs a lotta money. It took 18 months of your time.

Marc Raibert: I think it was worth it. (LAUGHTER)

Whether it'll be worth it to Boston Dynamics' new owners is less clear

The South Korean carmaker, Hyundai, has purchased a majority stake for nearly a billion dollars. It is Boston Dynamics' third owner in eight years. There's pressure to turn their research into revenue. 

And Boston Dynamics hopes this new robot will help. It's called Stretch and it's due to go on sale next year. This was the first time they'd shown it publically.

Kevin Blankespoor: Warehouses is really the next frontier for robotics. 

Stretch may not be that exciting to look at, but it's built with a definite purpose in mind.  It's got a seven-foot arm and they say it can move 800 boxes an hour in a warehouse and work for up to 16 hours without a break. Unlike many industrial robots that sit in one place, stretch is designed to move around. 

Kevin Blankespoor: You can drive it around with a joystick. And at times, that's the easiest way to get it set up. But once it's ready to go in a truck and unload it, you hit go and from there on it's autonomous. And it'll keep finding boxes and moving 'em until it's all the way through.

Robert Playter: This generation of robots is gonna be different. They're gonna work amongst us. They're gonna work next to us-- in ways where we help them but they also take some of the burden from us.

Anderson Cooper: The more robots are integrated into the workforce, the more jobs would be taken away.

Robert Playter: At the same time, you're creating a new industry. We envision a job-- we-- we-- we like to call the robot wrangler. He'll launch and manage five to 10 robots at a time and sort of-- keep them all working. 

Anderson Cooper: Is there a robot you've always dreamt of making (LAUGH) that you haven't been able to do yet?

Marc Raibert: A car with an active suspension essentially legs like w-- like a roller skating robot. And a robot like that, you know, could go anywhere on earth. That's one thing that maybe we'll do at some point. But, you know, really, the sky's the limit. There's-- there's all kinds of things we can and will do.

As with so many things Boston Dynamics does. It's hard to imagine how that would work, but then again, who'd have thought a bunch of metal machines would one day show us all how to do the Mashed Potato.

Produced by Nichole Marks. Associate producer, David M. Levine. Broadcast associate, Annabelle Hanflig. Edited by Sean Kelly.

• Boston Dynamics

Anderson Cooper, anchor of CNN's "Anderson Cooper 360," has contributed to 60 Minutes since 2006. His exceptional reporting on big news events has earned Cooper a reputation as one of television's preeminent newsmen.

More from CBS News

Meet Nvidia CEO Jensen Huang, and the $2 trillion company powering today's AI

Military families feel betrayed over jet fuel-tainted water at Hawaii base

16,000 people with disabilities are in institutions. Here's what experts say should change.

The potential threat to Honolulu's water supply lurking underground

• What is ChatGPT?
• How to Use Google Gemini

Boston Dynamics uses ChatGPT to create a robot tour guide

Boston Dynamics’ Spot robot has already impressed us with its astonishing agility , but now it can make a pretty good tour guide, too.

Keen to explore how generative AI tools like OpenAI’s ChatGPT chatbot can be used with its robot dog, the company’s clever engineers recently decided to create a tour guide using the technology.

• So THAT’S why Boston Dynamics retired its Atlas robot
• Apple finally has a way to defeat ChatGPT
• GPT-4 Turbo is the biggest update since ChatGPT’s launch

A video (top) demonstrating the effort reveals some very impressive results. Sporting a hat, plastic eyes, and fake eyebrows, and opening and closing its robotic gripper as it “talks,” Spot takes us on an entertaining tour of parts of Boston Dynamics’ facility in Massachusetts.

“We were interested in seeing how you could use technologies like these for robotics,” Matt Klingensmith, principal software engineer at Boston Dynamics, says in the video.

As part of preparations, the team gave Spot a brief script naming each room that it was to visit, along with a single sentence explaining the room’s purpose. Spot then combines that data with images from its built-in cameras before running it through what’s known as a “visual question answering model” to try to get more information about what it’s looking at so that it can offer a more elaborate spoken response.

The best part is how Spot behaves when instructed to adopt different personalities. Check out the British butler guide at the start of the video, for example, and the sarcastic guide a few minutes in. The Shakespearean actor is also very impressive.

“It would come up with these crazy personalities,” Klingensmith says. “It would incorporate its backstory into what it was seeing, reinterpret things that it was seeing. It was pretty incredible.”

The software engineer said he was also surprised by some of the responses. For example, when he asked Spot to show him its parents, the robot took him over to an early version of Spot among Boston Dynamics’ display of robots.

Klingensmith says AI might make it possible for robots “not just to follow our commands, but in some sense understand the actions that they can take in the context of the world around them,” adding that it could be useful for applications that “we haven’t imagined yet.”

Human tour guides needn’t worry just yet, though, as Klingensmith said that AI chatbots still have a tendency to make stuff up (known as “hallucinations”), which is definitely what you don’t want to happen inside somewhere like a museum.

It’s also susceptible to the internet connection going down, he added.

Editors' Recommendations

• Watch Boston Dynamics’ dog-like robot don a dog suit and dance
• Boston Dynamics retires its remarkable Atlas robot
• This one image breaks ChatGPT each and every time
• Nvidia’s supercomputer may bring on a new era of ChatGPT
• Boston Dynamics’ Spot robot to paint for an art exhibition
• Artificial Intelligence

Google has just announced the launch of its conversational AI, Bard. Bard is Google's response to the ever-popular ChatGPT, now in use by Microsoft in its own products.

The tech giant rushed to release Bard, and it is now ready for testing. Google is inviting users to test the AI, but as it notes, it might make mistakes.

The constant buzz around OpenAI's ChatGPT refuses to wane. With Microsoft now using the same technology to power its brand-new Bing Chat, it's safe to say that ChatGPT may continue this upward trend for quite some time. That's good news for OpenAI and Microsoft, but they're not the only two companies to benefit.

According to a new report, the sales of Nvidia's data center graphics cards may be about to skyrocket. With the commercialization of ChatGPT, OpenAI might need as many as 10,000 new GPUs to support the growing model -- and Nvidia appears to be the most likely supplier.

Microsoft has just unveiled the latest update to Windows 11 which has already started rolling out. There are plenty of changes on the horizon, including those involving the ChatGPT-based Bing AI search.

The new update lets Windows 11 users communicate with the AI-powered version of Bing right in their taskbar. The AI model itself also seems to have received an update that might make conversing with it less bizarre.

• Artificial Intelligence /

Boston Dynamics turned its robot dog into a talking tour guide with ChatGPT

The company used chatgpt to train its four-legged bot to answer questions and generate responses about its facilities..

By Emma Roth , a news writer who covers the streaming wars, consumer tech, crypto, social media, and much more. Previously, she was a writer and editor at MUO.

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

We saw Spot run , jump , and even dance ... but now we can see Spot talk . In a somewhat unsettling video posted by Boston Dynamics, we see its robot dog outfitted with a top hat, mustache, and googly eyes as it chats with staff members in a British accent, taking them on a tour of the company’s facilities.

“Shall we commence our journey?” Spot asks. “The charging stations, where Spot robots rest and recharge, is our first point of interest. Follow me, gentlemen.” As shown in the demo, Spot is capable of answering questions and even opens its “mouth” to make it seem like it’s actually speaking.

To make Spot “talk,” Boston Dynamics used OpenAI’s ChatGPT API, along with some open-source large language models (LLM) to carefully train its responses. It then outfitted the bot with a speaker, added text-to-speech capabilities, and made its mouth — er... gripper — mimic speech “like the mouth of a puppet.”

Matt Klingensmith, the principal software engineer at Boston Dynamics, says the team gave Spot a “very brief script” for each of the rooms at its facilities. The bot then combined that script with the imagery it gets from the cameras on its gripper and body, allowing it to “get more information about what it sees before generating a response.” According to the company, Spot uses Visual Question Answering models to essentially caption images and answer questions about them.

“Generator hums low in a room devoid of joy. Much like my soul.”

The “fancy butler” is not the only persona Spot assumes during the video. The four-legged bot also takes on the personality of a 1920s archaeologist, a teenager, and a Shakespearean time traveler. It even assumes a sarcastic personality, which, when asked to come up with a haiku, said: “Generator hums low in a room devoid of joy. Much like my soul.”

Boston Dynamics says it uncovered a few surprises when experimenting with Spot as a tour guide. In one instance, the team asked Spot who its “parents” were, and it went over to where the older Spot models are displayed in the company’s office. The company also notes that it still ran into some instances where the LLM made things up, such as suggesting that Stretch, its robot designed to move boxes , was made for yoga.

“We’re excited to continue exploring the intersection of artificial intelligence and robotics,” Klingensmith writes in a post on Boston Dynamics’ site . “These models [LLMs] can help provide cultural context, general commonsense knowledge, and flexibility that could be useful for many robotics tasks — for example, being able to assign a task to a robot just by talking to it would help reduce the learning curve for using these systems.”

While Spot may sound and look pretty silly during the video, it’s still a bit hard not to think about the dog-like robot’s ability to open doors and spy on people . After all, it’s used as a tool for the police and military .

Turns out the Rabbit R1 was just an Android app all along

Automatic emergency braking at speeds up to 90mph required under new rule, razer made a million dollars selling a mask with rgb, and the ftc is not pleased, elon musk goes ‘absolutely hard core’ in another round of tesla layoffs, binance founder’s sentencing hearing.

More from Artificial Intelligence

Interview: Figma’s CEO on life after the company’s failed sale to Adobe

The shine comes off the Vision Pro

Spike Jonze’s Her holds up a decade later

You sound like a bot

The May 2024 issue of IEEE Spectrum is here!

For IEEE Members

Ieee spectrum, follow ieee spectrum, support ieee spectrum, enjoy more free content and benefits by creating an account, saving articles to read later requires an ieee spectrum account, the institute content is only available for members, downloading full pdf issues is exclusive for ieee members, downloading this e-book is exclusive for ieee members, access to spectrum 's digital edition is exclusive for ieee members, following topics is a feature exclusive for ieee members, adding your response to an article requires an ieee spectrum account, create an account to access more content and features on ieee spectrum , including the ability to save articles to read later, download spectrum collections, and participate in conversations with readers and editors. for more exclusive content and features, consider joining ieee ., join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of spectrum’s articles, archives, pdf downloads, and other benefits. learn more →, join the world’s largest professional organization devoted to engineering and applied sciences and get access to this e-book plus all of ieee spectrum’s articles, archives, pdf downloads, and other benefits. learn more →, access thousands of articles — completely free, create an account and get exclusive content and features: save articles, download collections, and talk to tech insiders — all free for full access and benefits, join ieee as a paying member., boston dynamics' spot robot dog now available for $74,500, for the price of a luxury car, you can now get a very smart, very capable, very yellow robotic dog.

Boston Dynamics has been fielding questions about when its robots are going to go on sale and how much they'll cost for at least a dozen years now. I can say this with confidence, because that's how long I've been a robotics journalist, and I've been pestering them about it the entire time. But it's only relatively recently that the company started to make a concerted push away from developing robots exclusively for the likes of DARPA into platforms with more commercial potential , starting with a compact legged robot called Spot , first introduced in 2016 .

Since then, we've been following closely as Spot has gone from a research platform to a product, and today, Boston Dynamics is announcing the final step in that process: commercial availability. You can now order a Spot Explorer Kit from the Boston Dynamics online store for US $74,500 (plus tax), shipping included, with delivery in 6 to 8 weeks. FINALLY!

Over the past 10 months or so, Boston Dynamics has leased Spot robots to carefully selected companies, research groups, and even a few individuals as part of their early adopter program—that's where all of the clips in the video below came from. While there are over 100 Spots out in the world right now, getting one of them has required convincing Boston Dynamics up front that you knew more or less exactly what you wanted to do and how you wanted to do it. If you're a big construction company or the Jet Propulsion Laboratory or Adam Savage , that's all well and good, but for other folks who think that a Spot could be useful for them somehow and want to give it a shot, this new availability provides a fewer-strings attached opportunity to do some experimentation with the robot.

There's a lot of cool stuff going on in that video, but we were told that the one thing that really stood out to the folks at Boston Dynamics was a 2-second clip that you can see on the left-hand side of the screen from 0:19 to 0:21. In it, Spot is somehow managing to walk across a spider web of rebar without getting tripped up, at faster than human speed. This isn't something that Spot was specifically programmed to do, and in fact the Spot User Guide specifically identifies “rebar mesh" as an unsafe operating environment. But the robot just handles it, and that's a big part of what makes Spot so useful—its ability to deal with (almost) whatever you can throw at it.

Before you get too excited, Boston Dynamics is fairly explicit that the current license for the robot is intended for commercial use , and the company specifically doesn't want people to be just using it at home for fun. We know this because we asked (of course we asked), and they told us “we specifically don't want people to just be using it at home for fun." Drat. You can still buy one as an individual, but you have to promise that you'll follow the terms of use and user guidelines, and it sounds like using a robot in your house might be the second-fastest way to invalidate your warranty:

SPOT IS AN AMAZING ROBOT, BUT IS NOT CERTIFIED SAFE FOR IN-HOME USE OR INTENDED FOR USE NEAR CHILDREN OR OTHERS WHO MAY NOT APPRECIATE THE HAZARDS ASSOCIATED WITH ITS OPERATION.

Not being able to get Spot to play with your kids may be disappointing, but for those of you with the sort of kids who are also students, the good news is that Boston Dynamics has carved out a niche for academic institutions, which can buy Spot at a discounted price. And if you want to buy a whole pack of Spots, there's a bulk discount for Enterprise users as well.

What do you get for $74,500? All this!

• Spot battery (2x)
• Spot charger
• Tablet controller and charger
• Robot case for storage and transportation
• FREE SHIPPING!

You can view detailed specs here .

So is $75k a lot of money for a robot like Spot, or not all that much? We don't have many useful points of comparison, partially because it's not clear to what extent other pre-commercial quadrupedal robots (like ANYmal or Aliengo ) share capabilities and features with Spot. For more perspective on Spot's price tag, we spoke to Michael Perry, vice president of business development at Boston Dynamics .

IEEE Spectrum : Why is Spot so affordable?

Michael Perry: The main goal of selling the robot at this stage is to try to get it into the hands of as many application developers as possible, so that we can learn from the community what the biggest driver of value is for Spot. As a platform, unlocking the value of an ecosystem is our core focus right now.

Spectrum: Why is Spot so expensive?

Perry: Expensive is relative, but compared to the initial prototypes of Spot, we've been able to drop down the cost pretty significantly. One key thing has been designing it for robustness—we've put hundreds and hundreds of hours on the robot to make sure that it's able to be successful when it falls, or when it has an electrostatic discharge. We've made sure that it's able to perceive a wide variety of environments that are difficult for traditional vision-based sensors to handle. A lot of that engineering is baked into the core product so that you don't have to worry about the mobility or robotic side of the equation, you can just focus on application development.

The $75k that you'll pay for the Spot Explorer Kit , it's important to note, is just the base price for the robot. As with other things that fall into this price range (like a luxury car), there are all kinds of fun ways to drive that cost up with accessories, although for Spot, some of those accessories will be necessary for many (if not most) applications. For example, a couple of expansion ports to make it easier to install your own payloads on Spot will run you $1,275. An additional battery is $4,620. And if you want to really get some work done, the Enhanced Autonomy Package (with 360 cameras, lights, better comms, and a Velodyne VLP-16) will set you back an additional $34,570. If you were hoping for an arm, you'll have to wait until the end of the year.

Each Spot also includes a year's worth of software updates and a warranty, although the standard warranty just covers “defects related to materials and workmanship" not “I drove my robot off a cliff" or “I tried to take my robot swimming." For that sort of thing (user error) to be covered, you'll need to upgrade to the $12,000 Spot CARE premium service plan to cover your robot for a year as long as you don't subject it to willful abuse, which both of those examples I just gave probably qualify as.

While we're on the subject of robot abuse, Boston Dynamics has very sensibly devoted a substantial amount of the Spot User Guide to help new users understand how they should not be using their robot, in order to “lessen the risk of serious injury, death, or robot and other property damage." According to the guide, some things that could cause Spot to fall include holes, cliffs, slippery surfaces (like ice and wet grass), and cords. Spot's sensors also get confused by “transparent, mirrored, or very bright obstacles," and the guide specifically says Spot “may crash into glass doors and windows." Also this: “Spot cannot predict trajectories of moving objects. Do not operate Spot around moving objects such as vehicles, children, or pets."

We should emphasize that this is all totally reasonable, and while there are certainly a lot of things to be aware of, it's frankly astonishing that these are the only things that Boston Dynamics explicitly warns users against. Obviously, not every potentially unsafe situation or thing is described above, but the point is that Boston Dynamics is willing to say to new users, “here's your robot, go do stuff with it" without feeling the need to hold their hand the entire time.

There's one more thing to be aware of before you decide to buy a Spot, which is the following:

“All orders will be subject to Boston Dynamics' Terms and Conditions of Sale which require the beneficial use of its robots."

Specifically, this appears to mean that you aren't allowed to (or supposed to) use the robot in a way that could hurt living things, or “as a weapon, or to enable any weapon." The conditions of sale also prohibit using the robot for “any illegal or ultra-hazardous purpose," and there's some stuff in there about it not being cool to use Spot for “nuclear, chemical, or biological weapons proliferation, or development of missile technology," which seems weirdly specific.

I'm very glad that Boston Dynamics is being so upfront about requiring that Spot is used beneficially. However, it does put the company in a somewhat challenging position now that these robots are being sold. Boston Dynamics can (and will) perform some amount of due-diligence before shipping a Spot, but ultimately, once the robots are in someone else's hands, there's only so much that BD can do.

Spectrum: Why is beneficial use important to Boston Dynamics?

Perry: One of the key things that we've highlighted many times in our license and terms of use is that we don't want to see the robot being used in any way that inflicts physical harm on people or animals. There are philosophical reasons for that—I think all of us don't want to see our technology used in a way that would hurt people. But also from a business perspective, robots are really terrible at conveying intention. In order for the robot to be helpful long-term, it has to be trusted as a piece of technology. So rather than looking at a robot and wondering, “is this something that could potentially hurt me," we want people to think “this is a robot that's here to help me." To the extent that people associate Boston Dynamics with cutting edge robots, we think that this is an important stance for the rollout of our first commercial product. If we find out that somebody's violated our terms of use, their warranty is invalidated, we won't repair their product, and we have a licensing timeout that would prevent them from accessing their robot after that timeout has expired. It's a remediation path, but we do think that it's important to at least provide that as something that helps enforce our position on use of our technology.

It's very important to keep all of this in context: Spot is a tool. It's got some autonomy and the appearance of agency, but it's still just doing what people tell it to do, even if those things might be unsafe. If you read through the user guide, it's clear how much of an effort Boston Dynamics is making to try to convey the importance of safety to Spot users—and ultimately, barring some unforeseen and catastrophic software or hardware issues, safety is about the users, rather than Boston Dynamics or Spot itself. I bring this up because as we start seeing more and more Spots doing things without Boston Dynamics watching over them quite so closely, accidents are likely inevitable. Spot might step on someone's foot. It might knock someone over. If Spot was perfectly safe, it wouldn't be useful, and we have to acknowledge that its impressive capabilities come with some risks, too.

Now that Spot is on the market for real, we're excited to see who steps up and orders one. Depending on who the potential customer is, Spot could either seem like an impossibly sophisticated piece of technology that they'd never be able to use, or a magical way of solving all of their problems overnight. In reality, it's of course neither of those things. For the former (folks with an idea but without a lot of robotics knowledge or experience), Spot does a lot out of the box, but BD is happy to talk with people and facilitate connections with partners who might be able to integrate specific software and hardware to get Spot to do a unique task. And for the latter (who may also be folks with an idea but without a lot of robotics knowledge or experience), BD's Perry offers a reminder Spot is not Rosie the Robot, and would be equally happy to talk about what the technology is actually capable of doing.

Looking forward a bit, we asked Perry whether Spot's capabilities mean that customers are starting to think beyond using robots to simply replace humans, and are instead looking at them as a way of enabling a completely different way of getting things done.

Spectrum: Do customers interested in Spot tend to think of it as a way of replacing humans at a specific task, or as a system that can do things that humans aren't able to do?

Perry: There are what I imagine as three levels of people understanding the robot applications. Right now, we're at level one, where you take a person out of this dangerous, dull job, and put a robot in. That's the entry point. The second level is, using the robot, can we increase the production of that task? For example, take site documentation on a construction site—right now, people do 360 image capture of a site maybe once a week, and they might do a laser scan of the site once per project. At the second level, the question is, what if you were able to get that data collection every day, or multiple times a day? What kinds of benefits would that add to your process? To continue the construction example, the third level would be, how could we completely redesign this space now that we know that this type of automation is available? To take one example, there are some things that we cannot physically build because it's too unsafe for people to be a part of that process, but if you were to apply robotics to that process, then you could potentially open up a huge envelope of design that has been inaccessible to people.

To order a Spot of your very own, visit shop.bostondynamics.com .

A version of this post appears in the August 2020 print issue as “$74,500 Will Fetch You a Spot."

• Why Boston Dynamics Is Putting Legged Robots in Hospitals - IEEE ... ›
• How Boston Dynamics Is Redefining Robot Agility - IEEE Spectrum ›
• Boston Dynamics' Spot Robot Dog Goes on Sale - IEEE Spectrum ›
• Spot’s 3.0 Update Adds Increased Autonomy, New Door Tricks ›
• Boston Dynamics | LinkedIn ›
• [BostonDynamics] Spot's Got an Arm! ›
• Spot | Boston Dynamics ›

Evan Ackerman is a senior editor at IEEE Spectrum . Since 2007, he has written over 6,000 articles on robotics and technology. He has a degree in Martian geology and is excellent at playing bagpipes.

A Brief History of the World’s First Planetarium

Agricultural iot system sends power through the soil, this startup uses the mit inventor app to teach girls coding, related stories, spot’s 3.0 update adds increased autonomy, new door tricks, q&a: boston dynamics on atlas’s parkour skills, atlas shows most impressive parkour skills we've ever seen.

Exhibitors:

Boston Dynamics, Inc.

200 Smith Street Waltham, Massachusetts 02451 United States http://www.bostondynamics.com

Tel: (617) 868-5600 Click Here to Contact jshipley@bostondynamics.com

Proud Member of:

Boston Dynamics’ mission is to imagine and create exceptional robots that enrich people’s lives. Building machines that can approximate the mobility, dexterity and agility of people and animals is a grand challenge. Curiosity and respect for the natural world are at the heart of our work on robots. We see products derived from this work as the next step in the human history of building machines to reduce the danger, repetition and physically difficult aspects of work.

• Autonomous Mobile Robots
• Exhibitor List
• Keynote Speakers
• Theater Schedule
• Networking Party
• AMR Demo Schedule
• Hotel Information
• Sponsorship Opportunities
• Become a Sponsor
• Show Organizer
• Conference Agenda
• Conference Speakers
• Certified Vision Professional Program
• Why Exhibit
• Exhibitor Prospectus
• Become an Exhibitor

• Today's news
• Reviews and deals
• Climate change
• 2024 election
• Fall allergies
• Health news
• Mental health
• Sexual health
• Family health
• So mini ways
• Unapologetically
• Buying guides

Entertainment

• How to Watch
• My watchlist
• Stock market
• Biden economy
• Personal finance
• Stocks: most active
• Stocks: gainers
• Stocks: losers
• Trending tickers
• World indices
• US Treasury bonds
• Top mutual funds
• Highest open interest
• Highest implied volatility
• Currency converter
• Basic materials
• Communication services
• Consumer cyclical
• Consumer defensive
• Financial services
• Industrials
• Real estate
• Mutual funds
• Credit cards
• Balance transfer cards
• Cash back cards
• Rewards cards
• Travel cards
• Online checking
• High-yield savings
• Money market
• Home equity loan
• Personal loans
• Student loans
• Options pit
• Fantasy football
• Pro Pick 'Em
• College Pick 'Em
• Fantasy baseball
• Fantasy hockey
• Fantasy basketball
• Download the app
• Daily fantasy
• Scores and schedules
• GameChannel
• World Baseball Classic
• Premier League
• CONCACAF League
• Champions League
• Motorsports
• Horse racing
• Newsletters

New on Yahoo

• Privacy Dashboard

Arrests at UCLA, Columbia as campus demonstrations intensify

This is what happened when boston dynamics' robots started to speak, powered by chatgpt, what you need to know.

A group of engineers recently developed a robot by leveraging generative AI capabilities to serve as a tour guide across Boston Dynamics' premises.

Boston Dynamics trained its AI systems on a massive dataset. The development team admits that while the invention is impressive, they encountered several issues, including hallucination episodes.

The robot is based on OpenAI's GPT-4 model, but the development team employed prompt engineering techniques to establish control over its responses.

In every sense of the word, generative AI is reshaping how we go about our day-to-day activities in one way or another. From helping students solve complex math problems, writing poems, and even generating images based on your imagination and prompts . Based on these instances, the future looks bright as the technology will help users explore more avenues as well as untapped opportunities.

Speaking of untapped opportunities, Boston Dynamics recently documented new heights it achieved by leveraging generative AI capabilities. The company created a robot that's designed to serve as a tour guide using its Spot robot, integrated with ChatGPT and other AI models.

The American engineering and design company specializes in developing robots, including their famed dog-like "Spot." The robots are designed to make work easier for users as they take on repetitive, dangerous, and complex tasks, ultimately boosting productivity while simultaneously asserting the user's safety.

In particular, we were interested in a demo of Spot using Foundation Models as autonomy tools—that is, making decisions in real-time based on the output of FMs. Large Language Models (LLMs) like ChatGPT are basically very big, very capable autocomplete algorithms; they take in a stream of text and predict the next bit of text. We were inspired by the apparent ability of LLMs to roleplay, replicate culture and nuance, form plans, and maintain coherence over time, as well as by recently released Visual Question Answering (VQA) models that can caption images and answer simple questions about them.

The emergence of large AI systems trained on a massive dataset sparked Matt Klingensmith's interest (Principal Software Engineer) mainly because of the Emergent Behavior that's part of these models. Emergent Behavior is the ability of AI-powered chatbots to perform tasks outside the data they are based and trained on.

Matt viewed this as a great opportunity, which led to the commencement of the project earlier this year in the summer with the aim of exploring its impact on robotics development.

How does the robot tour guide work?

The software engineer disclosed that developing a robot tour guide was the easiest and fastest way to test this theory. Essentially, the robot has the capability to walk around the company premises looking at objects.

What's more, it leverages a VQA, which is a captioning model to describe the objects within its view, further elaborating on its description using a large language model (LLM). Through the LLM, the robot can also answer questions posed by its audience and even plan the next actions it should take.

While LLMs like Bing Chat have faced their fair share of setbacks including hallucination episodes , this was not a major concern for the robot's development team. Instead, the team was more focused on the entertainment and interactive aspects. Besides, the robot's ability to walk around was already figured out in Spot's autonomy SDK . Boston Dynamics leverages the Spot SDK to support the development of autonomous navigation behaviors for the Spot robot.

For communication purposes, the team 3D printed a vibration-resistant mount for a Respeaker V2 speaker bundled with a ring-array microphone with LEDs on the robot tour guide. This way, the robot is able to listen to its audience and respond to their queries.

Building on this premise the team integrated OpenAI's ChatGPT API starting with the GPT-3.5 model , but eventually transitioned to GPT-4 once it shipped to general availability to further improve the robot's communication skills. To ensure that the robot doesn't spiral out of control or give distasteful responses, the team employed prompt engineering techniques.

According to the robot's development team:

"Inspired by a  method from Microsoft , we prompted ChatGPT by making it appear as though it was writing the next line in a Python script. We provided English documentation to the LLM in the form of comments. We then evaluated the output of the LLM as though it were Python code."

The team also revealed that the LLM integrated into the robot also had access to the Spot autonomy SDK, a detailed map of the tour site bundled up with one-line descriptions of each location, as well as the capability to respond and ask questions.

Text to Speech conversion

While the robot heavily relies on ChatGPT for communication, it's obvious that the chatbot is text based. This is why the development the cloud service  ElevenLabs , to serve as a text-to-speech tool.

The team also incorporated robot’s gripper camera and front body camera into BLIP-2. This way, it's easier for it to interpret what it sees and provide context. According to the team that BLIP-2 ran the images and visuals "either in visual question answering mode (with simple questions like “what is interesting about this picture?”) or image captioning mode" at least once a second.

Life-like conversations

The team also wanted to present a life-like experience for the audience while interacting with the robot during the tour. As such, the team incorporated some default body language to bring this experience to life. Thanks to the Spot 3.3 release , the robot is able to direct its arm to the nearest person while explaining a particular concept.

The development process turned out to be quite the spectacle, as the team ran into several surprises. For instance, when asked who Marc Raibert was, the robot responded by stating that it didn't know and recommended heading to the IT help desk for further assistance. Strangely enough, the development team didn't prompt the LLM to seek further assistance. According to Matt, the robot must have associated the location of the IT help desk with the action of asking for help.

Matt admits that while the robot tour guide is impressive, it was spotted hallucinating severely and making up things. Not forgetting the adverse impact on the robot's performance in the event that it was unable to establish a stable internet connection.

Moving forward, the team aims to explore this avenue even more, especially after discovering that it is possible to integrate the results of several general AI systems together.

Recommended Stories

2024 nfl draft grades: denver broncos earn one of our lowest grades mostly due to one pick.

Yahoo Sports' Charles McDonald breaks down the Broncos' 2024 draft.

NFL Power Rankings, draft edition: Did Patriots fix their offensive issues?

Which teams did the best in the NFL Draft?

The It List: Met Gala 2024 is a fashion feast for the eyes, 'The Idea of You' will give you butterflies, Ryan Gosling plays 'Action Hero Ken' in 'The Fall Guy'

With highly anticipated new movies and the star-studded Met Gala on the horizon, it's a great week for celebrity enthusiasts.

New details emerge in alleged gambling ring behind Shohei Ohtani-Ippei Mizuhara scandal

It turns out the money was going from Ohtani's bank account to an illegal bookie to ... casinos.

Does castor oil really help with hair growth? We asked the experts, and their answer may surprise you

It's inexpensive, but is it effective? Dermatologists' verdict is in — and it's unanimous.

Formula 1: Miami Grand Prix sends cease and desist letter to prevent Donald Trump fundraiser during race

Race organizers say they'll revoke a Trump fundraiser's suite license if he holds an event for the former president on Sunday at the race.

NFL Draft grades for all 32 teams | Zero Blitz

Jason Fitz and Frank Schwab join forces to recap the draft in the best way they know how: letter grades! Fitz and Frank discuss all 32 teams division by division as they give a snapshot of how fans should be feeling heading into the 2024 season. The duo have key debates on the Dallas Cowboys, New York Giants, New Orleans Saints, Los Angeles Rams, New England Patriots, Las Vegas Raiders and more.

The best RBs for 2024 fantasy football according to our experts

The Yahoo Fantasy football analysts reveal their first running back rankings for the 2024 season.

10 cars with paint problems, according to Consumer Reports

Consumer Reports shares the ten vehicles most prone to paint problems, and they span quite an array of models.

Tight end rankings for 2024 fantasy football

The Yahoo Fantasy football analysts reveal their first tight end rankings for the 2024 season.

MLB Power Rankings: Braves move into the top spot followed by Dodgers, Phillies as injuries take a toll across the league

From the Braves to the Marlins, here's where all 30 teams stand after the season's first month.

The expanded 12-team College Football Playoff is here — and it already has problems

There is cause for excitement around the new playoff format. There's also lots of complaints and criticism to go around.

Tyrese Maxey saved the Sixers' season with one of the toughest playoff performances ever

On Tuesday, Maxey — an All-Star, the league’s Most Improved Player, and now, author of one of the most legendary postseason performances in Philadelphia basketball history — helped the Sixers survive to see a Game 6 against the Knicks.

The Spin: Fantasy baseball's top developments (good and bad) from MLB season's first month

Scott Pianowski analyzes who's been helping fantasy baseball managers win or causing frustration a month into the season.

The best QBs for 2024 fantasy football according to our experts

The Yahoo Fantasy football analysts reveal their first quarterback rankings for the 2024 season.

Rivian put out a feeler to test buyers' willingness to spend on a new R2

Members of the Rivian subreddit posted details of a survey they received, asking how much they'd be willing to spend on different R2 configurations.

Korey Cunningham, former NFL lineman, found dead in New Jersey home at age 28

Cunningham played 31 games in the NFL with the Cardinals, Patriots and Giants.

Best places to get your car maintained and repaired

Consumer Reports offers a quick guide, with a few examples, of when you should get your car maintained and repaired at a dealership, vs. an independent shop, vs. a chain.

Panthers owner David Tepper stopped by Charlotte bar that criticized his draft strategy

“Please Let The Coach & GM Pick This Year" read a sign out front.

2024 NFL Draft grades: Minnesota Vikings risked a lot to get J.J. McCarthy and Dallas Turner

Yahoo Sports' Charles McDonald breaks down the Vikings' 2024 draft.

Spot Training

Boston dynamics training services, learning pathways.

Select a pathway to learn more

Autonomous Inspections

Learn how to perform autonomous inspections with Spot using thermal and acoustic sensors. Learn more about the Autonomous Inspections learning pathway.

Spot and Orbit API

Learn how to use the Spot and Orbit API tools like Service Customizations, Area Callbacks, and Extensions for Spot for industrial applications. Learn more about the Spot and Orbit API pathway.

Spot Arm Manipulation

Learn how to use semi-autonomous actions, constrained manipulation, and beyond visual line of site manipulation with Spot Arm. Learn more about the Manipulation learning pathway.

Custom Training Options

Create the training option your team needs. Additional fees apply. For details contact your sales representative or log in below.

See course catalog

Log in to schedule

How It Works

Training services.

Boston Dynamics offers a flexible training program so you can gain practical skills and start working with Spot safely and successfully.

Spot Training is very hands on, so we recommend a group size of 4 to 6 people per trainer.

Larger groups may incur additional costs.

At Boston Dynamics headquarters

Training takes place at our Customer Learning Center in Waltham, MA. We provide robots, equipment, course materials, lunch, and a tour.

At your location

Customer covers travel, equipment, and shipping.

Customer provides operational robots and suitable, safe, space with a large screen and seating for the class.

To Boston Dynamics headquarters

Customer pays travel and lodging to Waltham, MA, USA. We can recommend hotels in the area if needed.

To your location

Customers pays travel and shipping as needed to facilitate training on-site. Additional fees may apply.

Log in to request training

Send us your scheduling and course details

Product Support

Safety & Compliance

©2024 Boston Dynamics. All rights reserved.

Salesforce is closed for new business in your area.