Category: Rescue Robots

#Mexico #Earthquake Overview of robots and earthquakes: background and how they can help

Our hearts go out to the victims, their families, and the responders in Mexico. CRASAR has not been contacted about robots but this blog may be of use in thinking about how to use robots.

Ground, aerial, and marine robots have been used in several earthquakes. A good overview of ground robots for structural collapse is at https://www.youtube.com/watch?v=5Cm2bGlUjbQ. It’s an older documentary but all the issues and gear are still the same.

Small unmanned aerial vehicles are probably everyone’s first thought for earthquakes, in part to map out the extent of the damage. They can also be used to help responders determine the shortest, most debris-free route to locations or interest.  For the reconstruction and recovery phases, UAS have been used to fly around and in large buildings that are suspected of being too dangerous for structural specialists to enter and assess the risk of further collapse- for example the cathedrals in Canterbury NZ, Mirandola ,Italy, and Amatrice, Italy.  After the Tohoku earthquake, many experts pointed out that UAVs should be used to determine the state of hospitals- both whether the hospital is still functional but also if it is being overwhelmed by patients.

Ground robots also have a role. An earthquake may cause buildings to completely collapse where there are no voids that a responder can get into. The general strategy is to use a canine team to determine if there are survivors in the rubble (dogs can tell if the person is still alive). Typically a boroscope or a camera on a wand is inserted to try to see if they can locate the survivors and also get a sense of the best was to remove material to get to them.  The boroscope or a camara on a wand can only go about 18 feet into the rubble, depending on how twisty the void is. In a major building collapse, survivors may be much further, which is why small, shoe-box sized robots such as an Inuktun VGTV may be used. The “cameras on tracks” robots can pull themselves into the rubble and also change shape to help get into tight spots.  These small robots will usually be tethered, with the tether acting as a belay line and the tether preventing loss of signal.

 

Some interesting robots are the snake robots being developed by Howie Choset at CMU and the Active Scope Camera caterpillar robot being developed by Olympus in conjunction with Japanese researchers led by Satoshi Tadokoro. The ASC was used at the Jacksonville building collapse in 2007.

 

Bigger ground robots, such as those used at Fukushima, can be used in bigger buildings but gepnerally can’t see the ceilings, which is usually very important and happened the New Zealand earthquake. They can’t go into small voids and may be too heavy- they could cause delicately balanced rubble to further collapse and kill a survivor underneath.

 

Marine robots, especially ROVs, are important as was seen in the Haiti earthquake and Tohoku earthquake and tsunami. The earthquake may have changed the shipping channels, damaged bridges and ports, and put debris in unexpected places. Thus shipping is stopped until the shipping channels are cleared- and as was seen at Haiti it’s hard to feed a country with one airport. Shipping is extremely important for getting relief supplies in.

 

There are more details in the case studies in Disaster Robotics  and the Springer Handbook of Robotics on what robots have been used and particular strategies. All of us are happy to answer questions. We wish everyone the best on this terrible event.

Update on Hurricane Irma Unmanned Aerial Systems: new record of 247 flights for public officials

FSU's Mike McDaniel with Collier County official flying a DJI MavicA major update from our earlier post    CRASAR was at Hurricane Irma supporting Roboticists Without Borders’ member Florida State University Center for Disaster Risk Policy deployment to Collier County, Florida. The CDRP effort was led by David Merrick, CDRP director, with Justin Adams, Kovar and Associates, who led the CRASAR Harvey response serving as his deputy. Collier County is in southern Florida near Naples and Marco Island. This is familiar territory to CRASAR, who responded to Hurricane Wilma in 2005 at Marco Island, sending out the first known use of unmanned marine vehicles. The county was exposed to severe wind damage and flooding from Irma which passed over as a Category 3 hurricane on September 10. The teams demobilized on September 16, with last flights on September 15.  Six UAS pilots flew 247 imaging flights covering over 491 critical infrastructure targets, as well as provided overwatch for FL Task Force 8 and made multiple maps. The flights started on September 11 and surpassed the record of 119 mission flights for public officials during a disaster set at Hurricane Harvey. FSU and Texas A&M plan to offer a joint day-long short course on small UAS for emergency management on October 21, expanding the course that they have taught at the Florida Governors Hurricane Conference in May 2017 and for Los Angeles County Fire Rescue last week. Contact Robin Murphy for more information about the Irma deployment and the upcoming course.

 

The FSU CDRP team under the direction of David Merrick was a major component of CRASAR’s record setting small UAS Hurricane Harvey response, leaving Texas with two days to prepare and predeploy for Irma. The use of small UAS at Irma initially followed a similar pattern to Hurricane Harvey, with UAS being used to assist with rapidly conducting search and rescue operations and determining the best route for US&R teams to reach people at risk but now has shifted to determining the state of over 1,500 critical infrastructure targets in the county. These targets, which include bridges and waste water treating stations,  impact the restoration of services and the economic recovery of the regions. The use of small UAS has significantly sped up the process, provide more complete assessment of all sides of a target, and multiple targets can be examined on one flight.

 

The FSU CDRP-led teams consist of pilots, data managers, platforms, and the RESPOND-R mobile lab. The teams are from FSU, Kovar and Associates, and CRASAR. The teams have 7 pilots with a fleet of 20 platforms including the DJI Mavic, DJI M600 Pro, Inspire, Intel Falcon 8, Disco, Phantom 3, and PrecisionHawk Lancaster rev 5. The teams are using the lessons learned at Harvey to improve rapid sortie planning, team situation awareness, and streamlining data management. Data is being collected by CRASAR and FSU from both deployments to permit developers to build better UAS optimized for the wind and operations tempo in a disaster and user interfaces that facilitate the data-to-decision process enabling county emergency management experts to rapidly get the right information and make good decisions. The data will also support the creation of  new visualization tools for responders, enabling them to sort through terabytes of aerial imagery, and serve as a foundation for machine learning and computer vision algorithms to process tetrabytes of data.

 

 

The Irma response differs from the Harvey response in at least four key ways.

  • the hurricane posed primarily wind damage with flooding as a secondary impact, whereas Harvey in Fort Bend County was primarily flooding from rainfall and then the risk of additional river flooding. This changed the style of search and rescue operations, especially as cellular coverage was affected and residents could not always call for help. The UAS teams assisted Florida Task Force 8 as the aerial view helped  the searchers determine where to go, which flooded house to inspect next, and to better coordinate operations. The wind damage in Irma made route clearing operations more important as downed trees could unpredictably be blocking roads as compared to flooding with tended to inundate specific areas and predicted from flood maps.
  • the majority of missions are for critical infrastructure property damage assessment. These assessments were normally being done in person.  This is time consuming for driving to the site (including determining alternative routes) and then requires a person’s time to survey the target. In many cases, the inspector cannot see or get to all sides of a facility. While both manual and UAS inspection require the same amount of time to get to a target, the use of UAS is being shown to be faster than walking around and more complete as the UAS can fly around the target and also provide an birds’ eye view as well. For example, on Sept. 13, two 2-person UAS teams were able to document 97 infrastructure targets with 12 flights. Our initial look at our logs indicate an average of 16 minutes on-site for a mission– that’s stop the car, turn on the UAS, fly the mission, get back in the car. And a flight covers an average of 2 targets per flight. It’s hard to believe that a person can walk around a tank farm faster.
  • A single flight (or sortie) at Irma typically covers multiple targets (an average of 2 targets), while at Harvey a single flight covered only one direct mission objective (though the aerial imagery was used to inform additional multiple emergency support functions).
  • Operationally, the resident population and airspace traffic is less dense, with less manned assets flying in the region and the operations did not require a Temporary Flight Restriction for safety. The Irma teams have less challenges in planning flights to avoid flying over people and did not encounter self-deployed teams or hobbyists interfering with UAS flights for officials; these challenges were seen at Hurricane Harvey.

Hurricane Harvey Deployment for Fort Bend County OEM: 119 flights over 11 days

[youtube]https://www.youtube.com/watch?v=Gm5tcJzS3Go&feature=em-subs_digest[/youtube]

Group photo, missing USAA and Hydronalix teams.
Group photo with TAMU, FSU, Kovar & Associates, LSUASC, PrecisionHawk and Intel, missing USAA and Hydronalix teams.

The Texas A&M Engineering Experiment Station Center for Robot-Assisted Search and Rescue (CRASAR) coordinated the largest known deployment of unmanned aerial systems (UAS) by public officials for a federally declared disaster- both serving as Air Operations for manned and unmanned aircraft and deploying small UAS ranging in size from DJI Mavics to the Insitu ScanEagle. The deployment was for the Fort Bend County Office of Emergency Management with whom CRASAR had provided assets for previous floods. CRASAR flew 119 mission flights from August 25 (preparing for landfall) to September 4 (when the emergency life-saving response and restoration of services phases of the disaster were largely over), with a record 61 flights on one day. The deployment was led by Justin Adams, who served as Air Operations branch director for Fort Bend County manned unmanned ops and CRASAR Roboticists Without Borders coordinator. Videos are available on Fort Bend County OEM’s website in accordance with the county’s drone data policy; there may be a backlog of posting due to the size of the event.

The UAS flights were for

  • rapid spot checks of situation awareness of people in distress
  • the extent of flood and tornado damage
  • how many people had not evacuated, access routes to neighborhoods
  • projecting how long the neighborhoods would be cut off, throughout the county (which is very large and hard to get a handle on)- based on information coming to EOC, the county’s projections and knowledge from 2016 floods- not just easiest or most compelling for media to fly or waiting for complete coverage by manned assets
  • inform the public and dispel rumors- allowed County Judge Hebert  to immediately and directly address Citizens’ concerns and dispel rumors, e.g., postings to social media about a particular neighborhood, then tasked to fly that neighborhood to get eyes on to inform the EOC as to the situation and to show the community; water is coming over the Richmond railroad bridge
  • systematically document damage for federal disaster relief and future planning
  • project river flooding by monitoring the river and confirming river flood inundation models
  • monitor the river and condition of over 100 miles of levees through out the county

There were additional flights for videography, training, and some flights were actually multiple flights where UAS had to return home for a battery change before continuing. Each mission flight was to satisfy a request made by officials throughout the county following their incident command system. CRASAR also directed Air Operations for the county, coordinating all manned and unmanned assets.

The center was requested by Fort Bend County Office of Emergency Management on Aug 24 and activated the Roboticists Without Borders program, which consists of companies, universities, and individual experts who have been trained in disaster response; volunteer their time, travel, and equipment; and conform to Fort Bend County OEM data management policies. The RWB brought in 24 unmanned aerial vehicles and 2 unmanned marine vehicles in from five institutional members: Florida State University Center for Disaster Risk Policy (5), GroundVu (2), Hydronalix (3),  Kovar & Associated (1), Lone Star UAS FAA Center of Excellence (3),  PrecisionHawk (1), and USAA (2) plus Intel (1). RWB provided 13 UAS pilots, 3 UMV operators, and 4 data managers. The team members were from 6 states: Arizona, California, Florida, Indiana, Texas, and Washington State. In addition PrecisionHawk donated five Lancaster UAS and access to their DataMapper software, Boeing Insitu deployed their Scan Eagle through Lone Star UASC,  and Intel loaned a Falcon 8 UAS designed from structural inspection, for a total of 25 UAS platforms. The platforms were, in alphabetical order, AirRobot 200, AirRobot 180, DJI Inspire, DJI Mavic, DJI M600 Pro, DJI Phantom 3 Pro, DJI Phantom 4 Pro, Insitu Scan Eagle, Intel Falcon 8, Parrot Disco, PrecisionHawk M100, PrecisionHawk Lancaster 5, 3DR Solo, and UAUSA Tempest.

In addition, there were other significant UAS donations by other groups. Rocky Mountain Unmanned Systems loaned a Z30 camera high resolution and high zoom payload that increased the area of view. AirMap adapted their popular UAS flight app for use by CRASAR, allowing the members to see which of their UAS were in the air at the same time as well as other UAS who reported that they were flying in the area. PrecisionHawk’s LATAS tags for the Lancasters were very helpful as well. LSUASC loaned access to Harris’ RangeVue software, allowing AirOperations to see manned flights and alert UAS teams of approaching low flying aircraft.  Other donations and support came from ESRI, FireWhat, GroundView, RemoteGeo, Salamander Technologies, Sweetwater Video, and TAC Aero.

Two EMILY unmanned marine vehicles, small robot boats, were available. One was outfitted with a side scan sonar and used to attempt to determine flow rates of the river. The other was outfitted for swift water but was not needed during the rescue phase of operations.

Hurricane Harvey: some video of flights in Fort Bend County

UAVs We’ve certainly been busy flying for Fort Bend County Office of Emergency Management plus our member Justin Adams serving as AirBoss for manned and unmanned aircraft- see some videos. The graphic is a quick look at the platforms on tap for the response here. Big shout out to Rocky Mountain Unmanned Systems and PrecisionHawk for their donation of equipment plus Insitu coming out with a Scan Eagle as part of the resources contributed by Lone Star UAS Center!

FBC policy is to post all their drone video- this worked great during the April 2016 floods where the people manning the phones could tell worried family members to go look at a particular video of the river by a senior assisted living facility.  They are a bit behind in posting but if you are interested some of the videos are now available on the Fort Bend County Office of Emergency Management YouTube channel.  The missions range from situation awareness of neighborhoods (who’s still there, how severely and long are they going to be cut off before the waters recede) to bridge and dam inspection. Our FSU Center for Disaster Risk Policy team has been flying extensively and even flew off a flat bottom boat.

Hurricane Harvey: How to Fly at Hurricane Harvey

Update: I was reminded to remind everyone about Texas UAV privacy laws – we can’t fly for things like damage assessment without the property owner’s permission unless you are explicitly working for a state agency. Louisiana laws may be different, be sure and check out the rules! Also, we’re rapidly moving out of the immediate life-saving response phase and into the initial recovery phase.
Even before I posted about flying since Friday, we’re getting swamped with requests from pilots asking about i) volunteering to fly for Roboticists Without Borders or ii) about volunteering for an agency or iii) telling me that they are self-deploying and asking where is the best place to go. Unfortunately, I don’t have the answers most people want to hear. I would direct most people to read Disaster Robotics, where in the last chapter I go through working with response professionals.  Flying for a disaster is very different than flying for a job and the best practices and use case aren’t enough  (those assumed you were trained for disasters)- I don’t have time to go through why it is so different, but here are some points to consider:
It’s too late to join Roboticists Without Borders to fly for the Harvey response. Everyone who is deployed with the team has to have been trained and participated in at least one of our disaster exercises before deploying to an incident. Disasters are different, just knowing how to fly isn’t enough. Think about all those cop movies or movies about or special ops like “Zero Dark Thirty,” there’s a whole language, expectations, working conditions, and a whole lot of prior training that’s involved. If you don’t have the equipment, clothing, the training and knowledge of how to fit in, then you slow the whole response down and worse yet may put yourself at risk. One of the reasons we are extended formal invitations to participate in disasters is that we only deploy people who have been trained. We are having a training exercise in November, though we may have another one sooner.
Self-deploying is illegal and unhelpful. The illegal part should be self-explanatory. As I describe in Disaster Robotics, the unhelpful part stems from the agencies being overwhelmed with their tasks and unable to absorb new technologies or anything that changes what they trained for. The idea of flying on your own, then sharing your data is interesting- sadly handing a thumb drive of video to someone at the front desk of an Emergency Operations Center doesn’t ensure anyone will ever see that video. If you read our papers and best practices, you’ll see that managing the data and sorting out what is important for which groups in a response is very important. So just flying and collecting data is maybe 25% of the job. If you aren’t associated with a response agency, you don’t have anyway of doing the other 75% of the job of getting the right information to the right people in time for them to make a right decision. 
 
It is generally too late to reach out an agency and even have them return your call. Again, as pointed out in Disaster Robotics, agencies can’t handle something new or a change in their procedures or anything that impacts manpower; are accountable to the public so must have vetting that the person is good at UAVs AND can work at a disaster; can’t handle the increased footprint (food, shelter, sanitation, gas) of more people. We  sent two team members back on Monday and one on Tuesday from our UMV team because we weren’t using them. We normally have a dedicated data manager but sent them back to further reduce our footprint. There are no gas stations open or hotels that aren’t already booked.
If you are flying at Harvey, even for an agency, be aware of manned and unmanned aircraft and TFRs. The Part 107 exam isn’t sufficient for understanding how disasters work- for example each jurisdiction will have an Airboss, director of Air Operators, and UAS have to coordinate with them. (Justin Adams is serving as AirBoss for Fort Bend.)  TFRs are important. Fort Bend just posted a TFR which the FAA emailed everyone about. The TFR means you can’t fly without explicitly coordinating with the agency posted as holding the TFR (generally the incident commanders designate an agency to manage it- often it is someone from the forestry service).  As you fly in areas without a TFR, there are a lot of tactical helicopter operations and medivac ops plus Blackhawks zooming around at low altitudes that you have to be aware of.  These things don’t show up airmap.io. Plus the use of UAS may not be advertised or posting to social media but many EOCs are using them, for example, CRASAR has been flying all over Fort Bend since Friday but didn’t post anything to social media about until yesterday (because publicity is low priority).
Which leads to, if you are flying, please remind the agency you are working for to check with their ICS staff on the AirOps hierarchy. We’ve seen agencies that fall under the county or city jurisdictions not realize that they need to coordinate their UAV use with AirOps just like they would if they were using a manned helicopter or CAP asset.  And if they promise to pay a UAV team and didn’t follow the hierarchy, they don’t get paid and thus may not pay the UAV team.
Finally be sensitive to the citizens. High resolution images of someone trapped on a roof is moving and compelling video but it can also come across as the pilot trying to benefit from someone’s suffering (even if you aren’t being paid, “likes” on a YouTube count in this category).

Hurricane Harvey: CRASAR deployed since 8/25 to Fort Bend County OEM

Experts in unmanned aerial and marine systems from the Texas A&M’s Center for Robot-Assisted Search and Rescue (CRASAR) have been assisting Fort Bend County Office of Emergency Management with Hurricane Harvey. Volunteer teams from the center’s Roboticists Without Borders program arrived on Friday and began immediately using small drones to support rapidly mapping areas at risk and estimating flooding, while another team employed a miniature robot boat with sonar to project river flow rates.   As Harvey hit, the drone teams shifted to surveying tornado damage and identifying neighbors cut off and in need of help. The boat, which is covered with enough floatation to support people and can pull a line to trapped people, has stayed on call for swift water rescue. CRASAR lead pilot, Justin Adams, is serving as Air Operations branch director, coordinating all air operations for manned and unmanned aircraft within the county.

The drone teams are drawn from researchers and students at the Texas A&M Engineering Experiment Station plus  Adams and Jess Gingrich (USAA). The teams use three different inexpensive DJI platforms.  The EMILY robot boats and experts are provided by Hydronalix, an Arizona company who has been active with CRASAR in humanitarian rescue of boat refugees in Greece, and lead by Capt. John Sims. A drone team from the Florida State University Center for Disaster Risk Policy will join the effort later in the week.

This is the fifth hurricane response that CRASAR has participated in. Under the direction of Dr. Robin Murphy, a professor of computer science and engineering at Texas A&M, CRASAR was the first group to fly small drones for a disaster, which was at Hurricane Katrina in 2005. The center has been working with Fort Bend County OEM for two years, learning from the county’s diverse emergency professionals on best practices for applying economical unmanned systems to save lives and accelerate economic recovery after a major meteorological event.

Videos from the unmanned system teams will be available through the Fort Bend County Office of Emergency Management YouTube channel.

Hurricane Harvey: CRASAR deployed since 8/25 to Fort Bend County OEM

Experts in unmanned aerial and marine systems from the Texas A&M’s Center for Robot-Assisted Search and Rescue (CRASAR) have been assisting Fort Bend County Office of Emergency Management with Hurricane Harvey. Volunteer teams from the center’s Roboticists Without Borders program arrived on Friday and began immediately using small drones to support rapidly mapping areas at risk and estimating flooding, while another team employed a miniature robot boat with sonar to project river flow rates.   As Harvey hit, the drone teams shifted to surveying tornado damage and identifying neighbors cut off and in need of help. The boat, which is covered with enough floatation to support people and can pull a line to trapped people, has stayed on call for swift water rescue. CRASAR lead pilot, Justin Adams, is serving as Air Operations branch director, coordinating all air operations for manned and unmanned aircraft within the county.

The drone teams are drawn from researchers and students at the Texas A&M Engineering Experiment Station plus  Adams and Jess Gingrich (USAA). The teams use three different inexpensive DJI platforms.  The EMILY robot boats and experts are provided by Hydronalix, an Arizona company who has been active with CRASAR in humanitarian rescue of boat refugees in Greece, and lead by Capt. John Sims. A drone team from the Florida State University Center for Disaster Risk Policy will join the effort later in the week.

This is the fifth hurricane response that CRASAR has participated in. Under the direction of Dr. Robin Murphy, a professor of computer science and engineering at Texas A&M, CRASAR was the first group to fly small drones for a disaster, which was at Hurricane Katrina in 2005. The center has been working with Fort Bend County OEM for two years, learning from the county’s diverse emergency professionals on best practices for applying economical unmanned systems to save lives and accelerate economic recovery after a major meteorological event.

Videos from the unmanned system teams will be available through the Fort Bend County Office of Emergency Management YouTube channel.

Hurricane Harvey: All about how unmanned aerial and marine vehicles have been used for flooding

With Hurricane Harvey approaching, here’s a list of our recent posts about using unmanned aerial and marine vehicles for flood disasters that may be helpful. Also don’t forget our 1 page best practices guides, especially Best Practices for Small Unmanned Aerial Systems for Floods

And yes, Roboticists Without Borders has been requested and is standing up, with Justin Adams as the UAS lead.

Free UAS Awareness and Best UAS Practices for Emergency Management Class at Governor’s Hurricane Conference

Roboticists Without Borders offered two sessions of a 3.5 hour class consisting of three modules: unmanned systems awareness, unmanned aerial systems awareness, and best UAS practices at the 2017 Governor’s Hurricane Conference in West Palm Beach, Florida. Over 35 emergency professionals representing over 28 local and state agencies attended and received certificates for participation. The class targeted chiefs and managers who are interested in what UAS (and robots in general) have been used for, what are the costs including the hidden costs of manpower, training and maintenance, what are the regulatory issues, and how to handle public perception. The class also went through the types of missions involved in each major type of disaster and the associated unique CONOPS and workflows for each mission. The class emphasizes data management and how to get, and share, actionable data in real-time.

The modules were created by Florida State University Emergency Management and Homeland Security Program and the Texas A&M Engineering Experiment Station Center for Robot-Assisted Search and Rescue. The material is based on formal training created by CRASAR originally funded by the State of Florida, plus lessons learned from over 30 deployments by RWB members, and studies by FSU and Texas A&M. The modules are normally incorporated in a longer class with hands-on demonstrations of unmanned systems.

The class is offered for free as part of the RWB mission to accelerate the adoption of unmanned systems by emergency professionals.

Columbia mudslide: recommendations for UAVs for search and for public works

Our thoughts and prayers are with the Columbians in the wake of the terrible flooding and mudslide.

CRASAR’s experiences with such situations suggest that it is a very difficult search and rescue (and victim recovery) problem. We’re assisted with the Oso Mudslides in partnership with the Fit Innovation Team (some of the video is the highlight reel) and also with the 2015 Texas Memorial Day floods with Lone Star UAS Center which swept over 40 people over a 5 mile area of remote river wilderness. Please note that while our UAV flights at the Memorial Day floods were victim search and recovery missions, the Oso Mudslides was for Public Works. When a disaster happens, while search and rescue teams are working, the engineering experts are also working to mitigate and prevent further catastrophes and to start on economic recovery. Both are important missions!

Here’s some videos that we prepared for the White House and Congress on the use of UAVs, artificial intelligence, and informatics technologies.

From our best practices guides, here are some recommendations for UAV operators:

Standard procedure is to take high resolution imagery and then have a group of trained experts examine each image.  Crowd sourcing can have two problems if not done correctly. One is that most people make major mistakes interpreting aerial images, particularly when the images may be from different altitudes or looking straight down. Hence “trained experts.” Formal methods exist for rating the accuracy of  people looking at the image (called coders). The other is unintentional violations of privacy– putting out images that may contain victims and saying “hey, everyone, come look at this”.

Since the images are geotagged, it doesn’t matter which images they look at.

Video generally isn’t helpful because of lower resolution and fuzziness when you try to pause.

Victims may be covered in mud and buried in debris so clumps large enough to contain a body may be put on the list for investigation by a ground team