#Mexico #Earthquake Overview of robots and earthquakes: background and how they can help

Our hearts go out to the victims, their families, and the responders in Mexico. CRASAR has not been contacted about robots but this blog may be of use in thinking about how to use robots.

Ground, aerial, and marine robots have been used in several earthquakes. A good overview of ground robots for structural collapse is at https://www.youtube.com/watch?v=5Cm2bGlUjbQ. It’s an older documentary but all the issues and gear are still the same.

Small unmanned aerial vehicles are probably everyone’s first thought for earthquakes, in part to map out the extent of the damage. They can also be used to help responders determine the shortest, most debris-free route to locations or interest.  For the reconstruction and recovery phases, UAS have been used to fly around and in large buildings that are suspected of being too dangerous for structural specialists to enter and assess the risk of further collapse- for example the cathedrals in Canterbury NZ, Mirandola ,Italy, and Amatrice, Italy.  After the Tohoku earthquake, many experts pointed out that UAVs should be used to determine the state of hospitals- both whether the hospital is still functional but also if it is being overwhelmed by patients.

Ground robots also have a role. An earthquake may cause buildings to completely collapse where there are no voids that a responder can get into. The general strategy is to use a canine team to determine if there are survivors in the rubble (dogs can tell if the person is still alive). Typically a boroscope or a camera on a wand is inserted to try to see if they can locate the survivors and also get a sense of the best was to remove material to get to them.  The boroscope or a camara on a wand can only go about 18 feet into the rubble, depending on how twisty the void is. In a major building collapse, survivors may be much further, which is why small, shoe-box sized robots such as an Inuktun VGTV may be used. The “cameras on tracks” robots can pull themselves into the rubble and also change shape to help get into tight spots.  These small robots will usually be tethered, with the tether acting as a belay line and the tether preventing loss of signal.

 

Some interesting robots are the snake robots being developed by Howie Choset at CMU and the Active Scope Camera caterpillar robot being developed by Olympus in conjunction with Japanese researchers led by Satoshi Tadokoro. The ASC was used at the Jacksonville building collapse in 2007.

 

Bigger ground robots, such as those used at Fukushima, can be used in bigger buildings but gepnerally can’t see the ceilings, which is usually very important and happened the New Zealand earthquake. They can’t go into small voids and may be too heavy- they could cause delicately balanced rubble to further collapse and kill a survivor underneath.

 

Marine robots, especially ROVs, are important as was seen in the Haiti earthquake and Tohoku earthquake and tsunami. The earthquake may have changed the shipping channels, damaged bridges and ports, and put debris in unexpected places. Thus shipping is stopped until the shipping channels are cleared- and as was seen at Haiti it’s hard to feed a country with one airport. Shipping is extremely important for getting relief supplies in.

 

There are more details in the case studies in Disaster Robotics  and the Springer Handbook of Robotics on what robots have been used and particular strategies. All of us are happy to answer questions. We wish everyone the best on this terrible event.

Update on Hurricane Irma Unmanned Aerial Systems: new record of 247 flights for public officials

FSU's Mike McDaniel with Collier County official flying a DJI MavicA major update from our earlier post    CRASAR was at Hurricane Irma supporting Roboticists Without Borders’ member Florida State University Center for Disaster Risk Policy deployment to Collier County, Florida. The CDRP effort was led by David Merrick, CDRP director, with Justin Adams, Kovar and Associates, who led the CRASAR Harvey response serving as his deputy. Collier County is in southern Florida near Naples and Marco Island. This is familiar territory to CRASAR, who responded to Hurricane Wilma in 2005 at Marco Island, sending out the first known use of unmanned marine vehicles. The county was exposed to severe wind damage and flooding from Irma which passed over as a Category 3 hurricane on September 10. The teams demobilized on September 16, with last flights on September 15.  Six UAS pilots flew 247 imaging flights covering over 491 critical infrastructure targets, as well as provided overwatch for FL Task Force 8 and made multiple maps. The flights started on September 11 and surpassed the record of 119 mission flights for public officials during a disaster set at Hurricane Harvey. FSU and Texas A&M plan to offer a joint day-long short course on small UAS for emergency management on October 21, expanding the course that they have taught at the Florida Governors Hurricane Conference in May 2017 and for Los Angeles County Fire Rescue last week. Contact Robin Murphy for more information about the Irma deployment and the upcoming course.

 

The FSU CDRP team under the direction of David Merrick was a major component of CRASAR’s record setting small UAS Hurricane Harvey response, leaving Texas with two days to prepare and predeploy for Irma. The use of small UAS at Irma initially followed a similar pattern to Hurricane Harvey, with UAS being used to assist with rapidly conducting search and rescue operations and determining the best route for US&R teams to reach people at risk but now has shifted to determining the state of over 1,500 critical infrastructure targets in the county. These targets, which include bridges and waste water treating stations,  impact the restoration of services and the economic recovery of the regions. The use of small UAS has significantly sped up the process, provide more complete assessment of all sides of a target, and multiple targets can be examined on one flight.

 

The FSU CDRP-led teams consist of pilots, data managers, platforms, and the RESPOND-R mobile lab. The teams are from FSU, Kovar and Associates, and CRASAR. The teams have 7 pilots with a fleet of 20 platforms including the DJI Mavic, DJI M600 Pro, Inspire, Intel Falcon 8, Disco, Phantom 3, and PrecisionHawk Lancaster rev 5. The teams are using the lessons learned at Harvey to improve rapid sortie planning, team situation awareness, and streamlining data management. Data is being collected by CRASAR and FSU from both deployments to permit developers to build better UAS optimized for the wind and operations tempo in a disaster and user interfaces that facilitate the data-to-decision process enabling county emergency management experts to rapidly get the right information and make good decisions. The data will also support the creation of  new visualization tools for responders, enabling them to sort through terabytes of aerial imagery, and serve as a foundation for machine learning and computer vision algorithms to process tetrabytes of data.

 

 

The Irma response differs from the Harvey response in at least four key ways.

  • the hurricane posed primarily wind damage with flooding as a secondary impact, whereas Harvey in Fort Bend County was primarily flooding from rainfall and then the risk of additional river flooding. This changed the style of search and rescue operations, especially as cellular coverage was affected and residents could not always call for help. The UAS teams assisted Florida Task Force 8 as the aerial view helped  the searchers determine where to go, which flooded house to inspect next, and to better coordinate operations. The wind damage in Irma made route clearing operations more important as downed trees could unpredictably be blocking roads as compared to flooding with tended to inundate specific areas and predicted from flood maps.
  • the majority of missions are for critical infrastructure property damage assessment. These assessments were normally being done in person.  This is time consuming for driving to the site (including determining alternative routes) and then requires a person’s time to survey the target. In many cases, the inspector cannot see or get to all sides of a facility. While both manual and UAS inspection require the same amount of time to get to a target, the use of UAS is being shown to be faster than walking around and more complete as the UAS can fly around the target and also provide an birds’ eye view as well. For example, on Sept. 13, two 2-person UAS teams were able to document 97 infrastructure targets with 12 flights. Our initial look at our logs indicate an average of 16 minutes on-site for a mission– that’s stop the car, turn on the UAS, fly the mission, get back in the car. And a flight covers an average of 2 targets per flight. It’s hard to believe that a person can walk around a tank farm faster.
  • A single flight (or sortie) at Irma typically covers multiple targets (an average of 2 targets), while at Harvey a single flight covered only one direct mission objective (though the aerial imagery was used to inform additional multiple emergency support functions).
  • Operationally, the resident population and airspace traffic is less dense, with less manned assets flying in the region and the operations did not require a Temporary Flight Restriction for safety. The Irma teams have less challenges in planning flights to avoid flying over people and did not encounter self-deployed teams or hobbyists interfering with UAS flights for officials; these challenges were seen at Hurricane Harvey.

Irma: Some Differences About Flying for Disasters Than from Regular Flying for Videography, Construction, and Agriculture

Please see our report on our 119 flights at Hurricane Harvey for Fort Bend County and Chapter 6 of Disaster Robotics on deploying to disasters.

It sounds like from social media that Florida and the surrounding states are going to have a lot of self-deployed UAS teams for the initial response and restoration phases and then later businesses may engage UAS services. Here are some observations that may be of use.
Keep in mind that the State of Florida and its State Emergency Response Team has worked for several years to integrate small UAS into their program and created state-wide policies for Hurricane Matthew. They flew UAS in Katrina (CRASAR were part of the team) and Rita. The Florida State University Disaster Incident Response Team (a CRASAR member and they flew at Harvey) is already in place, at least one state US&R team uses small UAS to determine travel routes and get local situation awareness, so official groups may be flying and they may be a state-wide plan, even if you aren’t aware of it.
The areas you think should be flown may be under a Temporary Flight Restriction (TFR)
  • You can see this through AirMap
  • A Temporary Flight Restriction is the FAA equivalent of the Police blocking off a road. Instead of police cars and tape, the FAA does it by posting the TFR to aviation websites. It’s the law. If you think the police shouldn’t have blocked off an area, you can’t say  “I’m doing a Thelma and Louise on the state highway patrol blockade because I’m a humanitarian” and not expect to get arrested or that you are interfering with how the authorities in charge are trying to handle the situation.
  • Remember, if you didn’t have a pilot’s license or could fly BVLOS before, you can’t get permission to fly in the off-limits area. Going back to the car analogy, if you didn’t have a commercial driver’s license to drive big trucks before the police waived you through the blockage, you can’t drive big trucks now.
  • You need a government agency sponsor for the E-COA
  • However, some smaller jurisdictions may not realize that having you fly a small UAS is the same as having a manned helicopter fly and that they need to go through the official air operations branch of the response to coordinate manned flights. So if they don’t follow official incident management command protocols, especially in Florida where there is a state policy, there are problems from that jurisdiction and their overall response efforts could become ineligible for reimbursement (which are dependent on following protocols).
There may be a high density of low-altitude manned aircraft, some of which may be flying very low, especially in urban and suburban areas. We saw one helicopter zooming at 50 feet AGL and many under 400 feet. At those altitudes, a helicopter having to suddenly swerve to avoid a UAS is dangerous to the people on board. Imagine what would happen if a UAS hit a windshield— even if it didn’t go through, the pilot may react in a way that leads to a crash.
  • have a plan for what to do. Sometimes you may have to “park it in the trees” because you can’t return to home fast enough or know where the hell is going. Sometimes you may have to go high, like when a crop duster descends because you they are going lower.
  • a VHF radio to monitor their traffic and to post that you are flying can be of great value; it’s nice to have an experienced civilian pilot on the team to help with this
  • Harris RangeVue or other ATC software package can be very valuable as well, as someone can monitor this and then radio out to the teams as to approaching aircraft and if it is going to enter the area where the UAS is operating and thus needs to get out of the way. Big shout out to Lone Star UAS Center, as a CRASAR member, we were able to use this
  • Tags such as PrecisionHawk’s LATAS were very, very helpful for us at Harvey
  • Don’t fly BVLOS, because you just can’t see what’s going on with a low flying manned aircraft (yes, they may be in violation of airspace regulations but there are people’s lives at risk- the pilot and people being evacuated or rushed for medical aid)
Access to launch/landing spots is very limited and generally not what you would have picked for normal operations and may not be safe
  • It often in very crowded settings with lots of trees.
  • Personal safety is more at risk. Think snakes. And floating balls of angry red ants. And swarming insects. And in Florida- alligators. Not only can it be dangerous, but by definition, no one can come rescue you.
  • Back to UAS safety: Be prepared for “walking off” or “fly aways”  high tension power lines- which can interfere with electronics, especially compasses. Combine that with heavy cloud cover reducing GPS coverage and you sometimes get unpredictable behavior from even the most reliable UAS.
If you do not have an agency official with you, then have lots of handouts about the agency’s services to give to locals and be able to explain the agencies’ data policy so they won’t get angry at you. In 2015 and 2016 we have been approached by locals who were initially hostile about the use of UAS, thinking that we were there to get pretty pictures to post to YouTube instead of directly helping them). In both Louisiana and Texas, having an agency person to confirm that it was for the agency and to answer questions turned tense moments into moments where the citizens were proud of their government.  For example, Fort Bend County sent CRASAR teams out with a health and human services representative to the densely populated areas just so there was someone knowledgable to answer questions and help (which is better than giving them the phone number for the EOC).  Fort Bend County’s policy is that all drone video is posted to their YouTube channel so we could direct them there. Even in rural areas people will see the drone and come over to ask questions.
If you aren’t flying for an agency and hoping to donate the video or images
  • if you find people in immediate distress or an immediately threatening situation (we found an unreported fire), call 911 and the agency you are working for (they will also try to get the info to the right group) if the cell phones are working
  • please don’t post to social media
    • emergencies managers won’t necessarily be looking at social media
    • they probably won’t have good internet access with electricity and cell towers being down due to wind damage
    • the agency may not be able to use the data because there is no chain of custody or way to verify that it is accurate; people have started rumors and posted photoshopped video in the past so there’s a tendency to ignore contributions
    • you may be liable for violations of personal identifiable information
  • provide a standard filename convention where members of the different response functions can easily find the data they need. Usually location and data
    • GPS coordinates are hard for all groups to parse, so street location (put a pin in Google Maps of where you, get an address, and then take a screenshot on your phone) or neighborhood if you know it
    • Provide a master kmz of all the locations you flew and annotations of anything particularly interesting
    • Think of this as documenting a crime scene and what  would be needed to admit this as evidence; that helps with documenting
Think through your data products. Photogrammetrics are nice but waiting 12 hours to see the extent of wind damage may be too late for anyone to make timely decisions.
  • Still images at nadir are extremely hard for normal humans to comprehend. For general assessment, video FPV may be the most effective.
  • Microsoft ICE is much faster than packages such as PrecisionHawk’s Datamapper which we used, which produce georeferenced pixels. In the initial week of the disaster, the responders didn’t need that level of detail anyway- it is “general outline of how bad is bad”  At Harvey we didn’t begin photogrammetric flights until after a week.
  • Remember that agencies may not have the ability to store terabytes of images, videos, and orthomosiacs. As we found in 2015 and 2016, no one can email a high resolution ortho of an area. But they can email a “framing shot” .jpg.

Hurricane Harvey Deployment for Fort Bend County OEM: 119 flights over 11 days

[youtube]https://www.youtube.com/watch?v=Gm5tcJzS3Go&feature=em-subs_digest[/youtube]

Group photo, missing USAA and Hydronalix teams.
Group photo with TAMU, FSU, Kovar & Associates, LSUASC, PrecisionHawk and Intel, missing USAA and Hydronalix teams.

The Texas A&M Engineering Experiment Station Center for Robot-Assisted Search and Rescue (CRASAR) coordinated the largest known deployment of unmanned aerial systems (UAS) by public officials for a federally declared disaster- both serving as Air Operations for manned and unmanned aircraft and deploying small UAS ranging in size from DJI Mavics to the Insitu ScanEagle. The deployment was for the Fort Bend County Office of Emergency Management with whom CRASAR had provided assets for previous floods. CRASAR flew 119 mission flights from August 25 (preparing for landfall) to September 4 (when the emergency life-saving response and restoration of services phases of the disaster were largely over), with a record 61 flights on one day. The deployment was led by Justin Adams, who served as Air Operations branch director for Fort Bend County manned unmanned ops and CRASAR Roboticists Without Borders coordinator. Videos are available on Fort Bend County OEM’s website in accordance with the county’s drone data policy; there may be a backlog of posting due to the size of the event.

The UAS flights were for

  • rapid spot checks of situation awareness of people in distress
  • the extent of flood and tornado damage
  • how many people had not evacuated, access routes to neighborhoods
  • projecting how long the neighborhoods would be cut off, throughout the county (which is very large and hard to get a handle on)- based on information coming to EOC, the county’s projections and knowledge from 2016 floods- not just easiest or most compelling for media to fly or waiting for complete coverage by manned assets
  • inform the public and dispel rumors- allowed County Judge Hebert  to immediately and directly address Citizens’ concerns and dispel rumors, e.g., postings to social media about a particular neighborhood, then tasked to fly that neighborhood to get eyes on to inform the EOC as to the situation and to show the community; water is coming over the Richmond railroad bridge
  • systematically document damage for federal disaster relief and future planning
  • project river flooding by monitoring the river and confirming river flood inundation models
  • monitor the river and condition of over 100 miles of levees through out the county

There were additional flights for videography, training, and some flights were actually multiple flights where UAS had to return home for a battery change before continuing. Each mission flight was to satisfy a request made by officials throughout the county following their incident command system. CRASAR also directed Air Operations for the county, coordinating all manned and unmanned assets.

The center was requested by Fort Bend County Office of Emergency Management on Aug 24 and activated the Roboticists Without Borders program, which consists of companies, universities, and individual experts who have been trained in disaster response; volunteer their time, travel, and equipment; and conform to Fort Bend County OEM data management policies. The RWB brought in 24 unmanned aerial vehicles and 2 unmanned marine vehicles in from five institutional members: Florida State University Center for Disaster Risk Policy (5), GroundVu (2), Hydronalix (3),  Kovar & Associated (1), Lone Star UAS FAA Center of Excellence (3),  PrecisionHawk (1), and USAA (2) plus Intel (1). RWB provided 13 UAS pilots, 3 UMV operators, and 4 data managers. The team members were from 6 states: Arizona, California, Florida, Indiana, Texas, and Washington State. In addition PrecisionHawk donated five Lancaster UAS and access to their DataMapper software, Boeing Insitu deployed their Scan Eagle through Lone Star UASC,  and Intel loaned a Falcon 8 UAS designed from structural inspection, for a total of 25 UAS platforms. The platforms were, in alphabetical order, AirRobot 200, AirRobot 180, DJI Inspire, DJI Mavic, DJI M600 Pro, DJI Phantom 3 Pro, DJI Phantom 4 Pro, Insitu Scan Eagle, Intel Falcon 8, Parrot Disco, PrecisionHawk M100, PrecisionHawk Lancaster 5, 3DR Solo, and UAUSA Tempest.

In addition, there were other significant UAS donations by other groups. Rocky Mountain Unmanned Systems loaned a Z30 camera high resolution and high zoom payload that increased the area of view. AirMap adapted their popular UAS flight app for use by CRASAR, allowing the members to see which of their UAS were in the air at the same time as well as other UAS who reported that they were flying in the area. PrecisionHawk’s LATAS tags for the Lancasters were very helpful as well. LSUASC loaned access to Harris’ RangeVue software, allowing AirOperations to see manned flights and alert UAS teams of approaching low flying aircraft.  Other donations and support came from ESRI, FireWhat, GroundView, RemoteGeo, Salamander Technologies, Sweetwater Video, and TAC Aero.

Two EMILY unmanned marine vehicles, small robot boats, were available. One was outfitted with a side scan sonar and used to attempt to determine flow rates of the river. The other was outfitted for swift water but was not needed during the rescue phase of operations.

Hurricane Harvey: some video of flights in Fort Bend County

UAVs We’ve certainly been busy flying for Fort Bend County Office of Emergency Management plus our member Justin Adams serving as AirBoss for manned and unmanned aircraft- see some videos. The graphic is a quick look at the platforms on tap for the response here. Big shout out to Rocky Mountain Unmanned Systems and PrecisionHawk for their donation of equipment plus Insitu coming out with a Scan Eagle as part of the resources contributed by Lone Star UAS Center!

FBC policy is to post all their drone video- this worked great during the April 2016 floods where the people manning the phones could tell worried family members to go look at a particular video of the river by a senior assisted living facility.  They are a bit behind in posting but if you are interested some of the videos are now available on the Fort Bend County Office of Emergency Management YouTube channel.  The missions range from situation awareness of neighborhoods (who’s still there, how severely and long are they going to be cut off before the waters recede) to bridge and dam inspection. Our FSU Center for Disaster Risk Policy team has been flying extensively and even flew off a flat bottom boat.