NBC Nightly News on Aug. 28, 2018, featured CRASAR deployed ground robots from the 9/11 World Trade Center, as well as marine vehicles used at the Japanese Tsunami and the Syrian Boat Refugee Crisis, plus UAVs at Harvey. The coverage is unusual that the producer deliberately wanted to focus on ground and marine robots, which is great because emergency professionals often forget about the value of these assets. But UAVs made it in there and many thanks to Fort Bend County Judge Hebert for his comments. The piece was expected to be longer but a whole lot of “hard news” has been happening. Robin Murphy represented CRASAR plus her work at Texas A&M, but the real stars are the members of CRASAR and Roboticists Without Borders and our agency partners!
With the extreme weather throughout the US, we’ve put together a 1-page just-in-time guide for public safety agencies capturing the 6 major recommendations for using sUAS during the response phase of the disaster. The guide is based on CRASAR’s deployment to four federally declared flooding events, including Hurricane Harvey, in the past 3 years for the Fort Bend County Office of Emergency Management, Tangipahoa Parish Office of Emergency Management, Washington Parish Office of Emergency, and the American Red Cross. The guide also summarizes the 4 most frequent missions for floods. The purpose is to help emergency responders confidently and effectively incorporate small UAS into their flood response activities. There are more lessons learned than can fit on one page; so check out our 13 April, 2018, webinar link to the YouTube clip which goes into more detail and gives the quantitative evidence for the recommendations. Contact email@example.com if you are a public safety agency and need advice or assistance.
Having recently supported the response to Hawaii volcano eruption at Kilauea Volcano Lower East Rift Zone, we offer the following insights and lessons learned regarding use of UAVs for volcano eruption response.
Night flights of UAVs are very effective.
Manned aviation generally cannot fly at night, meaning responders are essentially blind for 8-12 hours. But UAVs can fly at night. There are no challenges coordinating with manned aircraft since they are not flying, and it is easier to keep the UAV in visible line of sight.
Rotorcraft UAVs can effectively sample gas.
We used the Flymotion stinger attachment to carry a four-gas meter and map the Kilauea plume. Caution: If your UAV isn’t designed for attaching payloads, it may be unstable and crash.
Rotorcraft UAVs with thermal sensors are very effective.
Rotorcraft UAVs with thermal sensors are particularly effectivefor determining new lava flows from old flows and for seeing at night. We found the DJI Zenmuse XT2 to be particularly useful. It produces radiometric jpgs where the pixel value is the temperature.
Rotorcraft UAVs provide a quick look at lava flow rates.
Rotorcraft UAVs provide a quick means of determining how fast lava is moving. Fly the UAV with a nadir view to the edge of the lava, get the GPS coordinate, repeat in a few minutes, do the math.
Plumes will interfere with photogrammetric mapping.
Plumes will interfere with photogrammetric mapping. Expect errors, lower visibility, and failed stitches.
Hanger 360 rapidly produced panoramas.
Hanger 360 rapidly produced panoramas without flying over property or people. Even better, the panorama resides in the web so responders don’t have to have download apps or have a special mouse to move through the image. And, they can easily email or text url to others.
A five person volunteer team from the Center for Robot-Assisted Search and Rescue deployed to the Leilani Estates Eruption event (Hawaii) from May 14-19, 2018 with a cache of small unmanned aerial systems rotorcraft and sensors. CRASAR supplemented existing sUAS capabilities from the University of Hawaii Hilo, allowing them to focus on geographical and volcanology aspects, while providing expertise in tactical operations. During the 16 night flights when manned aviation was prohibited, the CRASAR team identified a new fissure and provided the data used by the USGS to determine the rapid acceleration of the lava flow. Other day missions charted SO2 emissions, mapped fissures, assisted with a live save, supplied tactical situation awareness, and reduced the risks and costs of manned helicopter flights. The sUAS flights were notable for the extent of night flying, flight operations in unfavorable rain and wind conditions, the use of the new X5S thermal sensor, air sampling with rotorcraft, the use of Hangar Enterprise Platform for rapid reconnaissance, flying at 1,000 ft AGL, and live streaming over FirstNet. The CRASAR pilots were Justin Adams (Constellation Consulting Group), David Merrick and Laura Hart (Florida State University Center for Disaster Risk Policy), Jon McBride (Rocky Mountain Unmanned Systems), and Robin Murphy (Texas A&M University). Funding was provided in part through research grants from an insurance partner and the National Science Foundation. At this time, the imagery and video collected at the event has not been released to the public by the Hawaii Civil Defense, and thus we cannot share it.
We’ve been watching the Hawaii volcano with interest- it is a first of its kind event for small UAS. Check out the short podcast. And hope the volcano doesn’t blow… best wishes to the responders and the people impacted.
With the 2018 hurricane season just weeks away, the independent not-for-profit Center for Robot-Assisted Search and Rescue (CRASAR) announced the first ever Disaster Robotics Awards. Awards were announced for the 2018 Ground, Aerial, and Marine Rescue Robots of the Year on April 14, 2018 during a National Robotics Week ceremony at SEAD Gallery in Bryan, TX.
CRASAR’s 2018 Aerial Rescue Robot of the Year is the DJI Mavic Pro. With a price tag of $1,000, experts found this drone was able to do everything they needed. Because batteries are expensive, the low price and the ability to recharge using a car charger are particularly important, as is the stabilizing camera with pan and tilt. The review team also noted DJI Mavic Pro’s first-person view, as well as programmed views provide the option of broad, NASA-style tiled mapping if needed. Most importantly, DJI Mavic proved itself a workhorse among responders and quickly became the tool of choice, flying 78 of the 112 flights during CRASAR’s Hurricane Harvey response and just weeks later, 247 of 247 flights when they responded to Hurricane Irma.
Another workhorse among the 2018 honorees is the 2018 Marine Rescue Robot of the Year, Hydronalix EMILY. Among the most used by lifeguards worldwide, this water-based robot boat has the ability to run rescue lines to people in distress and the capacity for five to eight people to hang on while being towed to safety. These features have made surfboard-sized EMILY a favorite among lifeguards in both routine and large-scale water rescues, including refugee rescues along the migrant route in the Aegean Sea.
A lesser known but very promising terrestrial robot captured the win as the 2018 Ground Rescue Robot of the Year. Carnegie Melon University’s aptly named Snakebot can propel itself into the smallest of spaces, allowing rescuers to search for signs of life where dogs and people cannot reach. The 2017 Mexico City earthquake marked the first use of Snakebot in the response phase of a disaster operation.
“These robots enable life-saving decision making for responders and emergency managers,” Murphy says. “Rescue decisions and critical infrastructure decisions during that response phase are made very rapidly based on the best available information at the time and these robots, well-deployed with the right teams of operators and experts, are getting key information to decision makers so they can save lives and efficiently manage risk.”
Spearheaded by internationally renowned Disaster Roboticist Dr. Robin Murphy, the Disaster Robotics Awards were created to raise awareness of the role and value of robots for disaster response and help prepare responders to put these resources to work in both routine response and disasters.
“The goal of the Disaster Robotics Awards is twofold,” Murphy explains. “One, we are letting people know what’s working and why. Two, we want to get people thinking about the next disaster.”
The ceremony also included awards and acknowledgements. Educator of the Year went to David Merrick at the FSU Center for Disaster Risk Policy, for his work in developing and offering classes in small UAS aimed specifically for the needs of emergency professionals. One of the courses Is now available online at https://tinyurl.com/suas-course. Agency Partner of the Year went to Fort Bend County Office of Emergency Management for partnership on the Hurricane Harvey preparations, response, and recovery activities. The Service Award was awarded to Justin Adams who coordinated CRASAR operations as well as served as the Air Operations Chief for manned and unmanned operations for Fort Bend County during Harvey, then was CRASAR operations for Hurricane Irma, and deployed a third time to Hurricane Maria for three weeks.
Hurricane season begins June 1 and experts are anticipating a strong season of storms again in 2018. Murphy and her colleagues based the 2018 Disaster Robotics Awards on what robots worked well for responders over the last year. The CRASAR team documented robot use in response to hurricanes Harvey, Irma, and Maria, the Mexico City earthquake, and water rescues of Syrian boat refugees and elsewhere, gathering data regarding number, duration and nature of deployment. With these insights into what robots work best for responder, the 2018 honorees were selected.
While the awards recognize the contribution of robots in disaster response over the last year, CRASAR’s insights into the value of and best practices for robots in disaster response date back to the first use of robots in disaster response at the World Trade Center in the aftermath of the September 11th attacks. Murphy was on hand in New York and has been deploying robots in disaster response globally ever since. Through her work with both CRASAR and the Humanitarian Robotics and AI Laboratory in Texas A&M University’s Computer Science and Engineering Department, Murphy and colleagues continue to promote best practices and offer support to responders seeking the benefits of disaster robots.
I’ve been getting questions about the use of unmanned systems for the recent horrific Montecito mudslides, so let me extend my earlier tweet, both in terms of thoughts and prayers to the victims and also the technical details. It does not appear that small UAS are being used by the formal responders for Montecito.
- Unmanned systems- ground and aerial- have been used since 2005 and there are opportunities for marine systems as well. Unmanned systems have been used at least three times in the United States for landslides, two by CRASAR teams. UGVs were used for the 2005 La Conchita, California mudslide (which was the first ever use of an unmanned system for a mudslide), while UAS were used for the 2014 Oso, Washington, and Collbran, Colorado, mudslides. Both fixed and rotorcraft UAS were used at Oso and Collbran.At Oso, it would have nice to have had an autonomous marine vehicle with sonar to map the underwater portions. See this paper on the 2005 La Conchita and blog mudslide and 2014 Oso mudslide plus more lessons learned are in Disaster Robotics and in our older blogs. Plus see video Oso composite of small UAS flights
- The primary use for UAS has been for reconnaissance and mapping, not life safety. In the La Conceit mudslide, CRASAR worked with Ventura County, Los Angeles County, and Los Angeles City fire departments using shoe-boxed sized ground robots to try to get through basement voids and narrow windows in order to search collaterally damaged buildings for a missing family. Search dogs had given ambiguous indications but it was too dangerous to send in a human responder and the dogs couldn’t squeeze in. The robots weren’t used in the primary mudslide area because this type of mud had penetrated every house in its path leaving no pockets and would require a burrowing robot such as the sand lizard styles being researched by Dan Goldman and Howie Choset. In the Collbran (Deputy Sheriff Ben Miller at Mesa County used their systems) and Oso mudslides (The Field Innovation Team/CRASAR team worked under the direction of Snohomish County), the UAS were used for gathering essential elements of information about the disaster area and assessing the situation. This includes determining the boundaries of the disaster area; access points to the disaster area; social, economic and political impacts (e.g., at Oso there were concerns about additional property damage and impacts on salmon); geophysical information (e.g., topology and volume of debris); projecting impacts; and initial needs and damage assessment. The Montecito mudslides illustrate that another EEI, assessing the status of transportation systems— what roads and trails were still open to get responders in and civilians evacuated, is also important. The UAS at Oso and Collbran also allowed the incident commander to see where teams actually were and also ensure that unauthorized personnel weren’t entering the cordoned off area.
- A UAS platform that can perform only autonomous mapping is highly limited for the immediate response. Both Oso and Collbran had missions that required first person view/manual flight where experts could direct the UAS in real-time in order to better assess the situation. At Oso, the PrecisionHawk Lancaster was extremely valuable in rapidly mapping the disaster, but the responders also used the AR100B quadcopter to hover and stare over the water to estimate flow rate, locate personnel, determine progress on mitigation efforts, and generally get closer looks at objects and activities that they didn’t know that needed to look at until they saw it from the air.
- The biggest unanticipated UAS challenge has been flying mapping missions in complex terrain, not endurance or distance. While having a UAS with the endurance and capability to cover an entire mountain from a single launch point is important, the limitations (and LOS regulations) are well-understood by the UAS operators. The unexpected problem was the complexity of the terrain. The autonomous mapping software programs assumed a flat earth, suitable for agricultural mapping but the areas of interest were in the “toe” of a mountain and a plain. The elevation of the landscape varied. In addition, the landscape had forests with some trees higher than others. This presented challenges for autonomous mapping, especially as it was desired to fly the UAS as low as practical in order to get the high resolution imagery. The operators had to create “stair steps” missions for fixed-wings and vertical slices for quadcopters. In Oso, the CRASAR team used a quadcopter to fly to the tallest tree in a “patch” and get the actual height so that the fixed-wing could be programmed to stay well above that height. The advances in obstacle avoidance by DJI would have been very helpful.
- Protecting personal identifying information (PII) and privacy has been the biggest barrier in deployment. Data management and chain of custody has long been a requirement of unmanned systems use by formal response agencies. The Oso mudslides illustrated how touchy PII and privacy can be. At the Oso Mudslides, UAS operations were suddenly cancelled due to concerns over victim privacy; this was later resolved when the incident command staff was able to clarify that the UAS would be flying over the “toe” of the mudslide and river and not where victims were being recovered. Ironically, the FIT/CRASAR team had talked to all the local property owners to assure them that the UAVs would not be flying over their land and any recorded imagery would belong to the county and released only with their permission.
If you have documentation on other deployments of unmanned systems to landslides, please send me the info and I’ll send you a CRASAR patch!
Our hearts go out to the victims, their families, and the responders in Mexico. CRASAR has not been contacted about robots but this blog may be of use in thinking about how to use robots.
Ground, aerial, and marine robots have been used in several earthquakes. A good overview of ground robots for structural collapse is at https://www.youtube.com/watch?v=5Cm2bGlUjbQ. It’s an older documentary but all the issues and gear are still the same.
Small unmanned aerial vehicles are probably everyone’s first thought for earthquakes, in part to map out the extent of the damage. They can also be used to help responders determine the shortest, most debris-free route to locations or interest. For the reconstruction and recovery phases, UAS have been used to fly around and in large buildings that are suspected of being too dangerous for structural specialists to enter and assess the risk of further collapse- for example the cathedrals in Canterbury NZ, Mirandola ,Italy, and Amatrice, Italy. After the Tohoku earthquake, many experts pointed out that UAVs should be used to determine the state of hospitals- both whether the hospital is still functional but also if it is being overwhelmed by patients.
Ground robots also have a role. An earthquake may cause buildings to completely collapse where there are no voids that a responder can get into. The general strategy is to use a canine team to determine if there are survivors in the rubble (dogs can tell if the person is still alive). Typically a boroscope or a camera on a wand is inserted to try to see if they can locate the survivors and also get a sense of the best was to remove material to get to them. The boroscope or a camara on a wand can only go about 18 feet into the rubble, depending on how twisty the void is. In a major building collapse, survivors may be much further, which is why small, shoe-box sized robots such as an Inuktun VGTV may be used. The “cameras on tracks” robots can pull themselves into the rubble and also change shape to help get into tight spots. These small robots will usually be tethered, with the tether acting as a belay line and the tether preventing loss of signal.
Some interesting robots are the snake robots being developed by Howie Choset at CMU and the Active Scope Camera caterpillar robot being developed by Olympus in conjunction with Japanese researchers led by Satoshi Tadokoro. The ASC was used at the Jacksonville building collapse in 2007.
Bigger ground robots, such as those used at Fukushima, can be used in bigger buildings but gepnerally can’t see the ceilings, which is usually very important and happened the New Zealand earthquake. They can’t go into small voids and may be too heavy- they could cause delicately balanced rubble to further collapse and kill a survivor underneath.
Marine robots, especially ROVs, are important as was seen in the Haiti earthquake and Tohoku earthquake and tsunami. The earthquake may have changed the shipping channels, damaged bridges and ports, and put debris in unexpected places. Thus shipping is stopped until the shipping channels are cleared- and as was seen at Haiti it’s hard to feed a country with one airport. Shipping is extremely important for getting relief supplies in.
There are more details in the case studies in Disaster Robotics and the Springer Handbook of Robotics on what robots have been used and particular strategies. All of us are happy to answer questions. We wish everyone the best on this terrible event.
A major update from our earlier post CRASAR was at Hurricane Irma supporting Roboticists Without Borders’ member Florida State University Center for Disaster Risk Policy deployment to Collier County, Florida. The CDRP effort was led by David Merrick, CDRP director, with Justin Adams, Kovar and Associates, who led the CRASAR Harvey response serving as his deputy. Collier County is in southern Florida near Naples and Marco Island. This is familiar territory to CRASAR, who responded to Hurricane Wilma in 2005 at Marco Island, sending out the first known use of unmanned marine vehicles. The county was exposed to severe wind damage and flooding from Irma which passed over as a Category 3 hurricane on September 10. The teams demobilized on September 16, with last flights on September 15. Six UAS pilots flew 247 imaging flights covering over 491 critical infrastructure targets, as well as provided overwatch for FL Task Force 8 and made multiple maps. The flights started on September 11 and surpassed the record of 119 mission flights for public officials during a disaster set at Hurricane Harvey. FSU and Texas A&M plan to offer a joint day-long short course on small UAS for emergency management on October 21, expanding the course that they have taught at the Florida Governors Hurricane Conference in May 2017 and for Los Angeles County Fire Rescue last week. Contact Robin Murphy for more information about the Irma deployment and the upcoming course.
The FSU CDRP team under the direction of David Merrick was a major component of CRASAR’s record setting small UAS Hurricane Harvey response, leaving Texas with two days to prepare and predeploy for Irma. The use of small UAS at Irma initially followed a similar pattern to Hurricane Harvey, with UAS being used to assist with rapidly conducting search and rescue operations and determining the best route for US&R teams to reach people at risk but now has shifted to determining the state of over 1,500 critical infrastructure targets in the county. These targets, which include bridges and waste water treating stations, impact the restoration of services and the economic recovery of the regions. The use of small UAS has significantly sped up the process, provide more complete assessment of all sides of a target, and multiple targets can be examined on one flight.
The FSU CDRP-led teams consist of pilots, data managers, platforms, and the RESPOND-R mobile lab. The teams are from FSU, Kovar and Associates, and CRASAR. The teams have 7 pilots with a fleet of 20 platforms including the DJI Mavic, DJI M600 Pro, Inspire, Intel Falcon 8, Disco, Phantom 3, and PrecisionHawk Lancaster rev 5. The teams are using the lessons learned at Harvey to improve rapid sortie planning, team situation awareness, and streamlining data management. Data is being collected by CRASAR and FSU from both deployments to permit developers to build better UAS optimized for the wind and operations tempo in a disaster and user interfaces that facilitate the data-to-decision process enabling county emergency management experts to rapidly get the right information and make good decisions. The data will also support the creation of new visualization tools for responders, enabling them to sort through terabytes of aerial imagery, and serve as a foundation for machine learning and computer vision algorithms to process tetrabytes of data.
The Irma response differs from the Harvey response in at least four key ways.
- the hurricane posed primarily wind damage with flooding as a secondary impact, whereas Harvey in Fort Bend County was primarily flooding from rainfall and then the risk of additional river flooding. This changed the style of search and rescue operations, especially as cellular coverage was affected and residents could not always call for help. The UAS teams assisted Florida Task Force 8 as the aerial view helped the searchers determine where to go, which flooded house to inspect next, and to better coordinate operations. The wind damage in Irma made route clearing operations more important as downed trees could unpredictably be blocking roads as compared to flooding with tended to inundate specific areas and predicted from flood maps.
- the majority of missions are for critical infrastructure property damage assessment. These assessments were normally being done in person. This is time consuming for driving to the site (including determining alternative routes) and then requires a person’s time to survey the target. In many cases, the inspector cannot see or get to all sides of a facility. While both manual and UAS inspection require the same amount of time to get to a target, the use of UAS is being shown to be faster than walking around and more complete as the UAS can fly around the target and also provide an birds’ eye view as well. For example, on Sept. 13, two 2-person UAS teams were able to document 97 infrastructure targets with 12 flights. Our initial look at our logs indicate an average of 16 minutes on-site for a mission– that’s stop the car, turn on the UAS, fly the mission, get back in the car. And a flight covers an average of 2 targets per flight. It’s hard to believe that a person can walk around a tank farm faster.
- A single flight (or sortie) at Irma typically covers multiple targets (an average of 2 targets), while at Harvey a single flight covered only one direct mission objective (though the aerial imagery was used to inform additional multiple emergency support functions).
- Operationally, the resident population and airspace traffic is less dense, with less manned assets flying in the region and the operations did not require a Temporary Flight Restriction for safety. The Irma teams have less challenges in planning flights to avoid flying over people and did not encounter self-deployed teams or hobbyists interfering with UAS flights for officials; these challenges were seen at Hurricane Harvey.
- You can see this through AirMap
- A Temporary Flight Restriction is the FAA equivalent of the Police blocking off a road. Instead of police cars and tape, the FAA does it by posting the TFR to aviation websites. It’s the law. If you think the police shouldn’t have blocked off an area, you can’t say “I’m doing a Thelma and Louise on the state highway patrol blockade because I’m a humanitarian” and not expect to get arrested or that you are interfering with how the authorities in charge are trying to handle the situation.
- Remember, if you didn’t have a pilot’s license or could fly BVLOS before, you can’t get permission to fly in the off-limits area. Going back to the car analogy, if you didn’t have a commercial driver’s license to drive big trucks before the police waived you through the blockage, you can’t drive big trucks now.
- You need a government agency sponsor for the E-COA
- However, some smaller jurisdictions may not realize that having you fly a small UAS is the same as having a manned helicopter fly and that they need to go through the official air operations branch of the response to coordinate manned flights. So if they don’t follow official incident management command protocols, especially in Florida where there is a state policy, there are problems from that jurisdiction and their overall response efforts could become ineligible for reimbursement (which are dependent on following protocols).
- have a plan for what to do. Sometimes you may have to “park it in the trees” because you can’t return to home fast enough or know where the hell is going. Sometimes you may have to go high, like when a crop duster descends because you they are going lower.
- a VHF radio to monitor their traffic and to post that you are flying can be of great value; it’s nice to have an experienced civilian pilot on the team to help with this
- Harris RangeVue or other ATC software package can be very valuable as well, as someone can monitor this and then radio out to the teams as to approaching aircraft and if it is going to enter the area where the UAS is operating and thus needs to get out of the way. Big shout out to Lone Star UAS Center, as a CRASAR member, we were able to use this
- Tags such as PrecisionHawk’s LATAS were very, very helpful for us at Harvey
- Don’t fly BVLOS, because you just can’t see what’s going on with a low flying manned aircraft (yes, they may be in violation of airspace regulations but there are people’s lives at risk- the pilot and people being evacuated or rushed for medical aid)
- It often in very crowded settings with lots of trees.
- Personal safety is more at risk. Think snakes. And floating balls of angry red ants. And swarming insects. And in Florida- alligators. Not only can it be dangerous, but by definition, no one can come rescue you.
- Back to UAS safety: Be prepared for “walking off” or “fly aways” high tension power lines- which can interfere with electronics, especially compasses. Combine that with heavy cloud cover reducing GPS coverage and you sometimes get unpredictable behavior from even the most reliable UAS.
- if you find people in immediate distress or an immediately threatening situation (we found an unreported fire), call 911 and the agency you are working for (they will also try to get the info to the right group) if the cell phones are working
- please don’t post to social media
- emergencies managers won’t necessarily be looking at social media
- they probably won’t have good internet access with electricity and cell towers being down due to wind damage
- the agency may not be able to use the data because there is no chain of custody or way to verify that it is accurate; people have started rumors and posted photoshopped video in the past so there’s a tendency to ignore contributions
- you may be liable for violations of personal identifiable information
- provide a standard filename convention where members of the different response functions can easily find the data they need. Usually location and data
- GPS coordinates are hard for all groups to parse, so street location (put a pin in Google Maps of where you, get an address, and then take a screenshot on your phone) or neighborhood if you know it
- Provide a master kmz of all the locations you flew and annotations of anything particularly interesting
- Think of this as documenting a crime scene and what would be needed to admit this as evidence; that helps with documenting
- Still images at nadir are extremely hard for normal humans to comprehend. For general assessment, video FPV may be the most effective.
- Microsoft ICE is much faster than packages such as PrecisionHawk’s Datamapper which we used, which produce georeferenced pixels. In the initial week of the disaster, the responders didn’t need that level of detail anyway- it is “general outline of how bad is bad” At Harvey we didn’t begin photogrammetric flights until after a week.
- Remember that agencies may not have the ability to store terabytes of images, videos, and orthomosiacs. As we found in 2015 and 2016, no one can email a high resolution ortho of an area. But they can email a “framing shot” .jpg.