Archive for the ‘Rescue Robots’ Category

Flooding and small UAVs

At this time, we aren’t involved in the horrible Texas and Oklahoma flooding. But we’ve been studying flooding since our deployment with Hurricane Charley in 2004. This blog will give a short background on what we do here at A&M and the upcoming 2015 Summer Institute on Flooding, then the history of small UAVs for flooding, and the potential uses generated by experts at our 2014 Summer Institute on Flooding.

About CRASAR and flooding

The Center for Emergency Informatics (CEI) here at Texas A&M is in the middle of a two-year exploration of information technologies, including unmanned systems, for floods. The CEI is a “center of center” that brings together practitioners, academics, and industry to fuse and apply their knowledge to disasters.  CRASAR is the “response” arm for CEI (actually we do participatory research versus response- similar to what anthropologists like Margaret Mead do. Sometimes the only way to learn is by doing by embedding with the responders, we don’t do disaster tourism).

The CEI is hosting a Summer Institute on Flooding June 16-18, the second on flooding. The choice of flooding was motivated by the fact that flooding is costing the U.S. over 80 lives and $8 billion in damages each year, excluding storm surge from hurricanes. Last year’s summer institute brought together 42 representatives and 12 states from 14 agencies, 14 universities, and 8 companies to consider what information technology is mature enough, or needs a bit of encouragement, to assist with flooding response.

This year’s Summer Institute consists of three exercises, each representing the key problem missions identified by the agencies:

  1. Swift water rescue (June 16)- helping responders rescue people suddenly cut off by rising waters. The plan for the swift water rescue reads like the campground flooding- showing the advantage of having the responders from Texas Task Force 1 and TEEX design exercises around real missions rather than around technologies.
  2. Life-saving response and immediate mitigation (June 17)- where’s the flood, who is at risk?
  3. Restoration and recovery (June 18)- restoring roads, electricity, sewage, etc. as well as insurance companies conducting property damage assessment  and cities generating debris estimation so that they can keep the rats out.

About small UAVs and flooding

Small UAVs have been used at least 8 disasters from flooding or had flooding associated with it: Hurricane Katrina 2005, Typhoon Morakot, Taiwan 2009, Thailand Floods 2011, Typhoon Haiyan Philippines 2013, Boulder Colorado floods 2013, Oso Washington Mudslides 2014, Balkans flooding Serbia 2014, and Cyclone Pamela Vanuatu 2014.

Historically they have been used for:

  • General reconnaissance: where’s the flooding, where are the people cut off by the flooding, and what roads are still passable.
  • Hydrological situation awareness, both real-time and post-processed. The flooding caused by the Oso mudslide was a real problem and rotorcraft were considered because they could hover and stare at the river, letting the hydrologists estimate the flow rate in different areas. The biggest push was for surveying the amount of flooding and- with the addition of photogrammetric image software- the terrain and potential for additional flooding (or the best place to put a dyke, channel or other mitigation). Our partners at FIT took it further and printed out a 3D model of the terrain to help everyone visualize the terrain.

Potential uses for UAVs

  • Overwatch for swift water rescue teams: our friends at South Carolina Task Force 1 have been pushing us to help create the protocols for using small UAVs to help them see that a logjam is coming down river and going to wipe them and the people they are trying to rescue out. This is one of the scenarios we will be playing in our Summer Institute on Flooding in June
  • Debris estimation: both the debris directly from the flood and the indirect debris a few days or weeks later from people having to rip out sheet rock and carpets. The advances in photogrammetrics make it possible to estimate the volume of debris— if you have the “before” survey of the area; we flew with PrecisionHawk at the Bennett Landfill superfund site in February in order to estimate the volume of toxic trash (which was on fire) that needed to be safely removed. The next step is to estimate the content, because vegetation and construction materials have to get handled and processed differently.
  • Deliver supplies to cut off regions. We learned during last year’s summer institute that if the locals can hold out for 72 hours, usually that’s sufficient. Indeed, in Texas, where breeding stock can represent hundreds of thousands of dollars of investment, it may make sense for someone to stay behind. But sometimes people get taken by surprise or a bridge washes out unexpectedly and need diabetes medicine and other perishables. Groups like Matternet, who like CRASAR are members of FIT,  are looking at delivering medicine with rotorcraft. A group of Texas A&M aerospace students won 2nd place for their small but hefty fixed-wing UAV that could be used to drop off heavier/bigger bundles of supplies from further distances
  • Delivering lifelines, life jackets, and small things to people trapped on roofs. A note about delivering things with a rotorcraft- You can use rotorcraft to carry a line or bottle to someone but the weight and distribution usually makes the UAV very sensitive to wind and control errors— if you are using a open rotor system just be aware and maintain more distance than normal from a person if at all possible. One senior design project that resulted from last year’s Summer Institute was to create a 2-way audio system for rotorcraft. The rotors generate about 85 dB of noise so if you’re a responders and want to try to talk with a person by hanging a microphone and speaker on the UAV, it probably won’t work. The Computer Engineering design team used noise reduction algorithms from National Instruments LabVIEW to prototype a lightweight 2-way audio system impervious to noise.

 

 

 

 

Nepal: recommendations for small UAVs

As the tragedy in Nepal unfolds, the immediate rescue response has ended and now efforts are shifting to agencies working on the mitigation of the event and dealing with continuing cascade of consequences and hopefully to recovery as well as humanitarian relief.  We have not been asked to participate and cannot self deploy but to those planning to fly small UAVs, I recommend that you look over the range of uses of small UAVs in the past 8 earthquakes in the past blog (and in more detail in Disaster Robotics). Plus:

  • Be aware that the altitude may change the performance of your platform
  • Working in complex terrains such as mountains will impact any preplanned paths. We have found that imagery reconstructions from fixed-wings will do better with a series of flights “stair stepping” along a hill or mountain than trying to cover the entire area at one altitude. Also a flight at one altitude may violate any flight AGL restrictions because to be high enough to fly at the top of the mountain, you’ve almost certainly exceeded the AGL limits for the lowest part of the terrain. We have found that rotorcraft flight plans work better as a set of vertical planes.
  • If you are planning to conduct structural inspection missions, you will most likely need to fly with 3-10m of the structure. Be aware that this creates wind effects and can interfere with GPS and wireless connectivity. Also, our research with civil engineers indicates that no matter how much video or photos we try to take, having a specialist who knows exactly what to look for is critical.
  • Expect engineers and structural specialists to use the raw images. Our studies at Disaster City indicate that orthomosaics do not accurately show straight edges on buildings and have a slight bit of ghosting, regardless if from fixed or rotorcraft.
  • Be aware that the country may have a temporary flight restriction in order to protect manned helicopters working at low altitudes in the area- that applies to anything that flies, there are no hobbyist exemptions. The normal procedure is for ANY aircraft, manned or unmanned, to coordinate with the air traffic control so that the manned helicopters can continue to operate. Regardless, manned systems cannot see small UAVs and thus cannot avoid. Should they see a small UAV operating and are not briefed, they typically have to return to base because of the possibility of a collision. Sending someone to the Air Branch of the incident command can go a long way to making sure ad hoc flights don’t accidentally interrupt other activities.

 

Nepal: disaster robots and earthquakes- history and uses

Our thoughts and prayers to the victims, families, and responders in Nepal where CNN is reporting over 777 people killed. Here is some information about how disaster robots have and can be used.

Uses: the primary use of disaster robots in 8 previous earthquakes have been to give authorities and experts rapid understanding of the damage and general situation, the state of the infrastructure – especially underwater portions of bridges and ports which is key for transportation of responders and supplies, and the state of building collapses- especially where there is the indication of survivors or where the building must be inspected by experts but it appears to be too unsafe to inspect.

History: Small unmanned systems have been reported for use in response and mitigation of 8 earthquakes. The first reported use in 2004- an experimental ground robot from IRS exploring a house in the Niigati Chuestsu (Japan) earthquake with the Japanese equivalent of FEMA. In 2009 a small UAV was deployed in Italy by La Sapienza with the Italian Fire Department for the L’aquila Earthquake. At the Haiti Earthquake in 2010, the Navy MSDU used underwater ROVs to clear the port so that ships could bring in responders and supplies without running aground or collapsing the piers. The Haitian airspace was under a temporary flight restriction but there was a  small UAV that self-deployed and performed reconnaissance. A small UAV was tried for indoor inspection of the cathedral at the Christchurch earthquake (2011) but the structural specialists shifted to ground robots.  Underwater robots were used extensively by municipalities with some use of the ground robots and small UAVs for structural inspection at the Tohoku earthquake/tsunami in 2011.  Small ground and UAVs were used by the NifTI team with the Italian Fire Department at the Finale Emilia earthquake for structural inspection in 2012. The Chinese military used small UAVs at the 2013 Lushan China earthquake and the 2014 Yunnan China earthquake for rapid reconnaissance of hard to reach areas.

 

 

Vanuatu: How disaster robots have helped in 12 similar events and might help there

I’m here at the UN World Disaster Conference where word of the destruction in Vanuata is coming in and our thoughts and prayers go to out the victims and families. It sounds like the effort is on humanitarian relief.  I’m not seeing any discussion of mitigation/response/recovery of critical infrastructure, which is the historical focus of disaster robotics. Here’s some information on how robots have helped in 12 similar disasters.

Small unmanned aerial systems have been used by rescue authorities in 8 storm or flooding events: Hurricane Katrina, Hurricane Wilma, Typhoon Morakot (Taiwan),  Thailand floods, Typhoon Haiyan (Philippines), Boulder Colorado floods, SR530 mudslide and flood, Serbia floods. UAS are most often used for rapid reconnaissance and mapping of the extent of devastation, condition of transportation routes and what areas are cut off, power lines, and general hydrological and geological mitigation needed to predict, contain, or drain water, etc.

Small marine vehicles have been used by rescue authorities in 3 storm and flooding events: Hurricane Wilma, Hurricane Ike, Tohoku tsunami (Japan).  They are mostly used to identify the state of bridges and ports, debris that is blocking ports or polluting fishing areas, and for the recovery of victims that were washed away into the sea. Plus they were used at the Haiti earthquake to clear underwater debris from the port to allow humanitarian relief supplies to be shipped in (it’s hard to feed a country via planes into a single airport). In Japan, the use of marine vehicles by the IRS-CRASAR team was credited for re-opening the Minamisaniruku new port 6 months earlier than would have been possible with manual divers and in time for the key salmon fishing season.

Unmanned ground robots are almost never used because commercial buildings rarely collapse in these conditions, it is mostly individual homes.

Tohoku earthquake anniversity and UN World Conference on Disaster Risk Reduction

wcdrr-logo-desktop-v3.0It is the 4th anniversary of the Tohoku earthquake and I am en route to Sendai where the United Nations World Conference on Disaster Risk Reduction. It is sobering to think that over 18,000 people died and that the great nation of Japan is still recovering.  It makes it even more appropriate for Japan to host the conference and at Sendai. I look forward to seeing my colleagues from the International Rescue Systems Institute who we worked side-by-side with unmanned marine vehicles.

I will be giving an invited talk on the current state and achievement of  disaster robotics during the Public Forum. In looking back between 2011 and now, the biggest surprise is that unmanned marine vehicles are not being used as much as I would have thought. The tsunami response and recovery showed the efficacy of these tools and how the can do in 4 hours what it take divers weeks to do (if they are available). There’s no surprise in that unmanned systems are being used more frequently! I’ll post the my findings when I give my talk– no spoilers!

Pleurobot is salamander-like robot with lifelike motion

A video showing “multimodal locomotion in a bioinspired robot” has been making the rounds, and the video demonstrates advances in robotics as scientific tools as well as potential robots for search and rescue operations. Its name is Pleurobot.

According to the video notes, the Pleurobot is being developed by the BioRob at EPFL and NCCR Robotics. The robot takes it cues from the salamander and the team is making use of cineradiography to advance their work. They recorded three-dimensional X-ray videos of salamanders, walking on ground, walking underwater and swimming. Tracking up to 64 points on the animal’s skeleton, they were able to record three-dimensional movements of bones in great detail. They deduced the number and position of active and passive joints needed for the robot to reproduce the movements with reasonable accuracy in three-dimensions.

Commenting on their work, Evan Ackerman in IEEE Spectrum said, “The key to Pleurobot’s lifelike motion is its design, which was based on 3D x-ray movies of a real salamander walking and swimming.”

Their main goal is understanding the way that the nervous system coordinates movement in vertebrates. “Pleurobot’s design, with 27 degrees of freedom, allows us to test more advanced mathematical models of the locomotor nervous system towards richer motor skills,” they said. The team said Pleurobot may also prove useful in other ways. “Because of its low center of mass and segmented legs it can navigate over rough terrain without losing balance. With a waterproof skin it can also swim. Those features may one day enable Pleurobot to help in search and rescue operations.”

Check out more information at phys.org

Thoughts on what the proposed FAA SUAS rules mean for search and rescue

by Brittan Duncan with Robin Murphy

Here are our notes on the recommended rules, which are currently open for public comment and are not in force, and how they might impact public agencies, especially fire rescue.  Current operations should still be run under the existing COA process for public agencies. Also, even though the rules are open for public comments for 60 days, that doesn’t mean the rules will be made official in 60 days or that these will indeed be the final rules.

In the future, COAs may not be necessary for flight, as the rules would let agencies pick and choose whether to fly as a civil operation or as a public entity.

If you are flying in Class G and under 500 feet AGL with a registered vehicle less than 55 pounds and a top speed of less than 100mph with a pilot who has passed the proposed UAS ground school (“aeronautical knowledge test”), you won’t need a COA. You will still have to comply with NOTAMs, TFRs, see-and-avoid rules, and rules for flying over people. Note: you will also need to consider any local regulations on privacy, etc. Remember, just because the FAA says you can fly without a COA in an area, it doesn’t mean your constituency finds it acceptable.

If you are flying in other classes of airspace, you have work with the local ATC. Exactly how this happens is unclear and we suspect that in practice many ATCs will say “get a COA” and in some of those cases, you may still have to have to pilot with a civilian pilot’s license. Hopefully not, but that’s still unclear. It would be the worst case scenario, though, which is manageable.  This gives more flexibility for emergency situations where you need to fly in a tower controlled area but hadn’t made prior arrangements with the ATC- they can say “ok, you can fly this one time but let’s work out a better plan after the emergency is over.” Of course, the ATC can also say “no.”

You can fly over people who are covered, such as those in their houses in a neighborhood or in their cars in traffic, but you are responsible for mitigation, as in: your agency is liable.  You would be allowed to fly over uncovered people only if they  are directly participating in the operation (like other fire fighters and police) and have received a briefing. However, as a public agency, that may not be realistic except during training or highly localized flights during an incident where the site was cordoned off. Fortunately, it looks like agencies will have some discretion in how they alert civilians. Informing residents that you are flying in their area to search for a person, asking people to stay indoors temporarily, posting signage, etc., may be sufficient. Your agency probably already has SOP for various activities and those can be expanded to handle SUAS.  Note that even if your agency feels it is acting within its bounds, SUAS make some people very antsy and thus anything you can do to proactively reduce misperceptions will probably benefit your agency in the long run.

You still have to maintain line of sight.  Still no looking through binoculars or cameras. This applies to both the operator and the visual observer, who must both be able to see the vehicle at any time and should be able to tell which direction it is facing, as well as direction of flight.

You can have a mobile operation with a visual observer on a moving boat, but not a moving car.   This is great news for agencies who want to map rivers or wetlands, “meh” for everyone else.

You can have multiple UAS in the same airspace, but each requires their own pilot and visual observer. We are looking forward to this as we see a need in wilderness search and rescue for a fixed-wing to conduct a thorough survey at a higher altitude while a rotor-craft is directed to the high probability spots for a missing person.

You can eliminate the separate visual observer if the pilot flies “heads up” and always keeps eyes on the UAS. Note: This is only allowed for flights in which you are not using any first person view- you can’t “fly the camera” as the pilot without a visual observer. Our studies with SUAS and decades of studies in manned aviation suggest a single person switching between first person view and external views can lead to errors, so although it adds manpower (sigh) we agree with this.

When you want to use first person view, in conjunction with a visual observer, the visual observer is not allowed to manipulate any flight controls, look through the camera to pan/tilt/zoom in on an object, or initiate any autonomy (e.g., select waypoints). Your mission specialist (for us, there’s usually a responder saying “no, this is what I want to look at”) can’t be your visual observer. This is important to note in systems that require multiple interfaces, such as a hand controller and a computer.  Another option is to fund our research in multi-modal user interfaces that don’t overwhelm the pilot and don’t require them to look at a screen!

You have to have 5 minutes of reserve power to insure a return to home and controlled landing. Note: in practice, you also want to keep track of the distance to home/time to home. We think the real point is always being able to return home unless there is a catastrophic failure of the SUAS.  5 minutes may be excessive for small areas where the SUAS can return safely in less than 2 minutes, but no one has ever been sad to have too much fuel.

You can’t fly at night. It looks like you will have to go through the COA process for this.  The SUAS rules document actually says “The FAA welcomes public comments with suggestions on how to effectively mitigate the risk of operations of small unmanned aircraft during low-light or nighttime operations.”  Which sounds like “if you’ve got any ideas, let us know, because we didn’t come up with anything that could be a hard and fast rule.”

Robot firefighter puts out its first blaze

In Mobile, Alabama, a humanoid robot looked on as a fire burned aboard the USS Shadwell. Its infrared eyes scanned the blaze to find its heart, and its robot arms grabbed a hose to spray water into the inferno.

This was the first live test for SAFFiR – the Shipboard Autonomous Firefighting Robot – and the first time a robot has ever fought a fire. Developed by roboticists at Virginia Polytechnic Institute and State University (Virginia Tech) in Blackburg for the US Navy, SAFFiR is intended to be part of the firefighting equipment of the future on board every Navy ship, tackling fires without risking human life.

The robot, which weighs 63 kilograms and stands 178 centimetres tall, uses dual cameras to help it see and move around, laser sensors to provide the exact distances between objects and thermal cameras to help it find fires. Although SAFFiR can operate autonomously and is able to walk and grab a hose on its own, the current version takes all other instructions from a human operator.

Check out more information at newscientist.com

Mexico City hospital collapse…

A building collapse is almost always terrible, a building collapse of a maternity ward is unthinkable. All of us send our thoughts and prayers to the families of this horrible event.

Although there is no news about any robots being used, robots were first used in disasters for commercial multi-story building collapses– notably the 9/11 World Trade Center. Commercial multi-story buildings present unique challenges for searching because the concrete floors can be densely pancaked in some areas with just inches of space and leave survivable voids in others.

Small robots like the shoe-boxed sized micro VGTV and micro Tracks by Inkutun were used the most at the WTC because they could go into the rubble where a person or dog could not fit and could go further than a camera on a wand. That is still the case, with small robots being used to go between tightly packed layers of rubble at the Berkman Plaza II collapse in Jacksonville (2007) and the Prospect Towers collapse in New Jersey (2010).

Bigger robots such as the IED robots like iRobot Packbot and QinetiQ Talon are often too big for the size of voids in the rubble of a pancake collapse. Really large, “maxi” robots such as the REMOTEC series are not only too big, but the weight poses a problem- as in they are so heavy they could cause a secondary collapse.

If anyone knows of other multi-story building collapses where robots were used, please let me know and a reference and I’ll send a CRASAR patch.

In the meantime, our thoughts and prayers to the families in Mexico…

Emergency Management Magazine…

There’s nothing like appearing on the home page of Emergency Management Magazine to trigger a “holy cow, I haven’t been keeping up the blog!” It’s been hugely busy here between working with students colleagues, and industry partners on

  • creating use cases for robots for Ebola and other infectious diseases with a grant from the National Science Foundation (Eric Rasmussen, MD FACP,  and our medical director for Roboticists Without Borders is the co-PI),
  • prepping UAVs for an upcoming wilderness search and rescue exercise with Brazos Valley Search and Rescue (big shout out to the FAA and CSA for their help!),
  • prepping for the Robot Petting Zoo we are doing with the Field Innovation Team at SXSW to show off real robots used in real disasters,
  • getting to work with Prof. Howie Choset at CMU and Prof. Dan Goldman at Georgia Tech on burrowing robots through a National Robotics Initiative grant from NSF, and
  • teaching an class overload (add case studies of robots at disasters to undergrad robotics as part of my Faculty Fellow for Innovation in teaching award, plus the AggiE Challenge advised by Profs. Dylan Shell, Craig Marianno, and myself on creating ground and water robots to detect radiation )

So things are happening!  Thank you for your donations that make it possible to bring robots to new venues such as wilderness search and rescue and public education events like the Robot Petting Zoo. Most of what we do is based on donations, so please donate here!