All posts by Dr. Robin Murphy

About the author:

avatar I am a founder of the field of rescue robotics; my personal goal is to enable emergency responders to routinely save lives through physically situated artificial intelligence (robots, sensor networks, etc.)

Tropical Storm Cindy: Best Practices and Regulations for Agencies Using Small UAS

As Tropical Storm Cindy brings rain and flooding to the Gulf Coast, it’s a good time to share the two “one-pagers” that the Roboticists Without Borders small UAS team members have put together.  We have several flyers on stand-by with small UAS ready to deploy upon request.

Here are two one-pagers that may be of use to responders:

http://tinyurl.com/crasar-1page-SUAS-regs  is a one-page guide to who can fly for an agency and where they can fly (and how they can easily tell using airmap.io where they are allowed to fly). This is aimed for emergency managers who haven’t worked with SUAS before and have heard the FAA regulations are daunting- but the rules are much simpler and permissions can be gotten within 1-2 hours if needed.

http://tinyurl.com/crasar-1page-SUAS-floods is a one-page compendium of the missions that SUAS can be used for before, during, and after the flooding. It also discusses imagery post-processing and the other important considerations (coverage, manpower, software, data management). This is based on our experiences with flooding and storm surges since 2005 and especially reinforced by our recent deployments with Fort Bend County, Texas, and with Tangipahoa and Washington Parishes, Louisiana, last year. A preprint of our 2016 paper detailing the case studies of the Fort Bend County floods is here (IEEE SSRR 2016) and the official paper is here.

We hope for the best for everyone in the path of the storm.

Murphy one of four breakthrough women

in IEEE Institute magazine.

My summary of the AI for Good Summit overall

File Jun 10, 3 37 33 PMWhat a treat to be a speaker and participant in the  X Prize ITU AI for Good Summit, helping advocate for intelligent unmanned systems and emergency informatics. I had feared that the summit would merely recycle the usual memes about AI but  it nimbly avoided that pitfall by bringing together a diverse set of experts in AI and ethics from academia and industry with dozens of UN agencies who could articulate real needs and valid concerns. I found the talks to be thought-provoking and inspiring, especially ACM President Vicki Hanson reminding us that “the measure of success for AI applications is the value they create for human lives.” There were unexpected twists which I will try to summarize and share here.

 

The summit did indeed touch on the usual memes that are now standard in any discussion of AI for good.  I’ll go through the four most frequent and then get to the unexpected.

 

File Jun 10, 3 38 59 PMThere were the frequent obligatory pronouncements of the exponential versus linear nature of computing and all things digital (though Gary Marcus’ contrarian ELIZA- Siri linear line provided a delightful counterpoint and Marcus Shingles gave a business-oriented discussion of the linear-exponential trope, endowing it with actionable relevance).

 

There was a little bit of the “there is no commercial economic incentive for AI for good,” especially for disasters, refugees, or poverty. However Anja Kaspersen and other boots-on-the-ground agency reps went one step further and pushed back on academia and industry, making a subtle point that there’s no economic incentive to stay with an AI for Good app after it’s been piloted because it is really, really hard to generate sustainable innovation that solves a real problem. And, as noted by a few cynical participants, perhaps after being milked for marketing, there’s no continuing benefit.

 

I had expected more of the “AI is actually increasing inequality rather than helping” meme, particularly in terms of exacerbating the digital divide, reducing jobs (or good paying jobs), and increasing the disparity of wealth.  This was touched on but the meme was  so clearly accepted by all participants that the discussions focused on how to fix it.
The most frequently occurring meme was that there should be a democratization of AI, specifically in terms of access to data, transparency in who owns the data, and how that data is being used, and that the data should be used for the good of all.  This was a near universal topic for talks, discussion questions, and recommendations.

 

Now for the unexpected. For me there were some new terms I hadn’t heard before (such as “Centaur” to refer to human-machine collaboration), some I-should-have-thought-of-this-myself moments (for example, that given that even the poorest people generally now have a smartphone or share one for their village,  AI needs to be aimed for working with that level of platform),  and some be-afraid-be-very-afraid moments (particularly Stuart Russell’s cautionary comments that  misuse of AI is probably the biggest current concern but we could be undergoing a malware revolution that will render everything, including AI, useless).

 

But the most striking insights for me were three analogies that really hit home.

 

35031691452_f68b5ae92e_oPeter Marx, in the lively panel that we were together on, offered a stunning analogy about menus. As Gary Marcus and I complained that too many people had no idea of the breadth of the field of AI and its rich set of techniques, Peter noted that most AI developers were not AI experts but rather picking AI techniques off a menu.  I immediately visualized a person ordering “deep learning” because it had a favorable buzz on Yelp.  The analogy implicitly raises the questions of Which restaurants? Who sets the menu? And what about the food pyramid- what goes with the other choices? What’s a balanced meal or system? What about nutritional food labels- do we know what are the ingredients of this particular AI and that there may be too much sugar and salt? Who is conducting the food and health inspections? The menu analogy also touches on the fundamental question of whether practitioners need to have a formal comprehensive introduction to AI?

 

A more speculative analogy by Robert Kirkpatrick about the potential impact of AI compared AI to atomic energy. Atomic energy is “leaky,” potentially hazardous and thus has to be handled carefully, and can be used for peace or war.  The analogy caused me to immediately think of the nuclear arms race, government regulations, and so on.
The most sobering analogy was Peter Lee’s analogy equating the need to learn AI with the need to learn to read. Not in the Reading Rainbow sense of needing to learn to read, but in the Gutenberg printing press sense of the word. Given that “that there were perhaps 30,000 books in all of Europe before Gutenberg printed his Bible; less than 50 years later, there were as many as 10 to 12 million books,” (from http://www.hrc.utexas.edu/educator/modules/gutenberg/books/legacy/) printing clearly presented a revolution. So what did watching the revolution in printing mean to you as a parent? How were you going to make sure your children could read and take advantage of books? Even though you didn’t necessarily know what books would be written, by whom, or about what. And the implication that AI is similar, what are we doing to educate the next generation about AI? What do they need to know, regardless of what they expect their occupation to be? Especially since AI, like reading, isn’t really limited to a specific profession.  Beyond making sure our families are left behind, how can we ensure that everyone is AI literate? If there was any doubt about the enormity of a severe digital divide, the Gutenberg analogy erases that doubt.

 

Overall, three days well-spent and my hat is off to ITU for hosting it!

Free UAS Awareness and Best UAS Practices for Emergency Management Class at Governor’s Hurricane Conference

Roboticists Without Borders offered two sessions of a 3.5 hour class consisting of three modules: unmanned systems awareness, unmanned aerial systems awareness, and best UAS practices at the 2017 Governor’s Hurricane Conference in West Palm Beach, Florida. Over 35 emergency professionals representing over 28 local and state agencies attended and received certificates for participation. The class targeted chiefs and managers who are interested in what UAS (and robots in general) have been used for, what are the costs including the hidden costs of manpower, training and maintenance, what are the regulatory issues, and how to handle public perception. The class also went through the types of missions involved in each major type of disaster and the associated unique CONOPS and workflows for each mission. The class emphasizes data management and how to get, and share, actionable data in real-time.

The modules were created by Florida State University Emergency Management and Homeland Security Program and the Texas A&M Engineering Experiment Station Center for Robot-Assisted Search and Rescue. The material is based on formal training created by CRASAR originally funded by the State of Florida, plus lessons learned from over 30 deployments by RWB members, and studies by FSU and Texas A&M. The modules are normally incorporated in a longer class with hands-on demonstrations of unmanned systems.

The class is offered for free as part of the RWB mission to accelerate the adoption of unmanned systems by emergency professionals.

Columbia mudslide: recommendations for UAVs for search and for public works

Our thoughts and prayers are with the Columbians in the wake of the terrible flooding and mudslide.

CRASAR’s experiences with such situations suggest that it is a very difficult search and rescue (and victim recovery) problem. We’re assisted with the Oso Mudslides in partnership with the Fit Innovation Team (some of the video is the highlight reel) and also with the 2015 Texas Memorial Day floods with Lone Star UAS Center which swept over 40 people over a 5 mile area of remote river wilderness. Please note that while our UAV flights at the Memorial Day floods were victim search and recovery missions, the Oso Mudslides was for Public Works. When a disaster happens, while search and rescue teams are working, the engineering experts are also working to mitigate and prevent further catastrophes and to start on economic recovery. Both are important missions!

Here’s some videos that we prepared for the White House and Congress on the use of UAVs, artificial intelligence, and informatics technologies.

From our best practices guides, here are some recommendations for UAV operators:

Standard procedure is to take high resolution imagery and then have a group of trained experts examine each image.  Crowd sourcing can have two problems if not done correctly. One is that most people make major mistakes interpreting aerial images, particularly when the images may be from different altitudes or looking straight down. Hence “trained experts.” Formal methods exist for rating the accuracy of  people looking at the image (called coders). The other is unintentional violations of privacy– putting out images that may contain victims and saying “hey, everyone, come look at this”.

Since the images are geotagged, it doesn’t matter which images they look at.

Video generally isn’t helpful because of lower resolution and fuzziness when you try to pause.

Victims may be covered in mud and buried in debris so clumps large enough to contain a body may be put on the list for investigation by a ground team

 

 

 

SXSW!

I will be participating in the SXSW panel on Beyond BB-8: When Robots Start Acting Human! Another opportunity to talk about disaster robotics and show how robots and AI are assisting the emergency and disaster communities.

It’s been a busy few months at CRASAR. More companies and universities have joined Roboticists Without Borders and we participated in a four county wilderness search and rescue exercise last month with small unmanned aerial systems and a small unmanned marine vehicle with sonar for recovering a submerged body.  The sUAS work yielded valuable data on the use of thermal imaging for finding survivors (short version: not great if the victim is under a tree), and general workflow and concepts of operations.

Three of our Roboticists Without Borders members- Justin Adams, David Kovar, and David Merrick- also chaired sessions at the first National Unmanned Aerial Systems (UAS) Drone Conference and Summit.

I’ve given several talks, including the Assessing the Technological Turn on Humanitarian Action workshop for the S. Rajaratnam School of International Studies (RSIS) in Singapore program in Humanitarian Assistance and Disaster Relief (HADR) and I’ll be at the UN in June.

But nearest and dearest to my heart is that we continue to forge forward on the use of unmanned surface vehicles, small UAVs, computer vision,  and LTE wireless solutions to assist with preventing the marine mass casualty drownings of the refugees. The two EMILYs donated in our deployment last year are still in use by the Hellenic Coast Guard (who used it to rescue over 20 refugees stranded on the rocks in high seas) and Hellenic Red Cross and we look forward to taking more autonomous versions back this summer. We got great feedback from the Italian Coast Guard. But it’s a been a year and we’d like to directly help…

See you at SXSW!

New Zealand: what can robots do for a tsunami and quake?

Our thoughts and prayers go out to the Kiwis and especially to our colleagues at the New Zealand Fire Service who have been diligently adopting robotics.

So when a tsunami strikes, what can robots do? As was shown at the 3/11 Japan tsunami, unmanned marine vehicles can accelerate economic recovery by inspecting critical underwater infrastructure

YouTube Preview Image As was shown by our Japan-US deployments at the invitation of two municipalities at the 3/11 Japan tsunami, unmanned marine vehicles  (UMV) can assist with the response and accelerate economic recovery by inspecting critical underwater infrastructure- the underwater portions of bridges, ports, and shipping channels that are vital for access by responders and for getting supplies to any cut off populations. Later, the UMVs can help with environmental remediation, finding fishing boats and cars leaking gas and oil into pristine fishing waters and identifying other sources of pollution or dangers to fishing and navigation.

UAVs could be used to assess the overall boundaries of the incident, though most of the damage is near the ground. Like flooding, this is hard to get the angles to accurately assess damage. In places such as New Zealand, the agencies (and news media) generally have enough resources to get a general aerial assessment.

 

Dufek wins Best Field Paper Award at IEEE SSRR!

Left to right: Jan Dufek, Dr. Auke Ijspert, Dr. Kamilo Melo

Left to right: Jan Dufek, Dr. Auke Ijspert, Dr. Kamilo Melo

I am proud to announce that Jan Dufek’s paper on using a small tethered Fotokite UAV to control the EMILY unmanned marine surface vehicle to rescue drowning immigrants won the best field paper award at the IEEE International Symposium on Safety Security and Rescue Robotics in Lausanne, Switzerland, last week. Jan is one of my Ph.D. students.  The paper was the preliminary work over the spring semester that is now funded by the National Science Foundation RAPID program.

Jan received 200 Euros

Jan received 200 Euros and his paper will be published as a journal article in Frontiers, a European conference.

More details about the conference are at http://ssrrobotics.org, but there were over 100 attendees from 17 countries. IEEE SSRR is the only conference dedicated

More details about the conference are at http://ssrrobotics.org, but there were over 100 attendees from 17 countries. IEEE SSRR is the only conference dedicated to robots for  homeland security and humanitarian operations. It was established in 2002, with Dr. Howie Choset (CMU) and myself as founding co-chairs.

It was pretty dark so the photo is poor. Jan is on the left with conference chairs Dr. Auke Ijspeert and Dr. Kamilo Melo.

Emergency Managers Find Small Unmanned Aerial Systems Effective for Flooding and Popular With Residents

A paper to be presented next week at the IEEE International Symposium on Safety Security and Rescue Robotics in Lausanne, Switzerland, details the use of small unmanned aerial systems in two recent Texas floods in Fort Bend County, a major Houston suburb and 10th largest populated county in Texas. The 21 flights over four days provided flood mapping and projection of impacts, helping the county prepare and respond to the floods. Surprisingly, the flights did not encounter public resistance and the videos became a popular and useful asset for informing the county residents as to the state of the flooding. A pre-print is available here.

The small unmanned aerial systems were deployed through the Roboticists Without Borders program of the Texas A&M Engineering Experiment Station Center for Robot-Assisted Search and Rescue for two flood events in April and May 2016. Both events were presidential declared disasters.  Experts from DataWing Global, CartoFusion Technologies, USAA, and Texas A&M embedded with the Fort Bend County Office of Emergency Management and the Fort Bend County Drainage District to fly low-cost DJI Phantoms and Inspires. The flights provided flood assessment including flood mapping and projection of impact in order to plan for emergency services and verification of flood inundation models, providing justification for future publicly accountable decisions on land use, development, and roads.

The paper, titled Two Case Studies and Gaps Analysis of Flood Assessment for Emergency Management with Small Unmanned Aerial Systems by Murphy,  Dufek, Sarmiento, Wilde, Xiao, Braun, Mullen, Smith, Allred, Adams, Wright, and Gingrich, documents the successful use of the small unmanned aerial systems for the two. It discusses the best practices that emerged but also identifies gaps in informatics, manpower, human-robot interaction, and cost-benefit analysis.

The annual IEEE International Symposium on Safety Security and Rescue Robotics was established in 2002 by the IEEE Robotics and Automation Society. It is the only conference dedicated to the use of ground, aerial, and marine robots for public safety applications. It typically attracts 60-150 researchers, industrialists, and agency representatives from North America, Europe, and Asia. This year’s conference will be held at Lausanne, Switzerland, see http://ssrrobotics.org/ for more information about the conference.

The TEES Center for Robot-Assisted Search and Rescue is the leader in documenting, deploying, and facilitating technology transfer of unmanned systems for disasters. It has inserted robots or advised on the use of robots at over two dozen events in 5 countries, starting with the 9/11 World Trade Center and including Hurricane Katrina and the Fukushima Daiichi nuclear accident.

For more information contact:

 

Justin Adams, US Datawing and UAS lead for Roboticists Without Borders, justin.adamas@datawinglobal.com , 832.653.1057

Dr. Robin Murphy, director for the Center for Robot-Assisted Search and Rescue, robin.r.murphy@tamu.edu, 813.503.9881

For Hurricane Matthew: Quick Guide For Agencies Flying small Unmanned Aerial Systems (SUAS) for Emergencies

The illustrated version in pdf is here.

This quick guide is aimed at helping emergency managers quickly determine how they can exploit small unmanned aerial systems (like quadcopters).  The guide covers our best understanding of who can fly?  where can they fly?, and  any additional considerations in planning. Our best practices series has other documents on what kind of data you can expect to get, flight duration, etc., but this guide is about how the new regulations impact emergency managers. It is based on our SUAS deployments since 2005 and lessons learned from deployments by our colleagues.

 

WHO CAN FLY?

 

If members of your agency own a small UAS or have friends with a small UAS, they cannot fly at the disaster- even if they aren’t asking for money. The FAA has repeatedly ruled that a) disasters are a business or government activity and  b) if the UAV flight is a donation to a business or government, it is the same thing as if the business or government agency flew directly.

 

Therefore, the only people/companies who can fly are those with a:

  • Part 107 license. The license is new and many people/companies don’t have these yet.
  • 333 exemption. Essentially a business license versus of the COA. Many hobbyist declared themselves a company to get a 333.
  • COA. Essentially a government or academic license.

 

Your agency does not have to have the 107, 333, or COA– just formally invite the group to fly on your behalf. If the group has one of the above, there are three important caveats.

 

1. Controlled airspace. They can fly at a disaster in uncontrolled airspace, but will need special permissions for controlled airspace. Keep in mind, many densely populated areas will be in controlled airspace.

 

2.  They have to obey all the flight restrictions for their license, including Temporary Flight Restrictions. Getting permission to fly under a Temporary Flight Restriction does not give them permission to change up the rules, it only means that they are now coordinated with the rest of the air traffic who will expect them to obey the same rules as in normal flights.

 

3. 24 hour notifications before flights may be required.  If the group is flying under a 333 or COA, they have to post an online notice of intention to fly in a specific area, called a NOTAM, 24 hours in advance. So if you think you are going to have a group fly, have them declare as soon as you know. There is no downside to filing a NOTAM and then not flying.

 

 

WHERE CAN THEY FLY?

 

For planning purposes there are 3 types of airspaces: uncontrolled, controlled, and TFRUncontrolled means they can fly anywhere that is not controlled according to their license. TFR was covered above. That leaves the controlled airspace.

 

You can quickly determine if an area you want a group to fly in is in controlled airspace by going to:

 

https://app.airmap.io/

 

and enter the nearest town, then click the appropriate boxes.  What is “Controlled airspace” and what you have to do to get permission to fly in it will depend on whether the group has a) a Part 107 license or b) a 333 exemption or COA.

 

a. Determining Part 107 controlled airspace.  If the group has a 107, click on the menu on the left that says Controlled Airspace and “all”. You will get something like this:

 

 

 

 

Anything in shade means that it is controlled airspace. This means that they can fly only IF they have an airspace authorization that they have applied for in advance online and gotten approval. Note: the FAA system is backlogged by weeks, so for Matthew, this may not make possible to get approval fast enough.

 

b. Determining 333 or COA airspace.

 

Clear airmap and instead click on “blanket COA”. You should get something like this:

 

 

 

Any area in orange means that the airspace is off limits without additional permissions- no matter what altitude you are flying at.  The controlled airspace is due to airports. A local group may already have permission to fly in those areas, but may not. If not, permission to fly in controlled airspace on short notice is handled through an Emergency COA, also called ECOA, process. The process takes about 1 hour to get through the FAA- assuming you have the GPS coordinates of where you want to fly, the COA number, etc.

 

The key is that the tower has to approve the flights (actually the approve the process of letting them know where you’re flying, when you take off, land, etc.) and the FAA has to agree to the temporary extension of the current license.

 

  • Note about 333 exemption. ECOAs are granted only to businesses or agencies, not individuals doing business as. Too many quasi-hobbyists were trying to fly at disasters without working with a response agency.

 

 

ARE THEY ANY OTHER CONSIDERATIONS?

 

There are three considerations:

 

  • Data. The data (images, video) really belongs to your agency and needs to be handled as such. It may have personal identifying information. Some groups may routinely post videos and images to the web or tweet, which might not be appropriate. Therefore, you may want to make clear what the data management policies are applicable to flights on your behalf.

 

  • Privacy, state laws, or other regulations plus the public perception.  There may be state or local rules that impact the use of SUAS. Regardless, if you have a group flying SUAS for disasters, the residents will need to be aware that they are legitimate- plus the teams will be magnets for residents asking for help or assistance. So you will probably want to plan to have an agency representative in uniform or vest with the team.

 

  • Some SUAS may be software disabled from flying in TFR areas. DJI Phantom 3 and Inspires, which are very common, are now disabled by the manufacturer when a TFR is in place. So that may be something to discuss with your SUAS team.  DJI does have a procedure that allows agencies to override the software and fly up to 1.5 nautical miles from an airport, trusting the group to have obtained permissions.