On Saturday, March 22, 2014, at 10:37 a.m. local time, a major mudslide occurred 4 miles east of Oso, Washington in the North Western United States. The mudslide covered over 2.5 Km2, buried 44 properties. More than 176 people were initially reported missing.
That morning, the Snohomish County Sheriff’s Helicopter Rescue Team was ready to launch on a training mission in their Bell UH-1H “SnoHawk 10” when the request came to survey an area where a mudslide was reported. The magnitude of the situation was not initially known, but once the aircrew arrived on scene and grasped the immensity of the situation, additional rescue and aviation resources were called in. The aircrew immediately started conducting life-saving hoist rescue missions together with the first responding Fire agencies from Oso and Darrington. U.S. Navy Whidbey Search and Rescue arrived with their Sikorsky MH-60 and assisted in conducting hoist rescues..Eight lives were saved that day due to the quick response and combined efforts of these agencies.
The responding aircraft, equipped with stabilized infrared imaging cameras and the latest augmented reality mapping systems were launched to be the “eyes in the sky” adding to the safety of the first responders and communicating valuable information about the scene to the crews on the ground. These air assets were able to provide a uniquely valuable perspective. The information that was able to be shared with the ground crews, decision makers and first responders utilized the latest technology that allowed the flight officers to toggle between live video and the pre-disaster aerial photography for a “before and after” view of the area.
This augmented reality technology also provided the aircrew the capability to display critical information including addresses, property boundaries and owner names of the homes affected by the disaster, overlaid on top of the live video. The aircrews were able to respond quickly to direct air and ground rescue efforts into the most devastated and populated areas.
Snohomish County Chief Pilot Bill Quistorf, recalls how the Augmented Reality technology aided them that morning, “ During our first search pattern Deputy Espeland exclaimed “There were 40 homes out here!” “40 homes where?” I asked him. “Right here!” he answered and pointed to the gravel and mud below us. I then looked over at the ARS monitor and saw that we were flying directly over Steelhead Haven, a residential area that had contained a series of streets and over 40 homes. The Churchill ARS allowed us to verify that we were directly over the top of the small development. I hovered two to three feet above the debris field so we could conduct a detailed visual search and detailed thermal search of the residential area. “We did not locate any additional survivors in the main debris field, but due to the real-time moving map and map overlay features of the Churchill ARS, I was confident that we had searched the area with 100% search coverage. The ARS also allowed us to visually compare the historic satellite image of Steelhead Haven with the physical destruction that we were witnessed and then realize the totality of that destruction.”
This Augmented Reality technology has proven to be an invaluable tool to first responders, supporting the expanded role of search and rescue, medical aid, and fire management missions.
Core capabilities such as the display of GIS information to include: addresses, street names, businesses and hiking trails on top of the live EO/IR video gives the operator a clear picture of exactly where the camera is looking. This overlay view saves the crew from the struggle of correlating between multiple displays or having to fumble with map books for identifying locations. It provides immediate familiarity with the area and ability to share this information with those in need on the ground.
The synthetic imagery layer, comprised of 3D high resolution, color satellite imagery is linked to the camera view. It can be used to compare a color image of the area if viewing the video in IR or at night, to gather information about the target area prior to it being in view by the camera, to extend the field-of-view, or as used by the first responders in the OSO mudslide disaster, to show a before and after comparison of the disaster area.
Search patterns can be calculated and displayed on-screen and the camera can be set to automatically follow the line of the pattern at a designated speed. Perimeters can be drawn around fires or areas of interest and locations can be marked and saved. To aid in reporting and information sharing, this information is easily exportable to Google Earth or ESRI Explorer type programs.
The airborne augmented reality system also adds mission management capability, interfacing and controlling a wide range of external equipment including steering the camera and searchlight, commanding the controls for the downlink and interfacing to a wide range or RADAR type targets. Targets such as emergency vehicles (AVL), boat locations (AIS), air traffic (TCAS or ADS-B) and even cell phone locations (when logged into a server) can be displayed over real-time video in the aircraft for up to the minute display on pertinent information and locations.
As other technologies are added to the platform these can also be integrated, bringing a heightened level of situational awareness. One such example is the integration of a Mobile Ad-Hoc Mesh network. These radios, which transmit voice similar to a traditional Motorola, are capable of self-forming networks which relay voice and data through intermediary radios. The ad-hoc mesh network can be used to transmit video to the ground, similar to a downlink system. Moreover, the radios can broadcast their GPS location (which some Motorola radios can do as well). The location signal can be integrated into the Augmented Reality mapping system on the aircraft, displaying an icon representing the firefighter or ground personnel over the live video from the gimbal in the aircraft – either in the air, or back on the ground (to the firefighters themselves, over the same mesh network, or at a command station). This not only gives the air crew the ability to easily direct the rescue forces to the most necessary areas, but provides the added and much needed capability to know the exact location of the ground teams and to keep them out of harm’s way, adding to the safety of our ground forces and decreasing the loss of life.
For more information, go to www.churchillnavigation.com
About the Author
Augmented Reality is a technology that superimposes a computer-generated image on a user’s view of the real world, or the integration of digital information with live video in real time.
