Improving Crew Resource Management

 Improving CRM in UAS operations

Crew Resource Management (CRM) is a method in aviation that increases the situational awareness and flexibility of an aircraft flight crew. This method has been shown to reduce errors and increase the safety of an aircraft in flight. Even though the flight crew has been removed from the aircraft in UAS operations CRM remains important, and I would argue is more important due to this fact. If the aircraft encounters a problem in flight the crew is not on board to react to the situation, and often must react from a distance that can distort the situation. For this reason the flight crew must preemptively respond to emergencies, and maintain effective communication and cooperation during the flight.

Tracking Aircraft use

The purposeful integration of CRM into UAS operations has begun with our C-Astral Bramor PPX aircraft. The complexity and frequent use of this aircraft made it a natural and important place to begin. The first step was to create a way to track aircraft use. Figure 1 shows the sheet created to track aircraft use. It is a very simple checkout sheet that would commonly be seen in any operation tracking material usage.

Figure 1: Checkout sheet

This simple sheet enables us to quickly determine when the vehicle was last flown, what sensor is equipped, where the vehicle was flown, and who is responsible for the aircraft at that time. This basic level of book keeping is the first step to enabling repeat flights and allows an individual to officially accept responsibility for the aircraft.

Standardized Metadata forms

The second, and larger, step to enabling repeat flights was to standardize the metadata collected and recorded during flights. Prior to the creation of a standardized form operators had a general idea of what they needed to collect, but it took a few flights for new operators to figure out what was and was not important. As operations expanded into multiple multirotors, multiple crews, the Bramor PPX (also with multiple crews) ensuring that each crew knew the correct data became very cumbersome. Figure 2 shows the sheet that was created.

Figure 2: Metadata form

This sheet was created as a text file so that it could be opened on any computer and without risk of format changes between versioning of software. This file provides little ambiguity on what needs to be collected, and separates the information into sections. Each of these sections provides information for different purposes. The "general" section provides the information required to replicate the flight and inform the crew of battery usage for cycle tracking. The "flight information" and "geolocating" sections provide information required for data processing and logbook entries. Knowing data collection start and end times, along with the location, allows the data analysts to access the CORS network and increase the accuracy of the PPK GPS equipped on the Bramor PPX. Knowing the coordinate system being used, if a PPK GPS is not in use, allows the analysts to process the EXIF data in images appropriately. The "weather" section is where the flight crew will enter the METAR from the nearest airport. METARs are put out every hour, and provide more local weather than many weather forecasting services. The "crew" section allows the recording of the core crew present for the operations, and allows for more accurate inquiry of operational details (asking the sensor operator about sensor specific questions or the PIC for flight specific questions). The "notes" section at the end is very important. This section allows the crew to note anything, positive or negative, unique that occurred during the flight and can provide additional context not present in the previous sections, and allows any member of the flight crew to have their opinion officially written down.

Crew formation

The Bramor PPX crew was made a standard three people with additional visual observers optional. Creating a standard crew prevented the need to constantly train new crew members, and allowed experience with the vehicle to be rapidly gained between the three crew-members. This experience allowed the crew to understand which portions of the operation needed improvement and which parts just needed additional training. For example, parachute packing and battery charging were discovered to be portions of the operation that required training to perform adequately. However, another part of the operation did need improvement, the checklists.

Checklist Creation

During repeat operations with the same crew we discovered that the checklist provided needed a few edits and clarifications. We began writing these in the margins and jumping around the checklist. After one flight of adding no edits to the checklist we began a full revision of the checklist. The biggest contributor to a full revision was the ordering of the sub-checklists in the checklist. Figure 3 shows the original and new checklist orders.

Figure 3: Checklist ordering

 
We believed that the original order had the flight crew moving back and forth too often and prevented an operational flow from developing. One benefit of the original checklist was that each item was given a number, and the numbering continued throughout the checklist. The new checklist includes all items from the original checklist, but now focuses on the stage of the operation being performed. The new checklist is shown in figure 4.

Figure 4: New Checklist

To the right of each heading a series of numbers can be seen, these are where the steps were located in the original checklist. All items from the original checklist are present in the new checklist, and items that had been written into the margins are also present. Not present in this checklist is a checklist for the sensor operator. This is currently in progress.

Crew Roles and Responsibilities

During checklist creation crew roles and responsibilities were defined. Three crew roles were defined pilot in command (PIC), sensor operator, and first officer (FO). PIC is responsible for the safe operation of the aircraft from arriving at the site to leaving the site. The sensor operator (Sensor) is responsible for ensuring successful operation of the vehicle payload. The FO is responsible for assisting both the PIC and Sensor in their operations pre-flight and act as an interface between visual observers (VOs) and the PIC in flight. The purpose of UAS operations is data collection, and this data collection must be conducted safely; for this reason the PIC and Sensor share command of the vehicle. During setup the FO will assist both the PIC and Sensor in their preparation for flight. Before vehicle launch two actions take place. First the PIC will check with the Sensor to ensure that the payload is ready for flight, the vehicle cannot be launched until the payload is ready. Second the PIC and Sensor both have an opportunity to cancel the flight if they are uncomfortable with sensor operation or flight safety, during this step the FO is encouraged to voice any concerns that they see. If any member of the flight crew brings up a portion of the checklist that they feel needs to be reviewed, then that portion of the checklist will be reviewed to ensure that the checklist has been completed adequately. Once the vehicle is in flight the FO's primary role is as an interface between the PIC and VOs. During flight the VOs will be updating the FO on which VO has visual contact with the vehicle, and if the PIC loses visual contact the FO will report that visual contact is maintained through a VO or if visual contact is lost entirely. If continuing the flight is determined to be unsafe then the PIC has final say in ending the flight.

The effect on operations

During our first operations it would take the flight crew up to 90 minutes to prepare for a flight, once reaching the site. After implementing the methods described above the flight crew can prepare for a flight in 15-20 minutes. This dramatic improvement is mostly due to changing the flight crew from "whoever is available and wants to fly" to "these are the only people who fly this vehicle", and the reorganization of the checklist. While the time from site arrival to vehicle launch has decreased the safety of the flight and data collection focus have not been compromised.

Assessing the effectiveness of a controlled burn

Controlled Burn at Doak Property

Multiple UAS operations are complicated and require a good amount of planning and coordination. A controlled burn at Purdue's Doak Field, shown in figure 1. The areas being burned are outlined in orange.

Figure 1: Burn areas

The goal for us was to collect a pre and post burn orthomosaic and constant EO/IR surveillance of the burn. In order to accomplish these goals we brought a Bramor PPX with an Altum multispectral sensor, and a DJI Matrice M600 equipped with a Zenmuse XT2, Sony Alpha A6000, and a PPK GPS. Menet aero assisted us with surveillance by providing a Bramor C4EYE platform and operators.

Having these three vehicles in the same operations area required a good amount of coordination between operators. Both sets of operators arrived at the burn site prior to the burn, and began to set up the operations area. Figure 2 shows the operations area, and will be further described below.

Figure 2: Operations area

Figure 2 shows our operations area set up in a small clearing in one of the fields. We created this clearing, using a weed eater, in a location that provided good take off (shown in blue) and recovery options (shown in green), in figure 3.

Figure 3: Operations area justification

As can be seen in figure 3 our positioning allowed us to take advantage of a driving path providing us an incredible length of clear area in a thin strip for takeoff and climb. This location also provided a large area of tall grass for recovery nearby so we would not have to go far to recover the vehicle. This close proximity proved beneficial during flights, and that will be discussed later. Most importantly this area kept us a good distance away from the fires that were being set. The winds were calm and variable, but generally provided a tailwind on takeoff. While a tailwind on takeoff is undesirable, the conditions in the field made it the best option. Had the winds been stronger we would have had to adjust the location of our operations area.

The first flight of the day took place about two hours before the burns were set to take place, and the Bramor PPX was equipped with an Altum Multispectral sensor and launched, flight plan shown in figure 4. The PPK GPS system tied to the Altum sensor was instrumental in the mission because the PPK data would allow the images to be processed accurately enough to create a comparison like the following: Story Map.

Figure 4: Bramor PPX flight plan

This flight plan provides 80% overlap and 80% sidelap for the images gathered, and at 400 feet provides a resolution of 5.2cm/px. This mission had a duration of a little over 10 minutes and has very little in the way of planning challenges. After the vehicle landed and was recovered we discovered that no images had been taken during the mission. When this was discovered I tasked a second flight crew with flying the area using a Matrice M600. This vehicle was equipped with a Sony Alpha a6000 camera tied to a PPK GPS, and would allow RGB data to be gathered with acceptable accuracy. This separate flight crew prepped their vehicle, created their flight plan, and conducted their mission as the Bramor PPX flight crew troubleshot the Altum sensor. Menet Aero provided assistance during this process and together we were able to get the problem solved. Once the problem was solve the Bramor PPX flight crew re-prepped the vehicle for flight, and waited for the M600 crew to return so the airspace would be free. The second Bramor PPX flight was able to collect data without a problem, but experienced frequent loss of communication issues. These issues were solved by mounting the Combox to a tall pole, and having a member of the flight crew follow the pilot in command. When the vehicle was recovered the controlled burns were about to begin. Due to the close proximity of the landing site, the inclusion of multiple crew members, and inclusion of data analysis capabilities in the flight planning we were able to quickly react to a sensor failure and successfully collect the required data.

During the Burn

As the controlled burns began airspace and air traffic considerations had to be made. Menet Aero was tasked with gathering persistent EO and IR data. To accompish this they brought a Bramor C4EYE. Purdue's flight crews were tasked with gathering spot EO and IR data, and brought a Matrice M600 equipped with a Zenmuse XT2. Prior to the first Bramor PPX flight it was determined that the C4EYE would fly at an altitude of 400 feet AGL and the M600 would fly at 200 feet AGL. These altitude ceilings were chosen in order to reduce the likelihood of traffic concerns, and increase the freedom of lateral movement for both vehicles during their flights.

Both the C4EYE and M600 were tasked with gathering EO and IR data during the controlled burns, but only the C4EYE was capable of staying aloft throughout all of the burns. With this in mind why bring the M600 when it creates traffic concerns? While the sensors on each vehicle gathered the same kinds of data the payloads were made with different intentions in mind. C4EYE video, shown in figure 5, is intended to provide surveillance capabilities and is capable of switching between EO and IR in order to more accurately track objects. The Zenmuse XT2 video, shown in figure 6 and 7, is intended to compare EO and IR side by side and records events in both spectrum.

Figure 5: C4EYE video

Figure 6: XT2 EO

Figure 7: XT2 IR

Figures 6 and 7 show the video collected by the M600's XT2 sensor over the same area, and demonstrates the benefit that IR data can provide. In figure 6 it is almost impossible to make out what is going on through the smoke, but in figure 7 it is clear as day where the fires are. The video from the C4EYE, figure 5, also shows this capability but because of it's higher altitude it can see more of the area and provide a large scale picture of the fire situation. Earlier it was mentioned that the C4EYE was capable of providing target tracking and GPS information on objects within it's field of view. Figures 8 and 9 show this.

Figure 8: Target

Figure 9: Target GPS information

In figure 8 we can see a person standing just above the targeting square in the image, and in figure 9 we can see the field of view of the sensor (the purple box) and the GPS location of the target. In this scenario the person is responsible for starting the fires in the controlled burn, and they were knowledgeable of exactly where the fires are. In another situation this information could be vital to providing actionable real time information to firefighters on the ground as the fires approach them. If the C4EYE provides so much more information than the M600 the question of why use both resurfaces. These two vehicles represent two different data requirements by different organizations. The M600 provides side by side data of the fires and how they spread with a more agile platform, but lacks the duration and target tracking capabilities of the C4EYE. This data is useful for situations that do not require immediate actionable information and provide very good data sets for later analysis. The C4EYE on the other hand sacrifices the amount of data collected to allow for a longer flight time, and the capability to provide real time actionable information in situations that require it.

Post Burn

As the burn began to near completion the Bramor PPX was prepared for a post burn data collection flight. After the commencement of the controlled burn the C4EYE was landed in order to open up the airspace at 400 feet. In light of the problem suffered during the first pre-burn flight led us to land the M600 and prepare it for a potential PPK mapping mission. We were confident that we had the Bramor PPX issue resolved, but we wanted to be sure we got some sort of post burn data for analysis in a worst case scenario. In order to recreate the pre-burn data-set as accurately as possible the pre-burn flight plan was unchanged and uploaded to the vehicle. The flight and recovery was uneventful and the data collection was successful. For a discussion of the data collection see the following link: Zach Miller's Blog.

Final Thoughts

The flight operations performed during the controlled burns at the Doak property were very successful and yielded a vast amount of good data. This data collection was possible due to constant coordination and communication that took place during the operations between the Purdue, Menet Aero, and burn teams. When reviewing the successes and improvement areas for the Purdue teams the following come to mind. On the positive side having a dedicated flight crew per vehicle, instead of one crew flying different vehicles during the different parts of the burn, allowed our teams to be more agile and respond to unexpected events. While the two flight crews allowed increased agility, two pieces of equipment played a larger role in increasing agility; a laptop and a generator. It sounds simple that a laptop would be taken during a flight it proved essential. After the first pre-burn flight we were able to quickly identify that the data was missing, and begin getting the vehicle prepped for another flight. This laptop also had an external hard drive which allowed easy data archiving after successful flights, and made transferring data to servers after the flight very quick. We were also able to quickly clear SD cards for their next flight, instead of swapping SD cards and attempting to remember which one had what data. The generator we brought was small, but allowed us to charge the M600 batteries after they were depleted in flight. This allowed us to get four flights out of three battery sets over the time of the burn.

There were two areas that I saw where improvement was needed; adequate preparation for full day missions, and regular flight crews with semi-rigid roles. Adequate preparation for full day missions was something that we attempted to do but missed a few key things. We brought a cooler with food and drinks, but given the weather (~90F and sunny) we did no have nearly enough. We also overlooked bringing a table, chairs, or a pop up shelter. Overlooking these items led us to using the bed of a pickup truck as a table and the cab for chairs. Thankfully Menet Aero, who has more experience in these types of operations, was able to provide us with water and a good example of what to emulate equipment wise. The second area of improvement, regular flight crews with semi-rigid roles, was again something that we thought we had but quickly broke down with the Bramor PPX. The M600 flight crew was smaller, two people, and had more experience with their vehicle so they were able to quickly get the vehicle prepped and in the air. Prior to this event the Bramor PPX was not flown as often due to the large area needed to make flying the vehicle "make sense" (justifying flying the Bramor PPX over the M600 for a 2 minute flight is difficult). The flight crew was also made up of three or four people with varying levels of experience with the platform, the PIC for the vehicle had remained constant but visual observers and other crew roles had not. This led to set up taking longer than necessary and a small amount of bickering during the checklist process. The flights were still conducted safely and effectively but the issues listed above really presented the issues with the more casual "who is available and wants to do this" attitude that we had towards filling in the flight crew. As the PIC for the Bramor PPX I looked at this situation and decided to begin to create a more constant flight crew, and begin to add rigidity to the crew roles mirroring manned aviation. This process will be covered in a future blog post.