Reducing false positives in Loc8

Finding GCPs in Loc8 was the topic of my previous blog post. I was able to find many of the points but was given many false positives. I believe this was due to using "Min pixel" at 1 and a large range of colors taken from the target. I also mentioned that the samples I took were affected by the distance from camera to target, the colors blurred together a bit. My first step in reducing false positives was to eliminate this issue, and hopefully this would reduce the color range collected.

Figure 1: GCPm2 samples

Figure 1: GCPa2 samples

When comparing the color ranges collected from aerial imagery (figures 3 and 4) to the ranges collected from the ground we can see that the ground collection has more vibrant colors.

Figure 3: GCPm color range

Figure 4: GCPa color range

Since I decided to change up the data sampling method in this test I decided to keep the settings the same, shown in figure 5. This way I could determine the effect of collecting data from images taken prior to the search.

Figure 5: Settings used

During this round of processing Loc8 alerted me to 18 images, 14 false positives and 4 positives. In my opinion this is already a massive improvement over the previous run. While I was still given mostly false positive alerts I was only given 14 of them instead of 132. The 14 false positives each had one or two alerts in each image which reduced the amount of time I had to spend checking each alert.

Figure 6: False positive

  

Figure 7: Found Aeropoint

 Figure 7 shows a successfully located aeropoint, but also shows two manual GCPs that were not found. Figure 8 zooms in closer to allow for a clearer view of all 3 targets.

Figure 8

Figure 9: Found manual GCP

Figure 9 shows a manual GCP being found, but multiple aeropoints being ignored. The "scents" given to the software were able to find both kinds of GCPs, but still generated a high ratio of false positives to positives. While the ratio was still high this method reduced the number of images flagged by false positives by ~90%, and the total number of false positives by more than that. There were also many false negatives given by this method, and that will need to be corrected for. I believe one way to find a balance between the two could be to set a color range based on an image captured before the flight.

Using Loc8 to find GCPs in the field

I decided to shift my focus from finding missing persons with Loc8 to locating important objects with Loc8. Using the dataset collected during the GCP demonstration a week ago I used Loc8 to find the GCPs. Since these objects were either bright yellow/black/white or black/neon pink/numbered they stood out from their surroundings, and their location was already known. I began by finding an image in the dataset that contained both kinds of GCPs used, figure 1.

Figure 1:Scent Image

I then created a color signature for each of the GCPs by using the viewer to sample colors from the scent image, results seen in figures 2 and 3.

Figure 2: Aeropoint color sample

Figure 3: Manual GCP color sample

These color samples are rather large, and the aeropoint set has some strange blues and greens in it. This is where the colors blurred together as they were zoomed in on. This would have likely been avoided if I had gathered the color range from images taken from a less extreme range.

Figure 4: Loc8 window

Figure 4 shows the settings I used for the processing performed. Using these settings, and the two spectral databases Loc8 returned 238 hits. Unfortunately 132 of these were false positives, and almost all of the 106 that contained the GCPs contained false positives as well. This is likely due to the large ranges of colors selected, and "Min Pixels" being set to 1. Even with a large number of false positives it was reassuring that the software was able to find each of the GCPs.

Figure 5

Figure 5 was the best find of the set having no false negatives or no false positives. Figure 6 zooms in on the found GCP and that shows that the marker was found based on one of the white squares on the point.

Figure 6

Figure 7

Figure 7 was another good find by the software, having no false positives, but it did have a few false negatives. These can be more clearly seen in figure 8 below.

Figure 8

 These GCPs ended up clustered pretty close together, and the software was able to find 2/5 in the image. I'm unsure of why it was unable to find 2 of the aeropoints, when it was able to find the other 2 in the same image. Similarly I'm not sure why it was unable to find the manual GCP. In practice this would still likely be adequate to find all the GCPs since they're so closely clustered together.

Figure 9

Figure 9 was able to find all 5 GCPs in the image, including manual and aeropoints. However most of the alerts in the image, 9/14, were false positives. Figure 10 more clearly shows four of the five GCPs that the software was able to locate.

Figure 10

Figure 11

Figure 11, similarly to figure 9, located multiple GCPs but provided mostly false positives. 

Following tests of the software will be focused on reducing the number of false positives.

Search and Rescue with Loc8

As mentioned in my previous post about GCP instruction, I had the students hide in a field before doing data collection. I used this data in Loc8 to simulate a search for missing persons. The students were divided into three groups based on what they were wearing, and given general directions on where to "hide". Group 1, figure 1, consisted of 3 students wearing entirely different colors.

Figure 1: Group 1

Thinking that red would be the easiest color for the software to find, I began by trying to find group 1 in the 252 images collected. I manually looked through the images and found that they were present in 30 of the images. I copied one of the images, figure 2, out of the data collected and began using it as a test image. I took color samples from this image and then ran Loc8 on the image to see if it would be able to find the sweatshirt.

Figure 2: Test image

Since the data collection was performed at 300 feet our missing persons can barely be seen in the image. This highlights the usefulness and need for this software. If a search for missing persons is done at a height that will allow the operator to cover a reasonable area in a timely manner, the people they are searching for will be easy to miss.

When I started working with the data set it seemed that this would also be a large problem for the software package. Regardless of the number of color samples I took from the target, I could not get the software to locate the target in the sample image. I zoomed in on the target, to prevent color averaging, and took as many samples as I could.

Figure 3: Zoomed target

This didn't work, so I tried different numbers of samples then averaging samples and zooming out for samples. After six attempts modifying the spectral sample taken, I started to play with the settings.Specifically I modified the minimum number of pixels for a hit. This setting appears to set a minimum threshold of grouped pixels to register as a hit, and helps prevent false positives. Reducing this number has the potential to increase the number of false positives shown.

Figure 4: Default settings

Figure 5: Modified settings

Modifying the settings to search for individual pixels was a bit of an extreme option, because the images were captured using a 20MP camera. I expected this to find the sweatshirt, but also expected it to take an excessive amount of time since the program would be looking through 20 million pixels. To my surprise it only took 15.85 seconds to find 4 clusters of the color in the image. 

Figure 6: Single pixel

Unfortunately when I used this method on the full set of images the program was only able to find the sweatshirt in this one image, and ignored the sweatshirt in the other 29 images where it is present. Reducing the minimum number of pixels also had the program alerting me to red covers on black tubs. 

Getting frustrated with this I had the program look for black in order to find another member of group 1.

Figure 7: Group 1 with black

I had more success searching for black in the image set, getting 24 useful tagged images. This run also found a member of group 2 who was also wearing a black shirt.

Figure 8: Group 2 find

While this pass through found 24 useful images I had to flag 95 images as not useful. Using the single pixel search alerted me to almost every shadow present in the imagery, as seen in figure 9.

Figure 9: Shadows

Experimenting with this data began showing be the capabilities of this software, but continued to reinforce the learning curve involved.

GCP instruction at Purdue Wildlife

05/14/2019

Since it was a beautiful day we took the students out to the Purdue Wildlife Area (PWA) and flew the Mavic to introduce Ground Control Points (GCPs) to students. GCPs are used as known "waypoints" in survey and structure from motion mapping. When creating structure from motion point clouds and maps from UAS flights the software will be told where these points are. This knowledge will reduce the amount of guessing that the software has to do in order to correctly match images for the point clouds. We brought two kinds of GCPs for demonstrations. Aeropoints, figure 1, and manual GCPs, figure 2.

Figure 1: Aeropoint

Figure 2: Manual GCP

The Aeropoints will collect GPS location, and create their own type of location network autonomously. The manual GCPs need some additional work. For the manual points a survey GPS is needed, and figure 3 shows Dr. Hupy instructing students on this process.

Figure 3: Manual GPS location instruction

Once students had been instructed on the purpose for GCPs, and how to manually create them, they were sent into the field to distribute them in the target area. After the GCPs had been distributed I sent the students out into the target area to hide. I'll be using the data we collected from this flight for more Loc8 work in the near future.

Loc8 testing

05/13/2018

I began testing the Loc8 software with images collected while working on my Master's Thesis. Many of these images contain traffic cones at certain places around the aircraft. Since Loc8 refers to their algorithms as the "Bloodhound Technique" I decided to call the images I'm looking for "Scent", and the images I'm searching "Targets". Using the viewer selection mode I selected a sample of colors present on the cones at the nose of the aircraft.

Figure 1: Scent image for cone finding

 Loc8 searched the 9 provided images for colors that matched the samples, and was able to locate cones in most other images.

Figure 2: Front view scent 1

In figure 2 we can see that Loc8 found two of the four cones,. In the bottom left corner below the image being displayed we can see "Image 6 of 7". This means that in 2 of the images no cone was found.

Another attempt was made with a larger sample of colors taken from the cones.

 

Figure 3: Front view scent2

For some reason this larger sample prevented the program from finding the cone on the left side of the aircraft. Not entirely sure why this is, when it expanded the objects found in other images. Figure 4 shows the program finding a fuel cap on the left wing with scent 2 that it did not find with scent 1.

Figure 4: Wing view scent 2

This software is clearly very capable, but has a bit of a learning curve that I will have to get through.