Comparing the senseFly S.O.D.A. and S110 sensors - Part 3
Audience: Pilots, Mission Planners, Post Processing
Classification: Public
Overview
The senseFly S110 RGB sensor has been a reliable part of our UAS processes. Coupled with the eBee RTK, the S110 sensor has proven capable when used with proper lighting and meteorological conditions.
In late 2016, senseFly introduced their new S.O.D.A. sensor for the eBee platform. This sensor has some material improvements over the S110 RGB and this brief is a summary of the initial findings on the operational use of the S.O.D.A. sensor and integrating it into the business processes.
This high-level overview is provided as guidance mainly for our pilots, mission planners, but should be considered by all team members. In the Part 1 brief, the S.O.D.A. sensor was examined at the hardware level, how the changes impact mission planning, and comparison with the S110 of the actual raw imagery from the flights. In Part 2 the orthomosaic output was analyzed. Readers will find it beneficial to review Part 1 and Part 2 prior to continuing.
Part 1 – Read it here.
Part 2 – Read it here.
Part 3 (this brief) of this series will examine the resulting point cloud quality produced from the imagery of the two sensors. Since Pix4D is recommended by senseFly, Pix4D version 3.1.23 is the post processing software. Pix4D processing is covered in Part 2 and not repeated here.
Evaluation Goals
Tight vertical cohesion in point clouds allows for precision when creating Digital Elevation Models (DEM). Using the Pix4D created point cloud outputs from our flights, the analysis and comparison of each data set will be centered around the following;
Determine point density from each data set.
Analyze Z axis point cohesion in open flat areas.
Analyze Z axis point cohesion in shaded area.
Analyze point cohesion around building.
Point solutions under tree forested canopy.
Compare the relative accuracy of the 1.2in GSD point clouds.
The techniques used to analyze the data sets will be 1) cross section analysis and, to best analyze the cohesion, 2) grid based Z axis deviation analytics. The final comparison to be reviewed will be DEM creation for both 1.2in GSD data sets and apply differential analytics. We are NOT covering absolute accuracy of any of the derived point cloud solutions in this brief.
Orthomosaic
Pictured below (Image 1) is the orthomosaic derived from the mission. This image is to provide visual reference for our discussion and to orient the reader to the analysis areas. The areas of interest are labeled A through G. Each area marked will be discussed with the relevant point cloud analysis from the above list of objectives.
Image 1 – Ortho of Mission with Marked Comparison Areas.
Point Clouds
In Table 1 below, the details of each derived raw point cloud solution from Pix4D is listed. This table data should largely be self-explanatory. However, the S110 1.8in GSD data set does contains larger point solutions of tree canopy, therefore a percentage of total point count is embedded vertical objects. These are reduced in the Final Density numbers shown. The data projection for this brief is WGS 84.
Table 1 – Data Set Point Cloud (Density is expressed as Points Per Square Meter)
Area A
In Area A (Image 2) the green building that has been referenced in briefs Part 1 and Part 2 is again used here, and is the sole analysis conducted of a structure. The profile pictured shows our mensuration area as a red box that is 10-meters wide (left to right) and 5-meters in width (top to bottom).
The S.O.D.A. images exhibited in brief Part 1 seemed to have better contrast quality than the images produced by the S110. Shaded areas are more difficult to derive an accurate point cloud solution from, and since the images produced by the S.O.D.A. sensor contained better contrast control and exposed more surface detail, then the expectation would be that the shadow area would contain a more precise point cloud in this area.
Image 2 –Mensuration Outline within Area A.
However, that is not the case. Applying cross section analysis to the mensuration area shown in Image 2, the cross-section analysis shown in Image 3 clearly shows that the S.O.D.A. point cloud has lower point solution cohesion (noisier) than the point cloud solution produced from the S110 at the edges of the building on the shaded side. Viewing the right half of Image 3, the building roof line is clearly demarcated in both data sets. The S.O.D.A. data set shows incorrect roof solutions present (black arrow). This will obviously impact any measurements that rely on the Z axis (vertical). There are also some incorrect solutions in the S110 1.2in GSD data set, but these are not as large as the S.O.D.A.
The two data sets are in agreement on Z axis (elevation) solutions, with both the ground on the left side and the roof line on the right side indicating the same elevation scale at each respective horizontal offset.
Image 3 – Profile of data points in Area A mensuration box.
Area B
Area B was also exhibited in briefs Part 1 and Part 2. The images produced by the S.O.D.A. sensor in this area were clearly better in image quality. Image 4 shows our mensuration area as a red box that is about 10-meters long (left to right) and 1-meter in width (top to bottom).
The mensuration box placement contains both grassy area and the sandy area. It is important to know if the point cloud maintains cohesion in this sandy area.
Image 4 –Mensuration Outline within Area B.
Viewing the data point solutions contained within the mensuration box outlined in Image 5, the S.O.D.A. solution appears slightly less cohesive. The S.O.D.A. solution does contain a higher point density so we expect more numerous points clustered. Note that in this cross section the Z axis is highly compressed, with each Z Axis line reflecting a 0.05-meter change (very small). The sandy area at from 0 - ~4-meters offset shows some extra noise in the both solutions.
Note too, that there is a slight differential in Z axis solution for the two data sets. The S.O.D.A. at the 5-meter offset mark shows approximately 95.60 meters, and the S110 1.2 solution indicating 95.55 meters, differing by 0.05-meter (small differential).
Image 5 – Profile of data points in Area B mensuration box.
Area C
In Area C (Image 6) we see the largest separation in point data solutions. The mensuration box (red rectangle) is about 10-meters in length (left to right) and 1-meter in width (top to bottom). It is located in the center of the road bed (visible in grey), which is a flat surface.
Image 6 –Mensuration Outline within Area C.
Viewing the profile of data points in Image 7, the S.O.D.A. solution shows considerable loss of cohesion. Interestingly, the S110 solution derived from the 1.8in GSD flight shows the best cohesion. Dashed lines are added to assist the reader in reading the high/low data marks from each set. The S.O.D.A. point solution has a >2-meter in high/low data points, the S110 1.2in GSD is slightly more than 1-meter, and the S110 1.8in GSD solution is slightly less than 1-meter.
Image 7 – Profile of data points in Area C mensuration box.
It is easier to visualize cohesion using the ΔZ analysis within an area. In Image 8 the deviation for the entire roadbed is provided. Using a grid size of 0.25-meter, an analysis of the highest and lowest data point is conducted and represented by a color gradient. ΔZ values of 0.015m deviation is blue (tight cohesion) and ΔZ of 0.20-meter deviation is red (loose cohesion).
In this view, it is clear that the S.O.D.A. sensor point cloud solution is much less coherent than the S110 1.2in point cloud solution. The S110 point cloud solution shows more uniform and tight cohesion across the majority of this surface.
Image 8 – Deviation Z Axis of Area C.
Area G
Area G (Image 9) contains a small, shallow pit and a pile of dirt to the side. The mensuration box (red rectangle) is about 15-meters in length (left to right) and 1-meter in width (top to bottom) and spans the pit and dirt pile approximately in the center of each. This area provides a test of terrain accuracy, and since it is on the edge of the field at the tree line, a test of point solution quality based on poor visibility (few images from the tree line side).
Image 9 –Mensuration Outline within Area G.
Viewing the profile of these data points in Image 10 both the S.O.D.A. and the S110 data point solutions are well defined. The S.O.D.A. shows a slightly higher pile profile. Each data set has high cohesion. The dashed line is present to assist the reader and represent the original elevation profile of the terrain. The S.O.D.A. solution better represents the actual lip around the pit. Both solutions show reduced point density on the shaded side of the pit.
Image 10 – Profile of data points in Area G mensuration box.
Area D
Area D (Image 11) represents an area of open field with well-groomed grass. This area is analyzed using ΔZ Deviation using a 0.25-meter grid and the results of the data sets are compared. Each area is scaled the same for comparison. The scale is from 0.008m deviation (blue) to 0.108m deviation (red).
The results for the S110 are clearly uniformly cohesive and the S.O.D.A. only slightly less so (see red ellipse). These ΔZ numbers are great for each sensor and are incredibly small considering they are derived from photogrammetry.
Image 11 –Deviation Z Axis within Area D.
Area E
Comparing Area E (Image 12) the ΔZ deviation method is again employed. This area has two elements to consider. There is a sunlit area of grass and a shaded area adjacent a tree line. The S110 point solution shows higher point cloud coherence (5 point Stars) in the open, sunlit grassy areas, which agrees with the Area D analysis described above.
The S.O.D.A. point solution shows much better coherence in the shaded areas (see Ellipses) than the S110 point solution. This is likely due to better contrast management and better image quality. Interesting the quality difference in coherence between this shaded area S.O.D.A. point cloud solution and that observed in Area C.
Image 12 –Deviation Z Axis within Area E.
Area F
Image 13 is a composite from all three data sets’ point cloud. The S.O.D.A. has the most complete solutions for areas under the tree canopy. The point solutions from the S110 1.8in GSD is more complete than those derived by the S110 1.2in GSD data set, but neither is close to the S.O.D.A. point solutions. Although not pictured here, the S.O.D.A. had returns from under the entire forested area, except an area under a small stand of conifers along the edge of Area C.
Image 13 – Point solutions in Area F.
Relative Accuracy of the Sensors
In the above examples the two sensors data point cloud solutions have been compared within selected areas. In several of these areas it was observed that S.O.D.A. and S110 point cloud solutions contained slight differences in Z axis solutions. How does the SOTA point cloud solution compare to the S110 in relative accuracy? An AGL analysis was conducted for both 1.2in GSD data sets and all vertical objects (poles, trees, buildings, etc.) were chopped off at ~1 meter AGL.
Since the focus of this brief is testing the S.O.D.A. sensor, a DEM is created from the S110 1.2in point cloud data set using a grid size of 0.5-meter and this S110 DEM is used as our reference base. Areas that do not have common solutions are not compared in this test (shown as white areas in Image 14). This includes most of the forest area, which is fine for the purpose here. The S.O.D.A. data is overlaid onto this DEM, and using a grid size of 0.1-meter for analysis, each grid is colorized based on ΔZ deviation from the S110 DEM (scaled clamped to +/- 0.2 meter).
In Image 14 it is evident that the open areas of our land surface show very high agreement between the two models altitude solutions. Green and Yellow are our friend in this image. The clear majority of the open surfaces shows that both data sets are in very tight alignment. Notice that the shadow side of the tree lines is one area where there exists consistent larger deviation, which is expected based the analysis in Area E. The road in Area C also shows large area of deviation in the solutions, which was observed in the point cloud analysis in Area C.
Image 14 – S.O.D.A. – S110 solution Differential Comparison. Areas C, D, E noted by black rectangles.
Point Cloud Analysis
The point cloud data from the S.O.D.A. sensor is comparable with the S110 sensor. The point cloud data is superior in certain areas and proved inferior in other areas. The S.O.D.A. performs well in penumbra shadow areas such as tree lines and poles, which is as expected based on the S.O.D.A. raw imagery. However, at least in this test, point cloud solutions in hard shadow (umbra) proved to be inferior, as in the case of the building edge shadow example in Area A. This is unexpected, again based on the raw images.
Considering the GSD of our flight test of 1.2in, the S.O.D.A. sensor had excellent solutions under the tree canopy. This is a positive, as it simplifies many processes when it comes to forest areas and improves accuracy. Although absolute accuracy will require testing in these areas, it is nice to see this capability.
Take Away / Conclusion
The S.O.D.A. sensor is overall an add for our production system. It is an improvement over the S110 in many ways; shorter flight times, better image quality, superior optics and sensor, and with no moving parts, durability will likely be better. BUT…
Do NOT throw away or retire the S110. At this stage the S110 seems to produce better orthos. As noted in Part 2, do not use the S.O.D.A. sensor payload for orthomosaic work until the ortho related issues are resolved. Based on the better image quality from the S.O.D.A. we should expect future improvement in both the workflow and the outputs.
Perhaps the S.O.D.A. requires different flight parameters than the S110, such as overlap percentages ratios and GSD, as the new ideal. Perhaps the post processing settings will require adjustments as these are also based on our processing experience of the S110.
Our testing to date has been based entirely on our experience with parameters for the S110, and our knowledge of that sensor behavior have been applied to the S.O.D.A. platform. As we continue to integrate this new sensor, expect changes in SOP related to the S.O.D.A. flight standards. Expect future briefs on the S.O.D.A. to be released as we perfect integration of this sensor platform into the business workflow and test absolute accuracy of the product.
Fly Safe and have Fun!
