Refraction-corrected bathymetric point cloud from the UAS survey of the coral reef off Waiakane, Molokai, Hawaii, 24 June 2018

Metadata also available as - [Outline] - [Parseable text] - [XML]

Frequently anticipated questions:

What does this data set describe?

Title:
Refraction-corrected bathymetric point cloud from the UAS survey of the coral reef off Waiakane, Molokai, Hawaii, 24 June 2018
Abstract:
This portion of the data release presents a bathymetric point cloud from an unoccupied aerial system (UAS) survey of the coral reef off Waiakane, Molokai, Hawaii, on 24 June 2018. The point cloud has been corrected for the effects of refraction using the techniques described in Dietrich (2017a). The point cloud was created from structure-from-motion (SfM) processing of aerial imagery collected using a UAS with a Ricoh GR II digital camera fitted with a circular polarizing filter. During the survey, a pressure sensor was deployed in the survey area to gain an accurate measurement of the water surface elevation (WSE). After a preliminary dense point cloud was derived from SfM processing, the WSE was used to calculate apparent water depths. These apparent depths along with the estimated camera positions and orientations were used as inputs for the multi-view refraction correction python script (py_sfm_depth.py) described in Dietrich (2017b). The refraction-corrected point cloud showed a substantial improvement in accuracy over the uncorrected point cloud. When compared to the 2013 U.S. Army Corps of Engineers Topobathy Lidar for the area in the central portion of the data set the vertical RMSE for the refraction-corrected point cloud was 0.241 meters with a mean residual of -0.010 meters, whereas the vertical RMSE for the uncorrected point cloud was 0.426 meters with a mean residual of -0.334 meters (see the Vertical Positional Accuracy Report in the metadata for a complete description of the accuracy analysis). For this data release, the final refraction-corrected point cloud is presented in the LAZ format (LAS 1.2 specification). The point cloud has 35,083,205 points with an average point spacing of 0.07 meters. Each point in the point cloud contains an explicit horizontal and vertical coordinate and red, green, and blue (RGB) color values. References Cited: Deitrich, J.R., 2017a, Bathymetric Structure-from-Motion: extracting shallow stream bathymetry from multi-view stereo photogrammetry: Earth Surface Processes and Landforms, https://doi.org/10.1002/esp.4060. Deitrich, J.R., 2017b, py_sfm_depth: Github online repository, https://github.com/geojames/py_sfm_depth.
Supplemental_Information:
Additional information about the field activity from which these data were derived is available online at:
https://cmgds.marine.usgs.gov/fan_info.php?fan=2018-617-FA
Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.
  1. How might this data set be cited?
    Logan, Joshua B., and Storlazzi, Curt D., 20220321, Refraction-corrected bathymetric point cloud from the UAS survey of the coral reef off Waiakane, Molokai, Hawaii, 24 June 2018: data release DOI:10.5066/P9XZT1FK, U.S. Geological Survey, Pacific Coastal and Marine Science Center, Santa Cruz, California.

    Online Links:

    This is part of the following larger work.

    Logan, Joshua B., and Storlazzi, Curt D., 2022, Aerial imagery and structure-from-motion-derived shallow water bathymetry from a UAS survey of the coral reef off Waiakane, Molokai, Hawaii, June 2018: data release DOI:10.5066/P9XZT1FK, U.S. Geological Survey, Pacific Coastal and Marine Science Center, Santa Cruz, CA.

    Online Links:

  2. What geographic area does the data set cover?
    West_Bounding_Coordinate: -157.15679
    East_Bounding_Coordinate: -157.15388
    North_Bounding_Coordinate: 21.09352
    South_Bounding_Coordinate: 21.08451
  3. What does it look like?
    https://www.sciencebase.gov/catalog/file/get/61b7b89dd34e9e224abffd87?name=Waiakane_2018-06-24_refraction_corrected_point_cloud_browse.jpg&allowOpen=true (JPEG)
    Top panel shows a perspective view of the Waiakane point cloud from the 24 June 2018 UAS survey; bottom left plot shows the vertical error distribution for uncorrected (top, purple) and refraction-corrected (bottom, orange) point clouds when compared to the USACE NCMP Topobathy Lidar; bottom right panel shows SfM depths vs Topobathy Lidar depths for the uncorrected (purple) and refraction-corrected (orange) point clouds.
  4. Does the data set describe conditions during a particular time period?
    Calendar_Date: 24-Jun-2018
    Currentness_Reference:
    ground condition at time data were collected
  5. What is the general form of this data set?
    Geospatial_Data_Presentation_Form: point cloud digital data
  6. How does the data set represent geographic features?
    1. How are geographic features stored in the data set?
      This is a Point data set.
    2. What coordinate system is used to represent geographic features?
      Grid_Coordinate_System_Name: Universal Transverse Mercator
      Universal_Transverse_Mercator:
      UTM_Zone_Number: 4
      Transverse_Mercator:
      Scale_Factor_at_Central_Meridian: 0.9996
      Longitude_of_Central_Meridian: -159.0
      Latitude_of_Projection_Origin: 0.0
      False_Easting: 500000.0
      False_Northing: 0.0
      Planar coordinates are encoded using row and column
      Abscissae (x-coordinates) are specified to the nearest 0.001
      Ordinates (y-coordinates) are specified to the nearest 0.001
      Planar coordinates are specified in meters
      The horizontal datum used is NAD83 (National Spatial Reference System PA11) (EPSG:6322).
      The ellipsoid used is GRS 1980 (EPSG:7019).
      The semi-major axis of the ellipsoid used is 6378137.0.
      The flattening of the ellipsoid used is 1/298.257222101.
      Vertical_Coordinate_System_Definition:
      Altitude_System_Definition:
      Altitude_Datum_Name: GRS 1980 Ellipsoid (EPSG:7019)
      Altitude_Resolution: 0.001
      Altitude_Distance_Units: meters
      Altitude_Encoding_Method:
      Explicit elevation coordinate included with horizontal coordinates
  7. How does the data set describe geographic features?
    Entity_and_Attribute_Overview:
    The attribute information associated with point cloud follows the LAZ file standard. Attributes include location (northing, easting, and elevation in the NAD83(PA11)/UTM zone 4N (EPSG:6634) horizontal and NAVD88 vertical coordinate systems), color (red, blue, and green components), intensity, and classification. All points are classified as 0 (unclassified), with intensity as 0.
    Entity_and_Attribute_Detail_Citation:
    American Society for Photogrammetry and Remote Sensing (ASPRS; 2013, https://www.asprs.org/committee-general/laser-las-file-format-exchange-activities.html) and Isenburg (2013, https://doi.org/10.14358/PERS.79.2.209)

Who produced the data set?

  1. Who are the originators of the data set? (may include formal authors, digital compilers, and editors)
    • Joshua B. Logan
    • Curt D. Storlazzi
  2. Who also contributed to the data set?
  3. To whom should users address questions about the data?
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    Attn: PCMSC Science Data Coordinator
    2885 Mission Street
    Santa Cruz, CA

    831-427-4747 (voice)
    pcmsc_data@usgs.gov

Why was the data set created?

These data were collected to characterize the morphology and rugosity of the shallow fringing coral reef off Waiakane, Molokai, Hawaii, as part of a larger USGS study of nearshore circulation and hydrodynamic properties of coral reefs. The point cloud can be used with geographic information systems (GIS) software or other three-dimensional analysis software for research purposes.

How was the data set created?

  1. From what previous works were the data drawn?
    Dietrich (2017a) (source 1 of 3)
    Dietrich, J.T., 2017, Bathymetric Structure-from-Motion: extracting shallow stream bathymetry from multi-view stereo photogrammetry: Earth Surface Processes and Landforms, journal article.

    Online Links:

    Type_of_Source_Media: online
    Source_Contribution:
    Journal article describing technique to correct structure from motion data for the effects of refraction in shallow water.
    py_sfm_depth (Dietrich, 2017b) (source 2 of 3)
    Dietrich, J.T., 2017, py_sfm_depth: GitHub, online software repository.

    Online Links:

    Type_of_Source_Media: online
    Source_Contribution:
    Online software repository with python script to correct structure from motion point clouds for the effects of refraction in shallow water.
    2013 USACE NCMP Topobathy Lidar (source 3 of 3)
    U.S. Army Corps of Engineers, Joint Airborne Lidar Bathymetry Technical Center of Expertise (JALBTCX), 2016, 2013 USACE NCMP Topobathy Lidar: Molokai (HI): National Oceanic and Atmospheric Administration (NOAA) Digital Coast Data Access Viewer, NOAA Office for Coastal Management, online.

    Online Links:

    Type_of_Source_Media: online database
    Source_Contribution: Topobathy lidar used for validation of SfM data.
  2. How were the data generated, processed, and modified?
    Date: 24-Jun-2018 (process 1 of 6)
    Aerial imagery was collected using a Department of Interior-owned 3DR Solo quadcopter fitted with a Ricoh GR II digital camera featuring a global shutter. The UAS was flown on pre-programmed autonomous flight lines which were oriented roughly shore-normal and were spaced to provide approximately 75 percent overlap between images from adjacent lines, at an approximate altitude of 100 meters above ground level (AGL). The camera was triggered at 1 Hz using a built-in intervalometer. Before each flight, the camera’s digital ISO, aperture, and shutter speed were adjusted for ambient light conditions. A total of five flights were conducted for the survey between 16:40 and 17:45 UTC (06:40 and 07:45 HST). Flight F01 was a reconnaissance flight, and no mapping imagery was collected during the flight. Flights F02 and F03 were conducted at an approximate altitude of 100 meters above ground level (AGL), resulting in complete coverage of the mapping area with a nominal ground-sample-distance (GSD) of approximately 2.5 centimeters per pixel. Flights F04 and F05 were conducted using the same flight lines and altitudes of F02 and F03, but the camera was fitted with a circular polarizing filter to reduced reflections and provide improved imaging of the seafloor through the water surface. Only the images collected with the polarizing filter (F04 and F05) were used for the creation of the point cloud presented in this data release. Person who carried out this activity:
    Joshua Logan
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    Physical Scientist
    2885 Mission Street
    Santa Cruz, CA
    US

    831-460-7519 (voice)
    831-427-4748 (FAX)
    jlogan@usgs.gov
    Date: 24-Jun-2018 (process 2 of 6)
    Survey control was established using twenty temporary ground control points (GCPs) distributed throughout the survey area. The GCPs were placed using a combination of kayaking, wading, and snorkeling. The GCPs consisted of: nine submerged targets consisting of small (80 centimeter X 80 centimeter) square tarps with black-and-white cross patterns anchored to the shallow (less than 1.5 meters deep) seafloor using 2 pound fishing weights; nine sub-aerial targets consisting of orange plastic five-gallon bucket lids (32 centimeter diameter) painted with a black “X” pattern and affixed in a horizontal orientation to vertical rebar stakes placed in areas of reef rubble to provide the targets with sufficient elevation to remain above the water surface during the survey; and two sub-aerial ground targets consisting of small (80 centimeter X 80 centimeter) square tarps with black-and-white cross patterns placed in the sand at the shoreline. Two of the submerged targets were disturbed by waves or currents during the survey and were not used for SfM processing. All GCP positions were measured using post-processed kinematic (PPK) GPS, using corrections from a GPS base station on a temporary benchmark (MK02) located approximately 1 kilometer away from the study area. Reference coordinates for MK02 were established using the mean position derived from four static GPS occupations with durations greater than 4 hours submitted to the National Geodetic Survey Online Positioning User Service (NGS OPUS). Person who carried out this activity:
    Joshua Logan
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    2885 Mission Street
    Santa Cruz, CA

    831-460-7519 (voice)
    jlogan@usgs.gov
    Date: 24-Jun-2018 (process 3 of 6)
    To measure approximate water surface elevations during the survey, a pressure sensor was temporarily deployed at a central location within the mapping area approximately 450 meters offshore. Water depths were recorded at 1 Hz and were adjusted to compensate for atmospheric pressure using an atmospheric pressure sensor concurrently deployed nearby. The elevation of the pressure sensor port was measured using the same PPK GPS used for the GCPs. The water surface elevation (WSE) was calculated using the sum of the measured ellipsoidal height of the sensor and the mean water depth measured during the duration of the UAS flights. To correct for potential low-frequency water level fluctuations during the survey (such as those caused by tidal fluctuation, or wind and wave setup), separate WSE were calculated for flights F02 and F03 (unfiltered, “non-polarized” imagery) and for F04 and F05 (imagery collected with a circular polarizing filter). Wave action was minimal during both time periods: depths varied by 0.138 meters with a standard deviation of 0.021 meters, and 0.170 meters with a standard deviation of 0.020 meters during the acquisition of the non-polarized and polarized imagery, respectively. Longer-frequency water level changes were also found to be minimal during the time period. Despite the different time periods, the resulting final mean WSE of the time periods for F02 and F03, and F04 and F05 differed by less than 0.001 meters. The final resulting mean WSE for both time periods was calculated as 16.327 meters above the NAD83 GRS80 ellipsoid. Person who carried out this activity:
    Joshua Logan
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    2885 Mission Street
    Santa Cruz, CA

    831-460-7519 (voice)
    jlogan@usgs.gov
    Date: 2018 (process 4 of 6)
    The image files were renamed using a custom python script. The file names were formed using the following pattern Fx-YYYYMMDDThhmmssZ_Ryz.*, where: - Fx = Flight number - YYYYMMDDThhmmssZ = date and time in the ISO 8601 standard, where 'T' separates the date from the time, and 'Z' denotes UTC ('Zulu') time. - Ry = RA or RB to distinguish camera 'RicohA' from 'RicohB' - z = original image name assigned by camera during acquisition - * = file extension (JPG or DNG)
    The approximate image acquisition coordinates were added to the image metadata (EXIF) ('geotagged') using the image timestamp and the telemetry logs from the UAS onboard single-frequency 1-Hz autonomous GPS. The geotagging process was done using the GeoSetter software package. To improve timestamp accuracy, the image acquisition times were adjusted to true ('corrected') UTC time by comparing the image timestamps with several images taken of a smartphone app ('Emerald Time') showing accurate time from Network Time Protocol (NTP) servers. For this survey, + 00:00:02 (2 seconds) were added to the image timestamp to synchronize with UTC time. The positions stored in the EXIF are in geographic coordinates referenced to the WGS84(G1150) coordinate reference system (EPSG:7660), with elevation in meters relative to the WGS84 ellipsoid.
    Additional information was added to the EXIF using the command-line 'exiftool' software with the following command: exiftool ^ -P ^ -Copyright="Public Domain. Please credit U.S. Geological Survey." ^ -CopyrightNotice="Public Domain. Please credit U.S. Geological Survey." ^ -ImageDescription="Low-altitude aerial image of the shallow coral reef off Waiakane, Molokai, HI, USA, from USGS Unmanned Aircraft System (UAS) survey 2018-617-FA." ^ -Caption-Abstract="Coral reef off Waiakane, Molokai, HI, USA, from USGS survey 2018-617-FA." ^ -Caption="Aerial image of the shallow coral reef off Waiakane, Molokai, HI, USA, from USGS Unmanned Aircraft System (UAS) survey 2018-617-FA." ^ -sep ", " ^ -keywords="coral reef, Molokai, Maui County, Hawaii, Waiakane, 2018-617-FA, Unmanned Aircraft System, UAS, drone, aerial imagery, U.S. Geological Survey, USGS, Pacific Coastal and Marine Science Center" ^ -comment="Low-altitude aerial image from USGS Unmanned Aircraft System (UAS) survey 2018-617-FA." ^ -Credit="U.S. Geological Survey" ^ -Contact="pcmsc_data@usgs.gov" ^ -Artist="U.S. Geological Survey, Pacific Coastal and Marine Science Center" Person who carried out this activity:
    Joshua Logan
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    2885 Mission Street
    Santa Cruz, CA

    831-460-7519 (voice)
    jlogan@usgs.gov
    Date: 2018 (process 5 of 6)
    Structure-from-motion (SfM) processing techniques were used to create the point clouds in the Agisoft Photoscan/Metashape software package using the following workflow with the JPG images from F04 and F05 (images collected with a polarizing filter): 1. Preliminary image alignment and sparse point cloud error reduction was performed using shoreline imagery to develop an a priori camera lens model. The lens model was then fixed for the subsequent SfM processing steps. 2. Additional image alignment for all the images was performed with the following parameters - Accuracy: 'high'; Pair selection: 'reference', 'generic'; Key point limit: 0 (unlimited); Tie point limit: 0 (unlimited). 3. Sparse point cloud error reduction was performed using an iterative gradual selection and camera optimization process in which all sparse points exceeding a Reconstruction Uncertainty of 10 were removed from the sparse point cloud. Additional sparse points obviously above or below the true surface were manually deleted after the last error reduction iteration, and a final camera optimization was performed. 4. Ground control points (GCPs) were manually marked for all GCPs. All submerged GCPs were disabled in subsequent processing steps to serve as validation check points. 5. A final camera optimization was performed, and a dense point cloud was created using the 'high' accuracy setting, with 'aggressive' depth filtering. 6. Some manual editing was performed to delete obvious high and low noise from the point cloud. Classification was run on the point cloud to identify 'low noise' and 'ground' points. 7. The estimate camera positions were exported for use in further processing. 8. The point cloud was exported to an LAZ file for further processing. Person who carried out this activity:
    Joshua Logan
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    Physical Scientist
    2885 Mission Street
    Santa Cruz, CA
    US

    831-460-7519 (voice)
    831-427-4748 (FAX)
    jlogan@usgs.gov
    Date: 2018 (process 6 of 6)
    Post-processing of the point cloud was performed to reduce the effect of refraction at the air-water interface using the multi-view refraction correction python script (py_sfm_depth.py) described in Dietrich (2017a). The script was used as part of the following workflow: 1. The point cloud was thinned to 5 cm point spacing using the LAStools 'lasthin' utility using the 'lowest' operator, keeping only points that were classified as 'ground'. 2. The thinned point cloud was exported to a csv text file using the LAStools 'las2txt' utility. 3. The water surface elevation of 16.327 meters relative to the GRS80 ellipsoid, was added as a field in the point cloud text file. 4. Multi-view refraction correction was performed on the point cloud text file using the 'py_sfm_depth' script (Dietrich, 2017b), with the estimated camera positions from the SfM processing steps, above. 5. The resulting refraction-corrected point cloud was converted to LAZ format using the LAStools 'txt2las' utility. Person who carried out this activity:
    Joshua Logan
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    Physical Scientist
    2885 Mission Street
    Santa Cruz, CA
    US

    831-460-7519 (voice)
    831-427-4748 (FAX)
    jlogan@usgs.gov
  3. What similar or related data should the user be aware of?

How reliable are the data; what problems remain in the data set?

  1. How well have the observations been checked?
    No formal attribute accuracy tests were conducted.
  2. How accurate are the geographic locations?
    Horizontal accuracy was estimated by using 7 submerged ground control points (GCPs) as check points. During acquisition, both submerged and sub-aerial GCPs were deployed near one another. Sub-aerial GCPs were deployed on vertical stakes to provide sufficient elevation to maintain the GCP above the water surface elevation. Submerged GCPs were deployed nearby on the seafloor in water generally less than 1 meter deep. The sub-aerial GCPs were used in the SfM processing and the submerged GCPs were withheld to serve as check points to evaluate the accuracy of the SfM products. The horizontal RMSE for the SfM derived positions of the submerged GCPs relative to their PPK GPS-measured positions was 0.133 meters. The addition of the estimated horizontal GPS accuracy (0.050 meters) in quadrature results in a total horizontal accuracy estimate of 0.142 meters for the point cloud. It should be noted that this error estimate is for areas near where GCPs were placed and in similar water depths. The effects of refraction at the air-water interface are likely to have caused significant displacement of the apparent position of objects relative to their true position, especially in deep water, or near the edges of the point cloud where the position of objects was likely derived from off-nadir sectors of the raw imagery. Additional sources of error such as poor image-to-image point matching due poor water clarity (such as on the shallow reef flat neat the shoreline), uniform substrate texture (such as mud and sand near the shoreline), or greater water depths resulting in poor surface reconstruction likely caused additional localized errors in some portions of the point cloud which exceed this estimate.
  3. How accurate are the heights or depths?
    Vertical accuracy was estimated by comparing the points in the central portion of the point cloud (between UTM easting 691605 and 691720, and UTM northing 2332805 and 2333245) to individual points in the 2013 USACE NCMP Topobathy Lidar data set. For this area, the vertical accuracy analysis was performed by selecting points from the NCMP lidar point cloud that were within 5 cm of a point in the SfM point cloud. For each SfM point only the nearest point was retained, resulting in a set of 130,444 point pairs. The vertical differences between these point pairs were compared and the vertical residuals were tabulated. The resulting vertical RMSE for these points was found to be 0.241 meters (MAE 0.154 meters), with a mean residual of -0.010 meters. The refraction-corrected point cloud showed a significant improvement over the uncorrected point cloud: the same analysis for the uncorrected point cloud showed an RMSE of 0.426 meters (MAE 0.343 meters), with a mean residual of -0.334 meters. To estimate total accuracy of the point cloud, the estimated vertical accuracy of the NCMP lidar (approximately 0.250 meters, calculated based on depth relative to a WSE of 16.327) was added to the RMSE in quadrature resulting in a total vertical accuracy estimate of 0.347 meters for the point cloud. It should be noted that this error estimate is for the defined central portion of the point cloud only. The effects of refraction at the air-water interface are likely to have caused significant vertical errors of the apparent position of objects relative to their true position, especially in deep water, or near the edges of the point cloud where the position of objects was likely derived from off-nadir sectors of the raw imagery. Additional sources of error such as poor image-to-image point matching due poor water clarity (such as on the shallow reef flat neat the shoreline), uniform substrate texture (such as mud and sand near the shoreline), or greater water depths resulting in poor surface reconstruction likely caused additional localized errors in some portions of the point cloud which exceed this estimate.
  4. Where are the gaps in the data? What is missing?
    Dataset is considered complete for the information presented, as described in the abstract. Users are advised to read the rest of the metadata record carefully for additional details.
  5. How consistent are the relationships among the observations, including topology?
    No formal logical accuracy tests were conducted.

How can someone get a copy of the data set?

Are there legal restrictions on access or use of the data?
Access_Constraints None
Use_Constraints USGS-authored or produced data and information are in the public domain from the U.S. Government and are freely redistributable with proper metadata and source attribution. Please recognize and acknowledge the U.S. Geological Survey as the originator(s) of the dataset and in products derived from these data. This information is not intended for navigation purposes.
  1. Who distributes the data set? (Distributor 1 of 1)
    U.S. Geological Survey - ScienceBase
    Denver Federal Center, Building 810, Mail Stop 302
    Denver, CO
    United States

    1-888-275-8747 (voice)
    sciencebase@usgs.gov
  2. What's the catalog number I need to order this data set? The topographic point clouds are available as LAZ files.
  3. What legal disclaimers am I supposed to read?
    Unless otherwise stated, all data, metadata and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected. Although these data and associated metadata have been reviewed for accuracy and completeness and approved for release by the U.S. Geological Survey (USGS), no warranty expressed or implied is made regarding the display or utility of the data on any other system or for general or scientific purposes, nor shall the act of distribution constitute any such warranty.
  4. How can I download or order the data?
  5. What hardware or software do I need in order to use the data set?
    This file contains a point cloud data in LAZ format (LAS 1.2 specification). The user must have software capable of displaying and processing the .laz format file.

Who wrote the metadata?

Dates:
Last modified: 21-Mar-2022
Metadata author:
U.S. Geological Survey, Pacific Coastal and Marine Science Center
Attn: PCMSC Science Data Coordinator
2885 Mission Street
Santa Cruz, CA

831-427-4747 (voice)
pcmsc_data@usgs.gov
Metadata standard:
Content Standard for Digital Geospatial Metadata (FGDC-STD-001-1998)
This page is <https://cmgds.marine.usgs.gov/catalog/pcmsc/DataReleases/ScienceBase/DR_P9XZT1FK/Waiakane_2018-06-24_refraction_corrected_point_cloud_metadata.faq.html>
Generated by mp version 2.9.51 on Tue Mar 22 13:38:42 2022