SfM Quantitative Underwater Imaging Device with 5 cameras (SQUID-5) – Field data from periodic surveys of the Florida Keys and other select shallow water environments

Metadata also available as - [Outline] - [Parseable text] - [XML]

Frequently anticipated questions:


What does this data set describe?

Title:
SfM Quantitative Underwater Imaging Device with 5 cameras (SQUID-5) – Field data from periodic surveys of the Florida Keys and other select shallow water environments
Abstract:
The U.S. Geological Survey (USGS) Remote Sensing Coastal Change (RSCC) and Processes Impacting Seafloor Change and Ecosystem Services (PISCES) projects collect underwater imagery of coral reefs and other scientifically interesting, submerged environments using the novel SfM (Structure-from-Motion) Quantitative Underwater Imaging Device with 5 cameras (SQUID-5) system. This sensor collects imagery with optimized endlap/sidelap and precise position information to create high-resolution orthomosaics, three-dimensional (3D) point clouds, and digital elevation/surface models (DEMs/DSMs) using SfM photogrammetry methods. These products are valuable for measuring submerged topographic and ecological change, and for understanding reef vulnerability and response to disturbance events. This is an ongoing collection of underwater imagery surveys of coral reefs and other clear water environments. Shallow water (approximately 2-10 meters [m] depth) digital imagery is acquired using the "SfM Quantitative Underwater Imaging Device with 5 cameras" (SQUID-5) towed surface sensor system. The system consists of five synchronized rigidly connected downward-looking digital cameras with overlapping views of the seafloor, along with a custom integrated survey-grade Global Navigation Satellite System (GNSS) receiver.
Supplemental_Information:
The USGS RSCC simple data distribution service is designed to release data using a standardized workflow in support of timely best science while maintaining USGS Fundamental Science Practices (FSP) and meeting the high-quality data standards of the USGS. The data service includes big data (up to hundreds of thousands of digital images) and related derivative data products, and serves as a digital data repository for manual, semi-automated, and fully automated retrieval of published data. The data service is structured in a folder (directory) hierarchy with subfolders corresponding to individual data collection platforms (DCPs) or groups of DCPs, which in turn contain folders with products grouped by collection effort. For more information on the RSCC simple data distribution service, refer to Ritchie and others (2023), https://doi.org/10.5066/P9M3NYWI.
  1. How might this data set be cited?
    Kranenburg, Christine J., Hatcher, Gerald A., Warrick, Jonathan A., Ritchie, Andrew C., Zawada, David G., and Yates, Kimberly K., 20231130, SfM Quantitative Underwater Imaging Device with 5 cameras (SQUID-5) – Field data from periodic surveys of the Florida Keys and other select shallow water environments: data service DOI:10.5066/P9M3NYWI, U.S. Geological Survey – St. Petersburg Coastal and Marine Science Center, St. Petersburg, Florida.

    Online Links:

    Other_Citation_Details:
    Suggested Citation: Kranenburg, C.J., Hatcher, G.A., Warrick, J.A., Zawada, D.G., and Yates, K.K., 2023, SfM Quantitative Underwater Imaging Device with 5 cameras (SQUID-5) – Field data from periodic surveys of the Florida Keys and other select shallow water environments, in Remote Sensing Coastal Change Simple Data Distribution Service: U.S. Geological Survey data service, accessed [MMMM d, YYYY], at https://doi.org/10.5066/P9M3NYWI. [Data directly accessible at https://cmgds.marine.usgs.gov/data-services/rscc/SQUID/]
    This is part of the following larger work.

    Ritchie, Andrew C., Triezenberg, Peter J., Warrick, Jonathan A., Hatcher, Gerald A., and Buscombe, Daniel D., 20230221, RSCC Simple Data Distribution Service: data service DOI:10.5066/P9M3NYWI, U.S. Geological Survey - Pacific Coastal and Marine Science Center, Santa Cruz, California.

    Online Links:

  2. What geographic area does the data set cover?
    West_Bounding_Coordinate: -180
    East_Bounding_Coordinate: 180
    North_Bounding_Coordinate: 90
    South_Bounding_Coordinate: -90
  3. What does it look like?
  4. Does the data set describe conditions during a particular time period?
    Beginning_Date: 09-Jul-2019
    Ending_Date: Present
    Currentness_Reference:
    ground condition
  5. What is the general form of this data set?
    Geospatial_Data_Presentation_Form: application/service
  6. How does the data set represent geographic features?
    1. How are geographic features stored in the data set?
    2. What coordinate system is used to represent geographic features?
  7. How does the data set describe geographic features?
    Entity_and_Attribute_Overview:
    This metadata record describes characteristics of the SQUID-5 system data collection platform (DCP) and the organization and processing steps for data collected from that platform. The images are downloadable in Tag Image File Format (TIFF, .tif) and corresponds to a single 3-band, 24-bits per pixel, RGB image containing imagery header metadata locating the image in space and time and describing basic documentation (including but not limited to: camera model, resolution, and copyright information). Datasets are organized by directory with each directory (folder) corresponding to a single collection effort. Product types (raw imagery, derivatives, and any ancillary products) for each collection effort are organized in subfolders with associated metadata and are populated as data are processed. In some cases, multiple versions or formats of a product may be released, including emergency and provisional data subject to change. Process steps in the collection-level metadata for each product describe naming conventions, methods, and disclaimers. Directory structure for each survey uses the following naming convention: {State[-State]}_{YYYY[MM][DD]}_{AOIName}_{[Event]}, where {State[-State]} is the two-letter abbreviation for the state where the survey occurred, {YYYY[MM][DD]} is the year and, if necessary, the month and day of collection, {AOIName} is a unique name or acronym for the specific survey location (for example, EasternDryRocks or LooeKey, for FL Keys surveys), and {[Event]} denotes a named event for which the data was collected (for example, a named hurricane). Items in square brackets [] are optional.
    Entity_and_Attribute_Detail_Citation: U.S. Geological Survey

Who produced the data set?

  1. Who are the originators of the data set? (may include formal authors, digital compilers, and editors)
    • Christine J. Kranenburg
    • Gerald A. Hatcher
    • Jonathan A. Warrick
    • Andrew C. Ritchie
    • David G. Zawada
    • Kimberly K. Yates
  2. Who also contributed to the data set?
  3. To whom should users address questions about the data?
    U.S. Geological Survey, St. Petersburg Coastal and Marine Science Center
    SPCMSC Data Management Group
    600 4th St South
    St. Petersburg, FL
    USA

    727-502-8000 (voice)
    gs-spcmsc_data_inquiries@usgs.gov

Why was the data set created?

The underwater images and associated location data were collected to provide high-resolution elevation data and precisely co-registered, full-color orthomosaic basemaps for use in environmental assessment and monitoring of coral reefs and other underwater environments. Additionally, the data were collected to evaluate their potential to improve USGS scientific efforts including seafloor elevation and stability modeling, and small-scale hydrodynamic flow modeling.

How was the data set created?

  1. From what previous works were the data drawn?
  2. How were the data generated, processed, and modified?
    Date: Not complete (process 1 of 5)
    IMAGE AND POSITION ACQUISITION The USGS operates the SQUID-5 system to conduct underwater imagery surveys over shallow water environments with optimized endlap/overlap to facilitate SfM processing (Hatcher and others, 2020 and 2023). The SQUID-5 system is comprised of five synced 5-megapixel (MP) machine vision cameras attached to a rigid frame, along with a custom integrated survey-grade Global Navigation Satellite System (GNSS) receiver, which is towed behind a motorized support vessel. The cameras are triggered by an acquisition computer which is time synchronized using a local GNSS regulated Network Time Protocol (NTP) server. An event mark is subsequently recorded and timestamped by the GNSS for each camera trigger signal. The raw dataset is comprised of the raw imagery, raw dual-frequency carrier phase GNSS data and photo capture event time data. Individual image names conform to the following naming convention: CAM##-YYYYMMDDhhmmss_fff-n[nnn].tif, where ## comprise the last 2 digits of the serial number of the camera that captured the image, YYYYMMDD represents the year (YYYY), month (MM) and day (DD) of capture; hhmmss_fff is the time of capture as hours (hh), minutes (mm), seconds(ss) and fractional seconds (fff) in Universal Coordinated Time (UTC) and n[nnn] is a sequence number (image counter) which is reset at the beginning of each survey line. For example, CAM82-20210508215908_001-115.tif was collected by the camera whose serial number ends in 82 on 2021-05-08 at 21:59:08.001 UTC, and it was the 115th shot of that survey line. Images from the camera whose serial number ends in 01 come off the system named CAM1- YYYYMMDDhhmmss_fff-n[nnn].tif. Images from this camera and any future one that ends in 0#, where # is in the range [1-9]), is renamed using the script below by inserting the missing ‘0’ such that it conforms to the naming convention and becomes CAM01-*. #!/usr/bin/bash for i in CAM1-*.tif; do echo "$i" mv "$i" "`echo $i | sed "s/CAM1/CAM01/"`"; done GNSS positions record the location of the phase center of the GNSS antenna at the moment an image capture is initiated (see Positional_Accuracy metadata element for uncertainty discussion). Lever arm distances are used to translate the position from the GNSS antenna reference point to the nodal point of the camera lenses; these are provided relative to the coordinate systems of each camera, where the camera is 0,0, and the axes run as follows: X axis runs from left (negative) to right (positive) Y axis runs from bottom (negative) to top (positive) Z axis runs from front of lens (negative) to back of camera (positive) The SQUID system hardware is oftentimes disassembled for transportation and reassembled at the survey site. It is also updated as new technological advances become available, which may include new cameras and/or new GNSS system components. Although every effort is made to reassemble the system in exactly the same positions, both these events cause changes to the lever arm distances. The methodology by which the original lever arm measurements were determined is described in Hatcher and others (2020). Starting lever arm measurements of the SQUID system are listed below; these distances are then refined during the SfM process. Lever Arm Distances: Center camera: 0.034 m, 0.011 m, and 0.84 m (x, y, z) Forward camera: -0.010 m, -0.594 m, 0.762 m Rear camera: 0.131 m, 0.559 m, 0.754 m Left camera: 0.273 m, -0.109 m, 0.916 m Right camera: -0.311 m, -0.029 m, 0.911 m Estimated uncertainties in the lever arm distances are 5 cm in each direction for the outboard cameras and 2.5 cm in each direction for the nadir (center) camera. Equipment specifications of the different SQUID versions and the dates they were in use are listed below. VERSION 1.1 Survey Dates Valid: 2019-07 to 2019-12 Camera Model: Teledyne FLIR BlackFly BFS-PGE-50S5C-C Lens focal length: 6 mm GNSS Receiver: Trimble R7 GNSS Antenna: Trimble Zephyr 2 VERSION 1.2 Survey Dates Valid: 2020-01 to 2022-06 Camera Model: Teledyne FLIR BlackFly BFS-PGE-50S5C-C Lens focal lengths: Center cam 8 mm, outboard cams 6 mm GNSS Receiver: Trimble R7 GNSS Antenna: Trimble Zephyr 2 VERSION 2 Survey Dates Valid: 2022-07 to Present Camera Model: Teledyne FLIR BlackFly BFS-PGE-50S5C-C Lens focal lengths: Center cam 8 mm, outboard cams 6 mm GNSS Receiver: Spectra Geospatial SP90M GNSS Antenna: Spectra Geospatial SPP 135000.00 Data sources produced in this process:
    • raw imagery
    • raw GNSS
    • event marks
    Date: Not complete (process 2 of 5)
    IMAGE AND GNSS QA/QC Quality control of the raw imagery and raw GNSS data occurs in the field where the entire processing workflow is performed on the day of collection. A preliminary processing of the GNSS data is done against either the closest continuously operating reference station (CORS) or a locally-established base station, at 1 hertz (Hz), using either broadcast or ultra-rapid ephemerides and assessed by reviewing plots and quality metrics from NovAtel’s GrafNav software. The resulting image positions are then ingested into Agisoft Metashape along with the raw imagery where a medium quality point cloud and medium resolution DEM are produced. If any issues that preclude successful model building are detected during this field processing, the batch is discarded, and the data is re-collected. If an issue is found by the data processor that affects two or fewer cameras, a re-collect is not performed, but the images from those cameras will be discarded during final processing. Hatcher and others (2023) determined that the SQUID-5 has enough redundancy to build high quality high-resolution models with as few as 3 cameras operating. Data from the local base station is converted to Receiver Independent Exchange Format (RINEX) using a proprietary utility provided by the manufacturer of the GNSS receiver, down-sampled to a 10 or 30-second interval using TEQC (https://www.unavco.org/software/data-processing/teqc/teqc.html) if necessary to comply with file size limits, and submitted to the National Oceanic and Atmospheric Administration (NOAA) Online Positioning User Service (OPUS) website (https://geodesy.noaa.gov/OPUS/) for validation. This website computes the position of the uploaded data and ties it to the high-accuracy National Spatial Reference System (NSRS) coordinate system. Data sources used in this process:
    • raw imagery
    • raw GNSS
    • event marks
    Data sources produced in this process:
    • validated imagery
    • validated GNSS
    • preliminary picture positions
    Date: Not complete (process 3 of 5)
    KMZ CREATION In order to provide a visual map of the survey extent, image capture locations, to aid a user in determining which images they might want to download, and to aid in completing the geospatial information in the collection-level metadata, a Keyhole Markup Language Zipped file (.kmz) is created for each collection using Global Mapper to convert the image positions file to a kmz. Data sources used in this process:
    • preliminary picture positions
    Data sources produced in this process:
    • kmz
    Date: Not complete (process 4 of 5)
    GNSS FINAL DATA PROCESSING Raw GNSS data received by the antenna mounted atop the SQUID-5 are recorded at 10 Hz by a dual-frequency survey-grade GNSS receiver. This reprocessing of the GNSS data uses precise ephemerides (obtained through the GrafNav software from external sources), multiple base stations if possible, and is run at 10 Hz, to achieve higher accuracy image positions than was obtained during the field processing described in the ‘IMAGE AND GNSS QA/QC’ step above. If the collection-level metadata describes the dataset as PROVISIONAL, it means the image positions were computed using a lower accuracy ephemeris. This is sometimes necessary if the imagery is collected to rapidly assess damage from natural or unnatural (e.g. boat groundings) disasters. The trajectories, in combination with precisely recorded image capture event times, are used to generate GNSS antenna positions at the moment each image capture is initiated and is provided for each collection. Images with missing positions are reconciled through direct interpolation based on the image and GNSS time stamps. The positions in the image positions file represent the position of the SQUID-5 GNSS antenna, not the position of the camera or features photographed. To determine actual camera positions, photogrammetric software such as Agisoft Metashape can be used to apply the lever arm offsets (in the camera frame of reference) from the GNSS antenna reference point to the camera lenses. Data sources used in this process:
    • validated imagery
    • raw GNSS
    • event marks
    Data sources produced in this process:
    • final picture positions
    Date: Not complete (process 5 of 5)
    IMAGERY HEADERS Georeferencing, copyright, and other relevant information is added to the imagery headers of each image using Phil Harvey’s ExifTool. To extract the information from the image headers using ExifTool, the following command can be used: exiftool -n -csv *.tif > allheaders.csv The -csv flag writes the information out in a comma-delimited format. The -n option formats the latitude and longitude as signed decimal degrees. Data sources used in this process:
    • validated imagery
    • final picture positions
    Data sources produced in this process:
    • populated imagery
  3. What similar or related data should the user be aware of?
    Hatcher, Gerald A., Warrick, Jonathan A., Kranenburg, Christine J., and Ritchie, Andrew C., 20230726, Accurate Maps of Reef-scale Bathymetry with Synchronized Underwater Cameras and GNSS: Remote Sensing 15(15), 3727.

    Online Links:

    Hatcher, Gerald A., Warrick, Jonathan A., Ritchie, Andrew C., Dailey, Evan T., Zawada, David G., Kranenburg, Christine, and Yates, Kimberly K., 20200626, Accurate Bathymetric Maps From Underwater Digital Imagery Without Ground Control: Frontiers in Marine Science Volume 7, Article 525.

    Online Links:


How reliable are the data; what problems remain in the data set?

  1. How well have the observations been checked?
    The accuracy of the position data is based on the accuracy and precision of the GNSS equipment, camera timing, number and length of baselines, and environmental conditions at the time of collection. Note that the positions in the imagery headers are GNSS antenna positions and are limited to six decimal places. Whereas the positions in the external navigation file are accurate to ten decimal places and are therefore highly recommended for use in SfM photogrammetric processing to produce the highest quality derived products. These data represent raw imagery, as collected, and subsequently used in SfM photogrammetric processing workflows. The user may find it useful and/or necessary to color correct these raw data, as appropriate for their application.
  2. How accurate are the geographic locations?
    Horizontal positions are provided in the image locations files (.txt) and generally have an estimated 2-sigma horizontal accuracy of 10 centimeters (cm). The accuracy of the positions in the imagery headers are reduced due to field size limitations for storing latitude/longitude. Any variations in positional accuracy are described in the collection-level metadata record.
  3. How accurate are the heights or depths?
    Vertical positions are provided in the image locations files and generally have an estimated 2-sigma vertical accuracy of 15 cm. Any variations in positional accuracy are described in the collection-level metadata record.
  4. Where are the gaps in the data? What is missing?
    This is an ongoing collection of underwater imagery surveys of shallow water environments. Some images may lack position information in their imagery header metadata resulting from a momentary disruption in GNSS signals, but in most cases interpolated positions were applied to these images.
  5. How consistent are the relationships among the observations, including topology?
    RGB (red, green, blue) values are 8-bits per band, generally not saturated or underexposed. Occasionally, usually at the start of a new survey day, some imagery may be over or underexposed due to inappropriate initial operator-set gain settings. Appropriate gain settings are usually achieved by the 2nd or 3rd survey line. GNSS values are checked for consistency and outliers. Bad data are discarded, and missing data are interpolated.

How can someone get a copy of the data set?

Are there legal restrictions on access or use of the data?
Access_Constraints None
Use_Constraints USGS-authored or produced data and information are in the public domain from the U.S. Government and are freely redistributable with proper metadata and source attribution. Please recognize and acknowledge the U.S. Geological Survey as the originator(s) of the dataset and in products derived from these data.
  1. Who distributes the data set? (Distributor 1 of 1)
    SPCMSC Data Management Group
    U.S. Geological Survey, St. Petersburg Coastal and Marine Science Center
    600 4th St. South
    St. Petersburg, FL
    USA

    727-502-8000 (voice)
    gs-g-spcmsc_data_inquiries@usgs.gov
  2. What's the catalog number I need to order this data set? CAM##-YYYYMMDDhhmmss_fff-n[nnn].tif
  3. What legal disclaimers am I supposed to read?
    This publication was prepared by an agency of the United States Government. Although these data were processed successfully on a computer system at the U.S. Geological Survey, no warranty expressed or implied is made regarding the display or utility of the data on any other system, nor shall the act of distribution imply any such warranty. The U.S. Geological Survey shall not be held liable for improper or incorrect use of the data described and (or) contained herein. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof.
  4. How can I download or order the data?

Who wrote the metadata?

Dates:
Last modified: 29-Nov-2023
Metadata author:
SPCMSC Data Management Group
600 4th St. South
St. Petersburg, FL
USA

727-502-8000 (voice)
gs-g-spcmsc_data_inquiries@usgs.gov
Metadata standard:
Content Standard for Digital Geospatial Metadata (FGDC-STD-001-1998)

This page is <https://cmgds.marine.usgs.gov/catalog/pcmsc/DataServices/rscc/SQUID/SQUID5_DCP_metadata.faq.html>
Generated by mp version 2.9.51 on Fri Jan 10 13:32:13 2025