PCMSC PlaneCam – Field data from periodic and event-response surveys of the U.S. West Coast.

Metadata also available as - [Outline] - [Parseable text] - [XML]

Frequently anticipated questions:


What does this data set describe?

Title:
PCMSC PlaneCam – Field data from periodic and event-response surveys of the U.S. West Coast.
Abstract:
This is an ongoing collection of aerial oblique and near-nadir images, ancillary data, and derivatives, from aerial surveys of coastal and near-coastal environments with a crewed light aircraft using the "PCMSC PlaneCam," a mounted fixed-lens DSLR camera with an attached consumer-grade GPS for time-keeping and approximate position, and a Global Navigation Satellite System (GNSS) for precise positioning. Data are collected and produced primarily for coastal monitoring using structure-from-motion photogrammetry. The intent is to develop and implement survey methods to study coastal geomorphic processes at high temporal and spatial resolution, and to make data available as rapidly as possibly while maintaining U.S. Geological Survey (USGS) Fundamental Science Practices (FSP) in the spirit of producing timely best science. Data products from the PCMSC PlaneCam data collection platform (DCP) are published in a directory structure with one folder per survey ("collect"), and within that folder, one folder per product. A process step or steps in metadata for each product describe naming conventions and methodology for each product. A detailed description of naming conventions can be found in the Entity and Attribute Overview and Process Steps of this metadata record.
Supplemental_Information:
Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.
  1. How might this data set be cited?
    Ritchie, Andrew C., Warrick, Jonathan A., and Hatcher, Gerald A., 20230221, PCMSC PlaneCam – Field data from periodic and event-response surveys of the U.S. West Coast.: data service DOI:10.5066/P9M3NYWI, U.S. Geological Survey, Pacific Coastal and Marine Science Center, Santa Cruz, California.

    Online Links:

    This is part of the following larger work.

    Ritchie, Andrew C., Triezenberg, Peter J., Warrick, Jonathan A., Hatcher, Gerald A., and Buscombe, Daniel D., 20230221, RSCC Simple Data Distribution Service: data service DOI:10.5066/P9M3NYWI, U.S. Geological Survey - Pacific Coastal and Marine Science Center, Santa Cruz, California.

  2. What geographic area does the data set cover?
    West_Bounding_Coordinate: -180
    East_Bounding_Coordinate: 180
    North_Bounding_Coordinate: 90
    South_Bounding_Coordinate: -90
  3. What does it look like?
  4. Does the data set describe conditions during a particular time period?
    Beginning_Date: 09-Dec-2015
    Ending_Date: Present
    Currentness_Reference:
    ground condition
  5. What is the general form of this data set?
    Geospatial_Data_Presentation_Form: application/service
  6. How does the data set represent geographic features?
    1. How are geographic features stored in the data set?
    2. What coordinate system is used to represent geographic features?
  7. How does the data set describe geographic features?
    Entity_and_Attribute_Overview:
    This metadata record describes characteristics of an aerial data collection platform (DCP), and the organization and processing steps for data collected from that platform. Data are organized by directory with each directory (folder) corresponding to a single collection effort. Product types (raw imagery, derivatives, and any ancillary products) for each collection effort are organized in subfolders with associated metadata and are populated as data are processed. In some cases, multiple versions or formats of a product may be released, including emergency and provisional data subject to change. Process steps in metadata for each product describe naming conventions, methods, and disclaimers. Directory structure for each survey uses the following naming convention: {Region[-Region]}_{SurveyDescriptor}_{YYYYMMDD[-[YYYY][MM][DD]]}, where {Region[-Region]} is the two-digit abbreviation for the survey region (in alphabetical order) where the survey event occurred, and survey regions correspond to Washington, abbreviated as WA, Oregon, abbreviated as OR, or Northern/Central/Southern California, abbreviated as NoCa, CenCA, or SoCA (for example, NoCA-OR for a survey that covered part of Oregon and Northern California), {SurveyDescriptor} is a unique name for the survey type (for example “coastal”, or “CZU-fire”), and {YYYYMMDD[-[YYYY][MM][DD]]} is the abbreviation for the date[-s] the survey occurred (for example 20201104-5 would be the abbreviation for a survey on November 4th and 5th, 2020). GNSS positions record the phase center of the GNSS antenna at the moment an image is captured (see accuracy information for uncertainty discussion). Lever arms are used to translate the position from the nodal point of the camera lens to the GNSS antenna phase center. These lever arms are recorded relative to the camera coordinate system, where the camera is 0,0, and the axes run as follows, described from a position as if you are holding the camera normally to take a picture: X axis runs from left (negative) to right (positive) Y axis runs from bottom (negative) to top (positive) Z axis runs from front of lens (negative) to back of camera (positive) PCMSC PlaneCam hardware has been through several iterations during development, and lever arms for early flights were not measured. Lever arm measurements, dates, and recommended values are described below. Estimated uncertainty for all axes is 5 cm, estimated as a function of both angle and distance measurement precision. Oblique lever arm 1 (use for oblique surveys before November 2020) [measured on 8-31-2018 or earlier, in lab, JAW and ACR] X = -0.18 Y = +0.1524 Z = +0.67 Oblique lever arm 2 (use for oblique surveys after October 2020) [measured on 1-12-2021 at WVI by ACR] X = -0.135 Y = +0.08 Z = +0.64 vertical lever arm (use for vertical surveys) [measured 11-27-2020 at WVI by ACR] X = -0.075 Y = -0.54 Z = +0.66
    Entity_and_Attribute_Detail_Citation: U.S. Geological Survey

Who produced the data set?

  1. Who are the originators of the data set? (may include formal authors, digital compilers, and editors)
    • Andrew C. Ritchie
    • Jonathan A. Warrick
    • Gerald A. Hatcher
  2. Who also contributed to the data set?
  3. To whom should users address questions about the data?
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    PCMSC Science Data Coordinator
    2885 Mission Street
    Santa Cruz, CA

    831-427 -4747 (voice)
    pcmsc_data@usgs.gov

Why was the data set created?

These data are provided in an effort to record ground conditions at a high temporal and spatial resolution, and after high-magnitude disturbance events, to improve our understanding of earth surface processes (natural hazards) affecting coastal resources.

How was the data set created?

  1. From what previous works were the data drawn?
  2. How were the data generated, processed, and modified?
    Date: Not complete (process 1 of 8)
    Acquisition -- Each image is acquired in camera raw format. Camera is triggered by an intervalometer, usually integrated with a survey-grade GNSS and triggered using a pulse-per-second (PPS) signal from the GNSS, routed through a microcontroller to the camera remote shutter, and recording an event mark to the GNSS when the shutter is actuated, through the flash-sync port on the camera. A consumer-grade GPS is typically attached to the camera keeping camera time synced with UTC time and providing approximate coordinates for EXIF data. Data sources produced in this process:
    • raw imagery
    • raw GNSS data
    Date: Not complete (process 2 of 8)
    Image renaming -- Raw Images are renamed using Phil Harvey's EXIFTOOL (https://exiftool.org) with the following command: exiftool -r -d %Y%m%d%H%M%S "-testname<CAM${SerialNumber;s/\d{4}//}_${SubSecDateTimeOriginal}_${SubSecTimeOriginal}%-c.%e" -ext nef .\* files follow the following naming convention: {CAM###}_{YYYYMMDDHHMMSS_ss}.jpg, where {CAM###} is the last 3 digits of the camera serial number, preceded by the letters “cam”, and where {YYYYMMDDHHMMSS_ss} is the image acquisition time in {YearMonthDayHourMinuteSecond_hundredths} expressed in 24-hour time, as recorded by the camera’s internal clock and written to the SubSecondDateTime field in the image EXIF data. For example, CAM001_202009182311_50 would be the timestamp for an image with a SubSecondDateTime EXIF time/date stamp of September 18th, 2020 at 11:11.50 pm (see accuracy report for details on time accuracy).
    Date: Not complete (process 3 of 8)
    Image conversion -- renamed images are batch- converted from .camera raw format to a format compatible with photogrammetry processing software. Software versions, conversion date, and other parameters are recorded in individual image EXIF data.
    Date: Not complete (process 4 of 8)
    Image QA/QC -- original and converted images are counted, and the beginning and end of the survey are reviewed to cull extraneous images such as camera testing or inadvertently triggered images and place both raw and converted images in an "outtakes" folder for a given flight (these images might not be released and are not processed). A multipage contact sheet of images is created to allow quick review ensuring that (a) conversion was successful with no corrupted imagery, and (b) "extra" imagery not capturing the intended view is culled and placed in the "outtakes" folder. If corrupted converted images are found, imagery is re-converted. If extra images are found, raw and converted images are moved to the "outtakes" folder. This step is iterated until a collection of valid images remains. Note that in some cases, images of water may be retained if there are visible or potentially visible bathymetric features, aquatic vegetation, or objects in the scene that may be useful for future data applications.
    Date: Not complete (process 5 of 8)
    GNSS data processing -- when collected, GNSS data are post-processed to derive the precise position of ground control points, check points, and/or the GNSS antenna at the time of image acquisition. If necessary for rapid processing, approximate (PROVISIONAL) positions are produced from rapid ephemeris data. After precise ephemerides are available, GNSS data are post-processed to calculate GNSS trajectory, and precise positions are derived from station occupations or event marks recorded with precision timing, placed along the GNSS trajectory. Processing workflows, accuracy estimates, software and version are detailed in GNSS metadata. Camera and ground control positions are post-processed together with images, camera lever arm data and photogrammetric software to derive accurate camera position and pose.
    Date: Not complete (process 6 of 8)
    Imagery EXIF coordinates reconciliation -- Note that EXIF camera positions are not intended to be suitable accuracy for precise photogrammetric surveys, but they are used to populate the EXIF metadata field and locate the image in space relative to other images for reference purposes. GNSS/SfM position data published separately from images should be used for SfM reconstruction if available. Camera positions are not always recorded for individual images due to GNSS collection conditions, malfunctioning equipment, or other reasons. Depending on availability of position data, coordinates for image EXIF position data are recorded in the following order of preference, with the appropriate indicator written to the EXIF:GPSAreaInformation tag: 1) If the image position is recorded to the EXIF field with the attached consumer-grade GPS device, that position is used. (EXIF:GPSAreaInformation="camera-integrated GPS") 2) If a survey is missing only a few image positions, positions are interpolated or extrapolated from consumer-grade GPS positions using adjacent images and a time-averaged velocity and direction algorithm. (EXIF:GPSAreaInformation="interpolated" or (EXIF:GPSAreaInformation="extrapolated") 3) If consumer GPS data are missing but GNSS event marks are present, GNSS data are used to fill in the EXIF field. (EXIF:GPSAreaInformation="GNSS event mark") 4) If a survey is missing many or all image positions, and/or another flight with valid position data exists for the same area, 4D SfM positions are used to fill in the EXIF field. These positions may be processed at lower accuracy/resolution, or with fewer images, than final positions used in photogrammetry, and are intended only to approximately locate images in space. (EXIF:GPSAreaInformation="structure-from-motion") 5) If a survey does not have image positions, but synthetic (derived from sources such as lidar or other imagery and elevation data sources) or manually placed ground control points have been measured, the imagery may be processed with 3D or 4D SfM techniques using ground control points to derive EXIF camera positions. These positions may be processed at lower accuracy/resolution, or with fewer images, than final positions used in photogrammetry, and are intended only to approximately locate images in space. (EXIF:GPSAreaInformation="structure-from-motion") 6) If no other imagery exists or has not been processed for a survey area, or if imagery needs to be published without accurate SfM processing to support emergency response or timely best science, then an approximate location may be assigned to some or all images based on best estimate of location. This may include assigning a single location to all images to approximately locate them based on the geographic area of a survey. (EXIF:GPSAreaInformation="estimated approximate location")
    Date: Not complete (process 7 of 8)
    Photogrammetric Processing -- 3D or 4D structure-from-motion (SfM) processing, as described in Over and others (2021) and Warrick and others (2017), is used to derive camera positions for surveys missing position data, and to produce camera models, camera orientation information, point clouds, surface models, depth maps, and orthocorrected imagery. SfM products are released with metadata detailing processing steps and workflows.
    Date: Not complete (process 8 of 8)
    Machine Learning -- Machine learning techniques as described in Buscombe and others (2022) and Buscombe and Goldstein (2022) are used to build, train and evaluate models, and generate labeled data. Labeled and segmented data products are released with metadata detailing process steps and workflow.
  3. What similar or related data should the user be aware of?
    Over, Jin-Si R., Ritchie, Andrew C., Kranenburg, Christine J., Brown, Jenna A., Buscombe, Daniel D., Noble, Tom, Sherwood, Christopher R., Warrick, Jonathan A., and Wernette, Phillipe A., 2021, Processing coastal imagery with Agisoft Metashape Professional Edition, version 1.6—Structure from motion workflow documentation: U.S. Geological Survey, Reston, VA.

    Online Links:

    Warrick, Jonathan A., Ritchie, Andrew C., Adelman, Gabrielle, Adelman, Kenneth, and Limber, Patrick W., 20170101, New Techniques to Measure Cliff Change from Historical Oblique Aerial Photographs and Structure-from-Motion Photogrammetry: Journal of Coastal Research vol. 33, issue 1, Coastal Education and Research Foundation, n/a.

    Online Links:

    Other_Citation_Details: pp. 39-55
    Buscombe, D., Goldstein, E.B., Sherwood, C.R., Bodine, C., Brown, J.A., Favela, J., Fitzpatrick, S., Kranenburg, C.J., Over, J.R., Ritchie, A.C., Warrick, J.A., and Wernette, P., 20220304, Human‐in‐the‐Loop Segmentation of Earth Surface Imagery: Earth and Space Science vol. 9, issue 3, American Geophysical Union (AGU), n/a.

    Online Links:


How reliable are the data; what problems remain in the data set?

  1. How well have the observations been checked?
    No accuracy calculations are performed on raw data. Camera EXIF position data are generally accurate to within at least a few hundred meters, and time data are accurate to within a few minutes. Camera clock was UTC through 7 March 2018, and switched to PST starting 29 March 2018 (to simplify processing across UTC date changes). Some images are over-exposed, some images are under-exposed, and no effort is made to optimize exposure through post-processing, nor to eliminate imagery with acquisition errors. These data represent raw imagery as-collected and subsequently used in Structure-from-Motion (SfM) photogrammetric processing workflows. GNSS position accuracy is limited by Precise Point Positioning (PPP) techniques and is assumed to be about 10cm horizontal and 15cm vertical based on post-processing reports, multiple comparisons with ground control points, and residual error after alignment for photogrammetry. Accuracy reports from each PPP post-processing report are included with GNSS data when published for a survey.
  2. How accurate are the geographic locations?
    A formal accuracy assessment of the horizontal positional information in the dataset has not been conducted. Most images have EXIF stamps from a consumer-grade GPS, updated every few seconds. Position accuracy for these images is generally within about 200m. Some images may lack GPS position in EXIF data, but in most cases we apply interpolated positions or positions derived through onboard survey-grade GNSS or 4D structure-from-motion. High-precision positions are produced as a separate product for most flights during post-processing. Interpolated positions are a linear interpolation of XYZ coordinates between images with known coordinates.
  3. How accurate are the heights or depths?
    A formal accuracy assessment of the vertical positional information in the dataset has not been conducted. Most images have EXIF stamps from a consumer-grade GPS, updated every few seconds. Vertical accuracy for these images is not assessed for accuracy. Some images may lack GPS position in EXIF data but in most cases we apply interpolated positions or positions derived through onboard survey-grade GNSS or 4D structure-from-motion. High-precision positions are produced as a separate product for most flights during post-processing. Interpolated positions are simply a linear interpolation of XYZ coordinates between images with known coordinates.
  4. Where are the gaps in the data? What is missing?
    Data are considered complete for the information presented, as described in the abstract. Users are advised to read the metadata record carefully for additional details, and to examine EXIF data for individual flights and images if required to determine camera settings such as exposure mode and shutter speed.
  5. How consistent are the relationships among the observations, including topology?
    RGB values are 8-bit, generally not saturated or underexposed. In a few cases of high or low light conditions, some imagery may be under- or over-exposed. GNSS values are checked for consistency and outliers. Bad data are discarded and missing data are interpolated or replaced with calculated positions from 4D SfM (see Over and others 2021 for methods).

How can someone get a copy of the data set?

Are there legal restrictions on access or use of the data?
Access_Constraints None
Use_Constraints USGS-authored or produced data and information are in the public domain from the U.S. Government and are freely redistributable with proper metadata and source attribution. Please recognize and acknowledge the U.S. Geological Survey as the originator(s) of the dataset and in products derived from these data.
  1. Who distributes the data set? (Distributor 1 of 1)
    U.S. Geological Survey - CMGDS
    2885 Mission Street
    Santa Cruz, CA

    831-427-4747 (voice)
    pcmsc_data@usgs.gov
  2. What's the catalog number I need to order this data set? Downloadable data in JPEG format are available in subfolders. Each JPG file corresponds to a single 3-band 24-bit RGB image with EXIF data locating the image in space and time and describing acquisition parameters including but not limited to camera model, camera settings, lens information, image conversion parameters, and environmental conditions. Derivatives and ancillary products may be contained in additional subfolders.
  3. What legal disclaimers am I supposed to read?
    Unless otherwise stated, all data, metadata and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected. Although these data and associated metadata have been reviewed for accuracy and completeness and approved for release by the U.S. Geological Survey (USGS), no warranty expressed or implied is made regarding the display or utility of the data on any other system or for general or scientific purposes, nor shall the act of distribution constitute any such warranty.
  4. How can I download or order the data?

Who wrote the metadata?

Dates:
Last modified: 21-Feb-2023
Metadata author:
U.S. Geological Survey, Pacific Coastal and Marine Science Center
Attn: PCMSC Science Data Coordinator
2885 Mission Street
Santa Cruz, CA

831-427-4747 (voice)
pcmsc_data@usgs.gov
Metadata standard:
Content Standard for Digital Geospatial Metadata (FGDC-STD-001-1998)

This page is <https://cmgds.marine.usgs.gov/catalog/pcmsc/DataServices/rscc/pcmsc_planecam/pcmsc_planecam/pcmsc_planecam_metadata.faq.html>
Generated by mp version 2.9.51 on Tue Sep 24 13:12:41 2024