Unprocessed aerial imagery from 13 October 2018 coastal survey of Northern California to Washington.

Metadata also available as - [Outline] - [Parseable text] - [XML]

Frequently anticipated questions:


What does this data set describe?

Title:
Unprocessed aerial imagery from 13 October 2018 coastal survey of Northern California to Washington.
Abstract:
This is a set of 11805 oblique aerial photogrammetric images and their derivatives, collected from OR-WA border to Mussel Rock CA with a fixed-lens digital camera from a crewed light aircraft, for processing using structure-from-motion photogrammetry and machine learning to study coastal geomorphic processes at high temporal and spatial resolution. JPG files in each folder follow the following naming convention: {CAM###}_{YYYYMMDDHHMMSS_ss}.jpg, where {CAM###} is the last 3 digits of the camera serial number, preceded by the letters "cam", and where {YYYYMMDDHHMMSS_ss} is the image acquisition time in {YearMonthDayHourMinuteSecond_hundredths} expressed in 24-hour time, as recorded by the camera's internal clock and written to the SubSecondDateTime field in the image EXIF data (for example CAM001_202009182311_50 would be the timestamp for an image with a SubSecondDateTime EXIF time/date stamp of September 18th, 2020 at 11:11.50 pm.
Supplemental_Information:
Additional information about the field activities from which these data were derived is available online at: https://cmgds.marine.usgs.gov/fan_info.php?fan=2018-698-FA Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.
  1. How might this data set be cited?
    Ritchie, Andrew C., Hatcher, Gerald A., and Warrick, Jonathan A., 20230221, Unprocessed aerial imagery from 13 October 2018 coastal survey of Northern California to Washington.: data service DOI:10.5066/P9M3NYWI, U.S. Geological Survey, Pacific Coastal and Marine Science Center, Santa Cruz, California.

    Online Links:

    This is part of the following larger work.

    Ritchie, Andrew C., Triezenberg, Peter J., Warrick, Jonathan A., Hatcher, Gerald A., and Buscombe, Daniel D., 20230221, RSCC Simple Data Distribution Service: data service DOI:10.5066/P9M3NYWI, U.S. Geological Survey - Pacific Coastal and Marine Science Center, Santa Cruz, California.

    Online Links:

  2. What geographic area does the data set cover?
    West_Bounding_Coordinate: -124.579112
    East_Bounding_Coordinate: -123.825943
    North_Bounding_Coordinate: 46.223168
    South_Bounding_Coordinate: 39.710378
  3. What does it look like?
  4. Does the data set describe conditions during a particular time period?
    Beginning_Date: 13-Oct-2018
    Beginning_Time: 125133
    Ending_Date: 13-Oct-2018
    Ending_Time: 162653
    Currentness_Reference:
    ground condition
  5. What is the general form of this data set?
    Geospatial_Data_Presentation_Form: raster image format
  6. How does the data set represent geographic features?
    1. How are geographic features stored in the data set?
    2. What coordinate system is used to represent geographic features?
      Horizontal positions are specified in geographic coordinates, that is, latitude and longitude. Latitudes are given to the nearest 0.000001. Longitudes are given to the nearest 0.000001. Latitude and longitude values are specified in Decimal degrees. The horizontal datum used is D_WGS_1984.
      The ellipsoid used is WGS_1984.
      The semi-major axis of the ellipsoid used is 6378137.0.
      The flattening of the ellipsoid used is 1/298.257224.
      Vertical_Coordinate_System_Definition:
      Altitude_System_Definition:
      Altitude_Datum_Name: WGS84 Ellipsoid
      Altitude_Resolution: 0.01
      Altitude_Distance_Units: meters
      Altitude_Encoding_Method:
      Explicit elevation coordinate included with horizontal coordinates
  7. How does the data set describe geographic features?
    Entity_and_Attribute_Overview:
    There are no additional attributes besides the embedded EXIF tags associated with these images
    Entity_and_Attribute_Detail_Citation: USGS Field Activity 2018-698-FA

Who produced the data set?

  1. Who are the originators of the data set? (may include formal authors, digital compilers, and editors)
    • Andrew C. Ritchie
    • Gerald A. Hatcher
    • Jonathan A. Warrick
  2. Who also contributed to the data set?
  3. To whom should users address questions about the data?
    U.S. Geological Survey, Pacific Coastal and Marine Science Center
    Attn: PCMSC Science Data Coordinator
    2885 Mission Street
    Santa Cruz, CA

    831-427-4747 (voice)
    pcmsc_data@usgs.gov

Why was the data set created?

Recording ground condition at a high temporal and spatial resolution, and after high-magnitude disturbance events, for processing using structure-from-motion photogrammetry and machine learning to study coastal geomorphic processes at high temporal and spatial resolution and to improve our understanding of earth surface processes (natural hazards) affecting coastal resources.

How was the data set created?

  1. From what previous works were the data drawn?
  2. How were the data generated, processed, and modified?
    Date: 13-Oct-2018 (process 1 of 8)
    Acquisition -- Aerial imagery was collected using a DOI-contracted aircraft operated by EcoScan Resource Data LLC and piloted by Bob Van Wagenen. Integrated GNSS and Camera hardware designed and built by the U.S. Geological Survey, Pacific Coastal and Marine Science Center and mounted inside the aircraft was oriented to capture [orientation] imagery through a port in a cargo door. Images were acquired in camera raw format. The camera is triggered by an intervalometer integrated with a survey-grade GNSS and triggered using a PPS signal from the GNSS, routed through a microcontroller to the camera remote shutter, and recording an event mark to the GNSS when the shutter is actuated, through the flash-sync port on the camera. A consumer-grade GPS attached to the camera keeps camera time synced with UTC time and provides approximate coordinates for EXIF data. Data sources produced in this process:
    • raw imagery
    • raw GNSS data
    Date: 2018 (process 2 of 8)
    Image renaming -- Raw Images are renamed using Phil Harvey's EXIFTOOL with the following command: exiftool -r -d %Y%m%d%H%M%S "-testname<CAM${SerialNumber;s/\d{4}//}_${SubSecDateTimeOriginal}_${SubSecTimeOriginal}%-c.%e" -ext nef .\* files follow the following naming convention: {CAM###}_{YYYYMMDDHHMMSS_ss}.jpg, where {CAM###} is the last 3 digits of the camera serial number, preceded by the letters "cam", and where {YYYYMMDDHHMMSS_ss} is the image acquisition time in {YearMonthDayHourMinuteSecond_hundredths} expressed in 24-hour time, as recorded by the camera's internal clock and written to the SubSecondDateTime field in the image EXIF data (for example CAM001_202009182311_50 would be the timestamp for an image with a SubSecondDateTime EXIF time/date stamp of September 18th, 2020 at 11:11.50 pm (See accuracy report for details on time accuracy)
    Date: 2018 (process 3 of 8)
    Image conversion -- renamed images are batch- converted from .camera raw format to a format compatible with photogrammetry processing software. Software versions, conversion date, and other parameters are recorded in individual image EXIF data.
    Date: 2018 (process 4 of 8)
    Image QA/QC -- original and converted images are counted, and the beginning and end of the survey are reviewed to cull extraneous images such as camera testing or inadvertently triggered images and place both raw and converted images in an "outtakes" folder for a given flight (outtakes might not be released and are not processed). A multipage tiff contact sheet of images is created to allow quick review ensuring that (a) conversion was successful with no corrupted imagery, and (b) "extra" imagery not capturing the intended view is culled and placed in the "outtakes" folder. If corrupted converted images are found, imagery is re-converted. If extra images are found, raw and converted images are moved to the "outtakes" folder. This step is iterated until a collection of valid images remains. Note that in some cases, images of water may be retained if there are visible or potentially visible bathymetric features, aquatic vegetation, or objects in the scene that may be useful for future data applications. The multipage contact sheet is generated with the open-source image processing system GraphicsMagick using the command: gm.exe montage -compress JPEG -quality 85 -sampling-factor 4:4:4 -font Arial-Narrow -pointsize 10 -tile 9x7 -label %t -geometry "135x90+1+0>" <ImageFolderPath>*.<ext> -define tiff:bits-per-sample=8 <SurveyFolderName>_thumbnail_contact_sheet.tif where <ImageFolderPath> is the path to the directory of images, <ext> is the image extension, and <SurveyFolderName> is the unique name of the survey folder. The GraphicsMagick version is recorded to the multipage contact sheet in the tiff metadata under the TIFFTAG_SOFTWARE tag. A csv file of a subset of image EXIF tags is then generated with the following exiftool command: exiftool -Model -LensID -ExposureProgram -FocalLength -ExposureTime -FNumber -ISO -SubSecDateTimeOriginal -DateTimeOriginal -GPSDateTime -GPSLatitude# -GPSLongitude# -GPSAltitude# -GPSPosition# -GPSAltitudeRef -GPSSatellites -CreatorTool -CameraProfile -SerialNumber -ImageNumber -f -csv -ext <ext> <SurveyFolderName>_EXIF.csv where <ext> is the file extension of the converted images used for photogrammetry and <SurveyFolderName> is the name of the folder for a specific survey. The csv file is processed with a custom python script to add camera positions from GNSS- or SfM-derived position files, or to interpolate or extrapolate positions, depending on available data.
    Date: 2018 (process 5 of 8)
    GNSS data processing -- when collected, GNSS data are post-processed to derive the precise position of ground control points, check points, and/or the GNSS antenna at the time of image acquisition. If necessary for rapid processing, approximate (provisional) positions are produced from rapid ephemeris data. After precise ephemerides are available, GNSS data are post-processed to calculate GNSS trajectory, and precise positions are derived from station occupations or event marks recorded with precision timing, placed along the GNSS trajectory. Processing workflows, accuracy estimates, software and version are detailed in GNSS metadata. Camera and ground control positions are post-processed together with images, camera lever arm data and photogrammetric software to derive accurate camera position and pose.
    Date: 2018 (process 6 of 8)
    Photogrammetric Processing -- 3D or 4D structure-from-motion (SfM) processing, as described in Over and others (2021) and Warrick and others (2017), is used to derive camera positions for surveys missing position data if necessary. This processing is done with reduced accuracy to provide approximate location only.
    Date: 2018 (process 7 of 8)
    Imagery EXIF coordinates reconciliation -- EXIF camera positions are not intended to be suitable accuracy for precise photogrammetric surveys but are used to populate the EXIF metadata field and locate the image in space relative to other images for reference purposes. GNSS/SfM position data published separately from images should be used for SfM reconstruction if available. Camera positions are not always recorded for individual images due to GNSS collection conditions, malfunctioning equipment, or other reasons. Depending on availability of position data, coordinates for image EXIF position data are recorded to imagery with one of the following methods, with the appropriate indicator written to the EXIF:GPSAreaInformation tag: 1) Image positions recorded to the EXIF field with the attached consumer-grade GPS device are labeled with the EXIF tag EXIF:GPSAreaInformation="camera-integrated GPS". 2) Image positions recorded to the EXIF field from timing-based GNSS event marks are labeled with the EXIF tag EXIF:GPSAreaInformation="GNSS event mark". 3) Image positions derived from 3D or 4D SfM processing with or without ground control points are labeled with the EXIF tag EXIF:GPSAreaInformation="structure-from-motion". 4) Image positions interpolated or extrapolated from between GPS, GNSS, or SfM-derived positions using adjacent image positions and a time-averaged velocity and direction algorithm are labeled with the EXIF tag EXIF:GPSAreaInformation="interpolated" or (EXIF:GPSAreaInformation="extrapolated". 5) If imagery needs to be published without more precise positioning to support emergency response or timely best science, or if no other way exists to locate images more accurately, then an approximate location is assigned to some or all images based on best estimate of location. This may include assigning a single location to all images to approximately locate them based on the geographic area of a survey. Any images located with this method are labeled with the EXIF tag EXIF:GPSAreaInformation="estimated approximate location"
    Date: 2018 (process 8 of 8)
    Image EXIF annotation -- Additional information was added to individual image EXIF tags using the command-line 'exiftool' software with the following command, as appropriate: #tags for provisional data exiftool ^ -P ^ -IPTC:Credit="U.S. Geological Survey" ^ -IPTC:Contact="pcmsc_data@usgs.gov" ^ -EXIF:Copyright="Public Domain" ^ -XMP:UsageTerms="These data are preliminary or provisional and are subject to revision. They are being provided to meet the need for timely best science. The data have not received final approval by the U.S. Geological Survey (USGS) and are provided on the condition that neither the USGS nor the U.S. Government shall be held liable for any damages resulting from the authorized or unauthorized use of the data." ^ -overwrite_original *.JPG #tags for approved data exiftool ^ -P ^ -IPTC:Credit="U.S. Geological Survey" ^ -IPTC:Contact="pcmsc_data@usgs.gov" ^ -EXIF:Copyright="Public Domain" ^ -XMP:UsageTerms="Unless otherwise stated, all data, metadata and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected. Although these data and associated metadata have been reviewed for accuracy and completeness and approved for release by the U.S. Geological Survey (USGS), no warranty expressed or implied is made regarding the display or utility of the data for other purposes, nor on all computer systems, nor shall the act of distribution constitute any such warranty." ^ -EXIF:ImageDescription="[single_image_EXIF_desc]" ^ -XMP:AttributionURL="https://doi.org/10.5066/P9M3NYWI ^ -EXIF:GPSAreaInformation="[position_method]" ^ -EXIF:GPSMapDatum="EPSG:4326 (WGS 84)" ^ -overwrite_original *.JPG tags were then duplicated to XMP headers to increase tag compatibility for image viewing software with the following exiftool command #tag duplication per phil harvey exiftool ^ -P ^ "-XMP-photoshop:Credit<IPTC:Credit" ^ "-XMP-iptcCore:CreatorWorkEmail<IPTC:Contact" ^ "-XMPdc:Rights<EXIF:Copyright" ^ "-XMP-dc:Description<EXIF:ImageDescription" ^ "-XMP-exif:all<GPS:all" ^ "-XMPexif:GPSLatitude<Composite:GPSLatitude" ^ "-XMP-exif:GPSLongitude<Composite:GPSLongitude" ^ "-XMPexif:GPSDateTime<Composite:GPSDateTime" ^ "-XMP-photoshop:DateCreated<EXIF:DateTimeOriginal" ^ "-XMPxmp:ModifyDate<EXIF:ModifyDate" ^ "-XMP-dc:Creator<EXIF:Artist" ^ "-XMP-tiff:Make<EXIF:Make" ^ "-XMPtiff:Model<EXIF:Model" ^ -overwrite_original *.JPG
  3. What similar or related data should the user be aware of?
    Over, Jin-Si R., Ritchie, Andrew C., Kranenburg, Christine J., Brown, Jenna A., Buscombe, Daniel D., Noble, Tom, Sherwood, Christopher R., Warrick, Jonathan A., and Wernette, Phillipe A., 2021, Processing coastal imagery with Agisoft Metashape Professional Edition, version 1.6-Structure from motion workflow documentation.

    Online Links:

    Warrick, Jonathan A., Ritchie, Andrew C., Adelman, Gabrielle, Adelman, Kenneth, and Limber, Patrick W., 20170101, New Techniques to Measure Cliff Change from Historical Oblique Aerial Photographs and Structure-from-Motion Photogrammetry.

    Online Links:


How reliable are the data; what problems remain in the data set?

  1. How well have the observations been checked?
    Camera EXIF position data are generally accurate to within a few hundred meters, and time data are accurate to within a few minutes. Camera clock was UTC through 7 March 2018, and switched to PST starting 29 March 2018 (to simplify processing across UTC date changes).
  2. How accurate are the geographic locations?
    A formal accuracy assessment of the horizontal positional information in the image data set has not been conducted. Most images have EXIF stamps from a consumer-grade GPS, updated every few seconds. Position accuracy for these images is generally within about 200m. Some images may lack GPS position in EXIF data, but in most cases, we apply interpolated positions or positions derived through onboard survey-grade GNSS or 4D structure-from-motion. High-precision positions, when available, are published as a flight derivative product with separate metadata describing their derivation and accuracy.
  3. How accurate are the heights or depths?
  4. Where are the gaps in the data? What is missing?
    Data are considered complete for the information presented, as described in the abstract. Users are advised to read the rest of the metadata record carefully for additional details, and to examine EXIF data for individual flights to determine camera settings such as exposure mode and shutter speed.
  5. How consistent are the relationships among the observations, including topology?
    RGB values are 8-bit, generally not saturated or underexposed. In a few cases of high or low light conditions, some imagery may be under- or over-exposed.

How can someone get a copy of the data set?

Are there legal restrictions on access or use of the data?
Access_Constraints None
Use_Constraints USGS-authored or produced data and information are in the public domain from the U.S. Government and are freely redistributable with proper metadata and source attribution. Please recognize and acknowledge the U.S. Geological Survey as the originator(s) of the dataset and in products derived from these data.
  1. Who distributes the data set? (Distributor 1 of 1)
    U.S. Geological Survey - CMGDS
    2885 Mission Street
    Santa Cruz, CA

    831-427-4747 (voice)
    pcmsc_data@usgs.gov
  2. What's the catalog number I need to order this data set? Downloadable data in JPEG format. Each JPG file corresponds to a single 3-band 24-bit RGB image with EXIF data locating the image in space and time and describing acquisition parameters including but not limited to camera model, camera settings, lens information, image conversion parameters, and environmental conditions.
  3. What legal disclaimers am I supposed to read?
    Unless otherwise stated, all data, metadata and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected. Although these data and associated metadata have been reviewed for accuracy and completeness and approved for release by the U.S. Geological Survey (USGS), no warranty expressed or implied is made regarding the display or utility of the data on any other system or for general or scientific purposes, nor shall the act of distribution constitute any such warranty.
  4. How can I download or order the data?
  5. What hardware or software do I need in order to use the data set?
    Downloadable data files are available in a standard image file format. Image processing and viewing software can be used to open and view the files.

Who wrote the metadata?

Dates:
Last modified: 21-Feb-2023
Metadata author:
U.S. Geological Survey, Pacific Coastal and Marine Science Center
Attn: PCMSC Science Data Coordinator
2885 Mission Street
Santa Cruz, CA

831-427-4747 (voice)
pcmsc_data@usgs.gov
Metadata standard:
Content Standard for Digital Geospatial Metadata (FGDC-STD-001-1998)

This page is <https://cmgds.marine.usgs.gov/catalog/pcmsc/DataServices/rscc/pcmsc_planecam/NoCA-OR-WA_coastal_20181013/NoCA-OR-WA_coastal_20181013_metadata.faq.html>
Generated by mp version 2.9.51 on Tue Sep 24 13:12:41 2024