Sonobuoy Seismic and Navigation Data Collected Using Sercel GI Guns and Ultra Electronics Seismic Sonobuoys During the Mid-Atlantic Resource Imaging Experiment (MATRIX), USGS Field Activity 2018-002-FA

PROCESS STEP 1:
Shearwater Reveal (version 4.1) seismic processing software was used to execute the following processing flows to produce MCS layback shot navigation.
1. Import SEG-D sequences: SegDRead read raw Geometrics SEG-D shot sequence files, extracted navigation fixes from the external headers, and wrote them to new header words. HeaderMath converted the source lat/lon positions from seconds of arc to decimal degrees (NRP_LAT, NRP_LON). UTMLatLong projected the geographic navigation to UTM Zone 18N WGS 84 meters (NRP_X, and NRP_Y). Output wrote the trace sequence to Reveal formatted ".seis" files.
2. Layback geometry assignment: Input read sequence files sorted by FFID/CHANNEL. The Python module ShotlineLayback (developed by Nathan Miller of USGS-WHCMSC) defined the source and streamer geometry based on measured horizontal offsets from the NRP to the center of the sources (cos) and the centers of the first and last 6.25 m spaced channel groups, and when necessary, the first and last 12.5 m spaced channel groups on the active streamer section. The following table lists the linear offsets from the NRP to center of source (COS) and index channels (IC) used to define layback geometry by line.
NRP to COS offset,NRP to IC offset(IC#) - Linelist -47.8,-174.6(1)/-922.69(120)/-932.38(121)/-1320.81(152) - Line MX01 -47.8,-176.6(1)/-924.69(120)/-934.38(121)/-1322.81(152) - Lines MX02 and MX03 -47.8,-176.6(1)/-874.38(112) - Lines MX04 - MX10 -47.8,-176.6(1)/-975(128)/-984.69(129)/-1373.12(160) - Lines MX11 - MX17 -47.8,-176.6(1)/-975(128)/-984.69(129)/-1272.56(152) - Lines MX18 and MX19
The module interpolated a sail line from the shot NRP positions (NRP_X and NRP_Y), then computed layback positions for the source and channel groups (values were interpolated for all channels in between the defined index channels) for each FFID by translating them back along the sail line by their respective offsets. Layback midpoint positions along the sail line were also computed for each shot/receiver pair. HeaderMath computed and populated a new header word for trace offset using OFFSET = ABS(NRP2CHAN) - ABS(NRP2COS). Output wrote the trace sequence with defined geometry to new files.
4. Geometry modification for tailbuoy navigation: Tailbuoy GPS navigation acquired during lines MX01 to MX07 and MX11 to MX19 were used to adjust the source and streamer geometry to incorporate trigonometrically estimated streamer feather into the CMP binning process. Input read the layback geometry files. Text files containing tailbuoy positions and UTC times extracted from HYPACK raw files were used as inputs for DBmerge, which matched times in the trace headers to times in the tailbuoy navigation file (through interpolation) and inserted tailbuoy positions into the trace headers (TB_LON and TB_LAT). UTMLatLong projected the geographic tailbuoy positions to WGS84 UTM Zone 18N meters (TB_X and TB_Y). HeaderMath estimated the streamer feather azimuth relative to the NRP using CABLE_AZ_RAD = atan2(NRP_Y - TB_Y, NRP_X - TB_X), then calculated new layback and feather adjusted source and receiver positions using SRC_X = NRP_X + NRP2COS * cos(CABLE_AZ_RAD), SRC_Y = NRP_Y + NRP2COS * sin(CABLE_AZ_RAD), REC_X = NRP_X + NRP2CHAN * cos(CABLE_AZ_RAD), REC_Y = NRP_Y + NRP2CHAN * sin(CABLE_AZ_RAD). DBWrite wrote the feather adjusted source positions (in UTM 18N WGS84 meters and geographic decimal degrees), FFID number, year, day number, and UTC times for the shots in each line to ASCII CSV files.
These process steps and all subsequent process steps were conducted by the same person - Wayne Baldwin.

PROCESS STEP 2:
The following processing flow was executed within a python notebook (FA2018002_SonobuoyMiniseed.ipynb) on raw sonobuoy traces and navigation by deployment.
1. Obspy (version 1.2.2) was used to read all raw miniSEED files contributing to a deployment into a merged three trace stream. Interactive plotting was used to identify the trace containing data for the deployment and determine the date and times of the hydrophone deployment and the first and last recorded shots. A position for the hydrophone deployment was manually retrieved from the Hemisphere GPS message in the HYPACK raw file corresponding to the determined time. During this process a row for each sonobuoy deployment containing columns for its coordinates, name, year, UTC day and time, image name, SurveyID, VehicleID, and DeviceID were entered into the ASCII CSV file '2018-002-FA_SB_depnav.csv'.
2. Python was used to parse HYPACK raw files for the FEI clock shot-instant messages within the first and last shot temporal limits and append them into a Pandas (version 0.25.1) dataframe containing columns for year, Julian day, hour, minute, second, microsecond, and Pandas datetime object. Similarly, the CSV version of the final MCS shot navigation ('2018-002-FA_MCS_shtnav.csv', available online at https://doi.org/10.5066/P91WP1RZ) was parsed to produce a dataframe with the same temporal limits containing columns for deployment longitude and latitude (static coordinates determined manually in previous operation), shot longitude and latitude, FFID, offset, year, Julian day, hour, minute, second, and Pandas datetime object. Offset is the distance in meters between the static deployment position and each shot position as calculated by pyproj (version 1.9.6) on the WGS84 ellipsoid. The offsets are nominal estimates and do not account for post-deployment drift of the sonobuoy.
3. The dataframes produced in the previous operation were merged, using the Pandas merge "asof" module with the datetime object as the join key, to produce a dataframe retaining the deployment longitude and latitude, shot longitude and latitude, FFID, and offset columns from the MCS shot navigation input and the year, Julian day, hour, minute, second, microsecond, and Pandas datetime object columns from the FEI clock shot-instant input. Generally, the "nearest" direction argument was specified during the "asof" merge to simply match the rows of each dataframe containing the closest datetime objects. However, problems with the network time protocol sever software on the Geometrics acquisition computer during deployments SB1 - SB10 resulted in MCS SEG-D header times up to 7 seconds behind the FEI clock shot-instant times. In these instances, the "backwards" direction argument was specified during the "asof" merge in order to match rows so that those with earlier datetime objects in the MCS shot navigation dataframe were matched with those containing later datetime objects in the FEI clock shot-instant dataframe. New columns containing the sonobuoy name, associated MCS line name, SurveyID, VehicleID, and DeviceID were added to the resultant merged shot time dataframe and it was exported to an ASCII CSV file. Upon completing this processing step for all deployments, the shot time ASCII CSV files were concatenated into a single file, '2018-002-FA_SB_shtnav.csv', and a header line was inserted.
4. The merged shot time dataframe was used along with the Obspy trace slice module to cut the continuously recorded sonobuoy record into an Obspy stream containing individual traces for each recorded shot. Iterating over the dataframe rows, Obspy was used to slice 15 second traces from the input record starting at each shot instant, create and populate a list of user defined SEG-Y header words (trace sequence number, ffid, trace id, trace within ffid, coordinate scalar, source x and y coordinates (shot positions in arcseconds), group x and y coordinates (static deployment position in arcseconds), offset (nominal distance between source and sonobuoy), coordinate units (2 = arcseconds), year, day, hour, minute, second, and time basis code (4 = UTC)), and write the resultant trace stream to a SEG-Y Rev. 1 formatted file (32-Bit IBM floating point).

PROCESS STEP 3:
Shearwater Reveal was used to execute the following processing flow for sonobuoy records.
1. Import read the SEG-Y traces, Detrend determined and removed any mean amplitude trend from the traces, Despike reduced large amplitude noise spikes, BandpassFilter applied a high pass filter to retain frequencies greater than 8 Hz, and Output wrote the processed traces to a new SEG-Y Rev. 1 formatted file (32-Bit IBM floating point). Each output SEG-Y file contains a textural file header similar to the example from sonobuoy SB01 included below.
Example sonobuoy file SEG-Y Textural Header:
C 1 U.S. GEOLOGICAL SURVEY COASTAL AND MARINE HAZARDS AND RESOURCES PROGRAM
C 2 SURVEY_ID: 2018-002-FA AREA: US ATLANTIC MARGIN VESSEL: R/V HUGH R. SHARP
C 3 YEAR: 2018 SONOBUOY NAME: SB01 MCS LINE NAME: MX01
C 4
C 5 ACQUISITION: UP TO 4 x 105 CU IN AIR GUNS, 30 METER SHOT INTERVAL,
C 6 ULTRA ELECTRONICS AN/SSQ-53D(3) SONOBUOY, 1 MS RECORDING SAMPLE INTERVAL,
C 7 CONTINUOUSLY RECORDED TO IRIS MINISEED FORMAT. SOURCE CHANGES THROUGHOUT
C 8 WHILE TROUBLESHOOTING.
C 9
C10
C11 PROCESSING: IMPORT AND CONCATENATE RAW MINISEED RECORDS, COMBINE
C12 MCS SHOT NAVIGATION (SEE HTTPS://DOI.ORG/10.5066/P91WP1RZ) WITH
C13 MICROSECOND RESOLUTION SHOT INSTANT TIMING FOR TEMPORAL RANGE OF INPUT
C14 RECORD, SLICE 15 SECOND TRACES FROM THE INPUT AT EACH SHOT INSTANT,
C15 COMPUTE NOMINAL OFFSET BETWEEN SHOT AND SONOBUOY DEPLOYMENT POSITIONS (IN
C16 METERS), CONCATENATE TRACES TO OUTPUT SEG-Y FILE WITH HEADER INFORMATION,
C17 DESPIKE TO REDUCE LARGE AMPLITUDE NOISE BURSTS, AND HIGH PASS FILTER
C18 (> 8 Hz).
C19
C20
C21 OUTPUT: 32-BIT IBM FLOATING POINT SEG-Y
C22 RECORD LENGTH: 15 SEC
C23
C24 MCS SHOT (SRC) AND SONOBUOY DEPLOYMENT (GRP) COORDINATES ARE STORED IN
C25 GEOGRAPHIC ARCSECONDS (SCALED BY 100). DIVIDE BY 360000 FOR DECIMAL
C26 DEGREES. OFFSET BETWEEN SRC AND REC IN METERS IS NOMINAL
C27
C28 SRC-X AND SRC-Y IN BYTES 73-76 AND 77-80
C29 GRP-X AND GRP-Y IN BYTES 81-84 AND 85-88
C30 COORDINATE SCALAR IN BYTES 71-72
C31 OFFSET IN BYTES 37-40
C32
C33
C34
C35
C36 FOR ADDITIONAL INFORMATION CONCERNING THIS DATASET REFER TO THE ASSOCIATED
C37 USGS SCIENCEBASE DATA RELEASE ONLINE AT:
C38 HTTPS://DOI.ORG/10.5066/P97LBUSP
C39
C40

PROCESS STEP 4:
The following flow was executed within a python notebook (FA2018002_SonobuoySQLdb.ipynb) to import sonobuoy deployment and shot data into a SpatiaLite (version 4.3.0) enabled SQLite (version 3.3.0) database.
1. Pandas 'read_csv' imported data from the files '2018-002-FA_SB_depnav.csv' and '2018-002-FA_SB_shtnav.csv' into new dataframes containing the deployment and shot information, respectively. Pandas 'to_sql' created and populated SQL database tables from each dataframe.
2. SQLite operations created ('AddGeometryColumn') and populated ('Update') point geometry columns in each of the new tables from the deployment and shot coordinates, respectively. Additionally, a new database table was created ('Create Table' and 'AddGeometryColumn') with a line string geometry column to contain tracklines generated ('Insert') from the shot point geometries for each deployment. The trackline database table was updated to include columns for the sonobuoy deployment name, image name, associated MCS line name, initial FFID, end FFID, Year, initial day and time, end day and time, SurveyID, VehicleID, DeviceID, and the line length in kilometers (calculated on the WGS84 ellipsoid).

PROCESS STEP 5:
The sonobuoy deployment and shot points and shot trackline features were added (Add Data) into ArcGIS Pro (version 2.3.3) from the SQLite database, then exported (using the Feature Class to Feature Class geoprocessing tool) to the new Esri point and polyline shapefiles '2018-002-FA_SB_depnav.shp', '2018-002-FA_SB_shtnav.shp', and '2018-002-FA_SB_shtTracklines.shp', respectively.

PROCESS STEP 6:
The Seismic Unix (version 4.3) script 'plot_sb' was used to read the processed SEG-Y files, balance the traces ('sugain' quantile clip and scale), and create variable density greyscale Postscript plots (using the Seismic Unix 'psimage' module) showing two-way travel time (seconds) along the y-axis (left margin) and along profile nominal offset (labeled at 1000 meter intervals) on the x-axis (along top of profile). The Postscript images were then converted to 200 dpi PNG formatted images using ImageMagick convert (version 6.9.10-78).