Split-beam Echo Sounder and Navigation Data Collected Using a Simrad EK80 Wide Band Transceiver and ES38-10 Transducer During the Mid-Atlantic Resource Imaging Experiment (MATRIX), USGS Field Activity 2018-002-FA

PROCESS STEP 1: ESP3 (version 1.8.1) and python were used to create volume backscatter strength (Sv) profile images of the split-beam echo sounder data, extract navigation data from the Simrad raw files, and process the extracted navigation data. The processing flows are summarized below.
1) Simrad raw files were imported into ESP3 ('open file') by survey day. Using the 'layer list' tab, input files were evaluated for their time length and quality. Shorter files were concatenated ('merge selected layers') to produce continuous data sections approximately 1 to 4 hours long, and longer files were left as is. Files in which acquisition settings caused changes in sample rate mid-file, an issue that ESP3 recognizes but does not correct for, were generally not concatenated or included as the last file in a longer concatenated section (see logical consistency report for a list of files effected by this issue). Settings in the 'Display Options' tab determined several elements of the images produced; 'Data' was always set to 'Sv' (in dB referenced to 1 inverse meter), X-Scale was always set to 300 seconds (for 5 minute annotation interval), the Y-Scale varied between 10 and 500 meters depending on the duration of the image, and default values were left in remaining fields. For each of the desired output sections, the zoom tool was used to restrict the image area to be above the deepest sea floor depth, and 'Export > Save Echogram' was used to create an output PNG file of the profile showing depth (meters, relative to instantaneous sea level) along the y-axis (left margin) and UTC time along the profile (labeled at 5-minute intervals), with a color bar scale for Sv (in dB referenced to 1 inverse meter) along the right margin. Navigation was extracted from all the Simrad raw files using 'Export > Export Navigation GPS to .csv or shapefile from raw files', specifying the all navigation option, and saved as CSV ASCII text.
2) The python notebook EK80_ESP3NavProc.ipynb utilized Pandas (version 0.25.1) 'read_csv' to import data from the CSV ASCII files into a dataframe containing columns for filename, Pandas datetime (UTC time), latitude, and longitude. Longitude values were converted from positive degrees measured from the zero degree origin to negative values in the western hemisphere (by subtracting 360 degrees), Pandas datetimes were rounded to the nearest second, and new records with navigation coordinates at even one minute datetimes were interpolated between the input time bounds of each filename. The dataframe was recast to produce a new dataframe with columns for longitude (Lon), latitude (Lat), linename (LineName), a combined year and Julian day string (Year_JD), and a combined Julian day and UTC time string (JD_UTC); the last two were produced via the Pandas datetime string function. Additional columns were added to contain survey (Survey_ID), device (Device_ID), and vessel (Vehicle_ID) identifier strings. A subset of the data, maintaining the first, last, and even one minute time records for each filename, was stored in an additional dataframe. The 1-minute interval was chosen for convenient correlation to the 5-minute annotation interval provided along the bottom of the PNG profile images. Pandas 'to_sql' was used to create and populate SQL database tables from the complete and 1-minute subset navigation dataframes within a Spatialite (version 4.3.0) enabled SQLite (version 3.3.0) database. The resulting database columns for each table consist of Lon, Lat (WGS84 dd), LineName, Year_JD, JD_UTC (DDD:HH:MM:SS), SurveyID, VehicleID, and DeviceID. SQLite operations created ('AddGeometryColumn') and populated ('Update') point geometry columns in each of the new tables from the navigation coordinates. A third database table was created with a line string geometry column ('Create Table' and 'AddGeometryColumn') to contain tracklines generated ('Insert') from the complete set of input navigation point geometries for each line (sorted by LineName and UTC Time), and the line length in kilometers was calculated. The trackline database table was updated to include columns for the year and Julian day and Julian day and time of the first and last (YJD_inti, JDUTC_init, YJD_end, and JDUTC_end) navigation records for each filename. The resulting database columns of the line geometry table consist of LineName, YJD_init, JDUTC_init, YJD_end, JDUTC_end, SurveyID, VehicleID, DeviceID, and Length_km.
These process steps and all subsequent process steps were conducted by the same person - Wayne Baldwin.

PROCESS STEP 2: The EK80 1-minute navigation points and trackline features were added (Add Data) into ArcGIS Pro (version 2.3.3) from the SQLite database, then exported (using the Feature Class to Feature Class geoprocessing tool) to the new Esri polyline shapefiles '2018-002-FA_EK80NavPoints_1min.shp' and '2018-002-FA_EK80Tracklines.shp'. The field 'EK80_Image' was added (Add Field) to the attribute table of '2018-002-FA_EK80Tracklines.shp' and manually populated by entering the name of the PNG profile image encompassing the time period for each raw filename.