Point shapefile of navigation and best depth values at ship positions during continuous resistivity profiling data collection in the Indian River Bay, Delaware, on April 14, 2010, on U.S. Geological Survey Field Activity 2010-006-FA (JD104GPS_BESTDEPTH.SHP, Geographic, WGS 84)

The continuous resistivity profile (CRP) system used on this cruise was an AGI SuperSting marine system described at the website: www.agiusa.com/marinesystem.shtml. The particular system used for this acquisition was a 50-m streamer with an 11 electrode array with electrodes spaced 5 meters apart. The source electrodes are graphite, while the receiver electrodes are stainless steel. A dipole-dipole configuration was used for the data collection in which two fixed current electrodes are assigned with the measurement of voltage potential between electrode pairs in the remaining electrodes. The maximum depth below the water surface the streamer can reach is approximately 1/4 the streamer length. So for the 50-m streamer, maximum depth is about 12.5 meters. Each line of data acquisition records several files. The two files necessary for processing are the *.stg and the *.gps file. The STG file contains the resistivity data, while the GPS file contains the navigation information. The navigation system used in concert with the CRP system is a Lowrance LMS-480M with an LGC-2000 GPS antenna and a 200 kHz fathometer transducer. The antenna and fathometer transducer were mounted on the starboard side of the boat. The streamer tow point was on the port side aft. The layback offset between the navigation antenna and the first electrode was 17.6 meters on April 13 and 14. On April 15 the antenna and transducer were moved 1.6 m aft changing the layback offset to 16 m. This layback offset is accounted for by the acquisition system. The approximately 2 m lateral offset is not accounted for. The Lowrance transducer also contains a temperature sensor. Lowrance indicates the speed of sound used by the system is 4800 feet/second. Both the temperature and depth information are recorded in the logged GPS file. There are instances where no depth or temperature information is recorded due to an equipment problem. The CRP system images the subsurface electrical properties of an estuarine, riverine or lacustrine environment. Resistivity differences can be attributed to subsurface geology (conductive vs less conductive layers) and hydrogeologic conditions with fresh water exhibiting high resistivity and saline conditions showing low resistivity.

The data were transferred from the logging computer via AGISSAdmin software version 1.3.2.165. These files were then transferred via email to the processing computer. The files included in this publication are the *.crs, *.cmd, *.gps, and *.stg. The two files essential for processing are the GPS and STG files. The GPS file contains the navigation, and in the case of the Lowrance system also includes water depth and temperature. The STG file contains the resistivity measurements from each of the electrodes. The CRS file contains the contact resistance readings. The CMD file contains the parameters for data collection. These last two files aren't necessary for data processing, but can be useful in terms of troubleshooting. This process step, along with all subsequent process steps, was performed by the same person: VeeAnn A. Cross.

An AWK script was used to extract the navigation, bathymetry, and temperature information recorded in each individual GPS data file for each day of data acquisition. AWK script "awkhold":

BEGIN {
FS = ","
}
{
FS = ","
ARGC = 2
depth = -9999
temp = -9999
if ($1=="$GPRMC")
	{
	utctime = $2
	utcdate = $10
	latdeg = substr($4,1,2)
	latmin = substr($4,3,6)
	declat = latdeg + (latmin/60)
	londeg = substr($6,1,3)
	lonmin = substr($6,4,6)
	declon = -1 * (londeg + (lonmin/60))
	if (NR==1) {
		holddepth = -9999
		holdtemp = -9999
		}
	else {
		printf("%s, %s, %9.6f, %9.6f, %5.1f, %5.1f, %s\n", holdutctime, holdutcdate, holddeclon, holddeclat, holddepth, holdtemp, ARGV[2])
	}
	holdutctime = utctime
	holdutcdate = utcdate
	holddeclon = declon
	holddeclat = declat
	holddepth = -9999
	holdtemp = -9999
	}
if ($1=="$SDDPT")
	{
	depthreal = $2
	holddepth = depthreal
	}
if ($1=="$SDMTW")
	{
	tempreal = $2
	holdtemp = tempreal
	}
}
END {
printf("%s, %s, %9.6f, %9.6f, %5.1f, %5.1f, %s\n", holdutctime, holdutcdate, holddeclon, holddeclat, holddepth, holdtemp, ARGV[2])
}

This AWK script was initialized by "dohold" - shell script run under CYGWIN (UNIX like environment that runs under Windows). This shell script reads all the files in a folder with the extension "gps" and processes them. This is the script used for the April 14 data collection:

files=`ls *.gps | cut -d. -f1 | tr "[A-Z"] ["a-z"]`
for file in $files
do
	awk -f awkhold $file.gps $file > $file.holds
done

Each of the resulting *.holds files were concatenated together using a shell script running under Cygwin. The shell script is as follows:

cat l6f1_mod.holds \
l6f2.holds \
l6f3.holds \
l6f4.holds \
l6f5.holds \
l6f7_mod.holds \
l6f9.holds \
l6f10.holds \
l6f11.holds \
l6f12.holds \
l7f1.holds \
l7f7_mod.holds \
l7f10.holds \
l7f12.holds \
l7f13.holds \
l7f14.holds \
l7f17.holds \
l7f18.holds \
f7l19.holds \
l8f1.holds \
l9f1.holds \
l9f3.holds \
l10f1.holds \
l11f1.holds \
l11f2.holds \
l12f1.holds \
l13f1.holds \
l14f1.holds \
l15f1.holds \
l16f1.holds \
l17f1.holds \
l18f1.holds \
l19f1.holds \
l19f2.holds \
l19f3.holds > jd104gps_mod.csv

The files with "mod" in the filename have had the original GPS files edited to remove obviously bad points.

The text editor VI was used under Cygwin to add the following header line to the text file:

gpstime, gpsdate, longitude, latitude, depth_m, temp_c, line

This text file was then imported to ArcMap 9.2 using Tools - AddXY Data. The X field is longitude; Y field is latitude, and the coordinate system was defined as Geographic, WGs84. This "Event Theme" was converted to a shapefile by right-mouse clicking on the layer - Data - Export Data.

Using ArcMap 9.2 - ArcToolbox - Projections and Transformations - Feature - Project, project the shapefile from Geographic, WGS84 to UTM, Zone 18, WGS84. Parameters: input - jd104gps_mod.shp; input coordinate system - GCS_WGS_1984 (default, read from file); output - jd104gps_mod_utm18.shp; output coordinate system - WGS_1984_UTM_Zone_18N. No transformation necessary.

With the jd104hypack.shp (available from https://pubs.usgs.gov/of/2011/1039/html/ofr2011-1039-catalog.html) loaded in ArcMap 9.2, set a definition query on the shapefile where depth_m <> -9999. Then with jd104gps_mod_utm18.shp the active shapefile in the table of contents do a join on jd104gps_mod_utm18.shp: Join data based on spatial location; join layer - jd104hypack. Select the option to give all the attributes of the joined shapefile, along with a distance field. Output to a new file: jd104gps_mod_spatjoin.shp. The distances reported are in the units of the data layer initiating the join - hence the reason for projecting the CRP navigation shapefile to UTM, Zone 18. Units of meters are much easier to make sense of than decimal degrees distances.

The join function changes the order of the records in the original starting shapefile (jd104gps_mod_utm18.shp). So to put the records back in the order of time, the shapefile needs to be sorted by the FID_1 attribute. Resorting the shapefile was accomplished with an extension written by VeeAnn Cross in Woods Hole, MA - VAC Extras v. 2.1. Using VAC Extras - FeatConv - Table Sort, jd104gps_mod_spatjoin.shp was sorted with the primary sort field FID_1 in ascending order, no secondary field chosen, and the output sent to jd104gps_mod_spatjoin_sort.shp.

Several additional attributes were added to the shapefile: timediff, fixdepth. Because these depths are not tide corrected, not only do I need to replace the missing CRP depths with values that are close in proximity, but have to be cognizant of the time offset. And although the gpstime and gpstime_1 are in the format HHMMSS, subtracting these values still has some benefit. First, set a definition query on jd104gps_mod_spatjoin_sort.shp to only display the records with depth_m = -9999. Then use the field calculator to calculate gpstime - gpstime_1. For the new fixdepth attribute, select all records where depth_m <> -9999, then use field calculator to set fixdepth = depth_m. Basically, this is just bringing over the good depths. To work on the rest of the depths used a query to select all records where ("timediff" < 60 AND "timediff" > -60) AND "Distance" < 20 AND "depth_m" =-9999. This query attempts to select those records that are spatially close (less than 20 meters), where there are no valid depths (depth_m = -9999) and have a relatively small time difference, which means tidal effect should be minimal. With these records selected, use the field calculator to have fixdepth = depth_m_1. There are a number of anomalous depth values in these records where the HYPACK depth values are 198.5. These values are clearly wrong and the fixdepth values were set to surrounding depth readings of 0.6. The remaining "fixdepth" values were set manually. In most cases this means extrapolating the value of close (spatially and temporally) valid points from either the HYPACK or the CRP system. In some instances these values are probably acceptable, but in some places the gaps are too large to trust these values.

Using a bathymetry surface (of all positive values) generated from tide corrected bathymetry points (see irb_bathy to understand processing available at https://pubs.usgs.gov/of/2011/1039/html/ofr2011-1039-catalog.html), use VACExtras v 2.1 to extract the bathymetry values at each point location and place in the new attribute hyptidecor. However, the extracted depths are tide corrected, and what are needed in this case is the uncorrected bathymetric values - so the tide information needs to be added back into these values. Within ArcMap 9.2, also add the attribute jday to place the Julian day (104).

Then use XTools Pro 5.2 to export the shapefile to a text file. Export jd104gps_mod_spatjoin_sort to jd104gps_mod_spatjoin_sort_exp.txt with the exported fields of gpstime; longitude; latitude; hyptidecor; jday.

Run the files through an AWK script to reformat for loading into MATLAB to extract tide information. The tidal correction procedure is described in detail in irb_bathy (available at https://pubs.usgs.gov/of/2011/1039/html/ofr2011-1039-catalog.html). In this case, because the tides are being added back in, the MATLAB code is modified slightly so instead of:

h_corrected_ir=htrack-tidetrack_ir;
h_corrected_rd=htrack-tidetrack_rd;
use:
h_corrected_ir=htrack+tidetrack_ir;
h_corrected_rd=htrack+tidetrack_rd;

Proceed with converting the results to a shapefile and using the weighted distance calculations described in irb_bathy (available at https://pubs.usgs.gov/of/2011/1039/html/ofr2011-1039-catalog.html) to ADD the tide information back into the depth values. Now in the jd104gps_mod_spatjoin_sort.shp, add the attribute "addedtide" as double. Since the spatial locations are the same, a spatial join is not necessary, and the shapefiles can be joined record for record based on the FID attribute. Once joined, copy the value from the "tideadded" attribute to the newly created "addedtide" attribute. Then removed the join.

In order to select the best depth value for every CRP location, it's a combination of already valid values, valid values based on temporal and spatial offset, and values derived by adding tides back to a tidally corrected surface. For this reason, in ArcMap 9.2 a new attribute was added called "bestdepth" as type double to the shapefile jd104gps_mod_spatjoin_sort.shp. First priority is to use actual values, then fill in voids with derived values. Did a query where depth_m <> -9999 and then copied those values to bestdepth. Then did a query: ("timediff" <= 10 AND "timediff" >= -10) AND "timediff" <>0. The "timediff" <> 0 was necessary because where timediff = 0 is where I have valid original CRP depths. The timediff attribute is the result of gpstime - gpstime_1 where gpstime_1 is the HYPACK navigation time. Copy these selected records over to bestdepth from depth_m_1. A value of 10 was chosen because the boat was travelling around 4 knots (2m/sec). That means a value of 10 represents points within 20 meters of the actual navigation reading. That seems reasonably conservative. What this leaves is some within 10 seconds, but that go around the minute mark. For example, if the gpstime is 130355 and the gpstime_1 is 130404, then the timediff is displayed as -49. But in reality, it's only 9 seconds - all an issue with how time is recorded as an attribute. Additional queries were run to accommodate this time oddity.

Query: "timediff" <= -41 AND "timediff" >= -50

These are all within 10 seconds, so set bestdepth = depth_m_1.

Likewise, the opposite situation needed to be handled such as gpstime = 131601 and gpstime_1 = 131554.

Query: "timediff" >= 41 AND "timediff" <= 50

And if there's an hour change:

"timediff" >=4041 AND "timediff" <=4050 (this didn't have any records) "timediff" >=-4050 AND "timediff" <=-4041 (this didn't have any records)

Now any records without a "bestdepth" ("bestdepth" = 0) the values from "addedtide" were copied over.

Using ArcMap 9.2, jd104gps_mod_spatjoin_sort.shp was projected from UTM, Zone 18, WGS84 to Geographic, WGS84 using ArcToolbox - Data Management Tools - Projections and Transformations - Feature - Project. The output file was jd104gps_bestdepth.shp (no transformation necessary).

Edits to the metadata were made to fix any errors that MP v 2.9.36 flagged. This is necessary to enable the metadata to be successfully harvested for various data catalogs. In some cases, this meant adding text "Information unavailable" or "Information unavailable from original metadata" for those required fields that were left blank. Other minor edits were probably performed (title, publisher, publication place, etc.). Attempted to modify http to https where appropriate. Moved the minimal source information provided to make it the first process step. The metadata date (but not the metadata creator) was edited to reflect the date of these changes. The metadata available from a harvester may supersede metadata bundled within a download file. Compare the metadata dates to determine which metadata file is most recent.

Added keywords section with USGS persistent identifier as theme keyword (20200908). Fixed a USGS Thesaurus term (20240715).