MCR movie with speeds and directions

I've been running NM for several radar runs, and as a test of how things were going, I plotted the phase speed and directions for the runs that have been completed. Results are encouraging, although you may notice that I am not showing the same frequency for all runs. But as an initial step...

Note that the labels for the axis DON'T make sense. Should be Eastings and Northings relative to the radar.

Spectra

This is a comparison of the directional spectra for our pet run at Duck (2711215), from both video and radar against the 8m spectrum.

Very good agreement (though clearly with coarser resolution), even at high frequencies the directions seem to be quite reasonable. Radar still has a lot of power at zero angle though.



However, this is encouraging

Que bien!!!!

It is an old step for humankind, but at least I am moving in the right direction.

I coded today the functions required to get 3D spectrum from the data we have, and I tested succesfully a nonlinear fit of the dispersion curve to determine the depth in the absence of currents. Once I turn on the option with currents, the results deviate a lot, but I think the reason is that I have very few points to give statistical robustness to the least squares method.

Anyway, I tested the codes for video data (2711215 and outside the surfzone, ala Dugan et al.) and it yielded a mean water depth of 6.11 m, which is pretty reasonable for the region x=300:500 m in Duck.

In this movie it is possible to see the directional spectrum for the frequencies in the range f=0.03-0.05 Hz. On top of the spectra is plotted the theoretical dispersion curve obtained with that frequency and the depth obtained. Agreement in general is pretty good. It can also be seen that the dominant direction is slightly off from the shorenormal, as seen from the data movies, but I haven't really computed that value.

So, all in all, progress.

Updated breaking, now with FBB

I updated the methods to compare the radar signal with SWAN data. In particular, since the SNR from the specral analysis should be more related to significant wave height, ala Arpers and Hasselman (1982), now I am including that comparison.
In consequence, to compare the fraction of breaking waves we tried a variable that should have more physical sense than the simple varince. To do this, we used the old method to simply threshold the time stacks (or the cube in this case), assuming that values exceeding the threshold are breaking events. Then we count them and normalize the result by the fraction of breaking waves.

A couple of things to notice about the method. Number (or fraction) of breaking waves is essentially a variable with a spatial variation, but for the present analysis, since we are considering a characteristic value over a spatial box, I am using the mean value of breaking events over the box.

In addition, a rigorous method requires counting the actual number of waves at each location. Since this is not possible for the radar data (it is possible that not all the waves were imaged), I had to rely on an average value, calculated simply as the runlength (in seconds) divided by the average period (given by the buoy, 46029). Note that the average period is shorter than the dominant period, thus yielding a higher number of waves.

For the threshold, I used thresh=Imean+Istd, where Imean and Istd are the mean and standard deviation computed over the whole image (corrected to have a normalized jetty), rather than over the individual box. I did this to ensure that the same criterion was used in both boxes.

Results:

First I tried using the average period. It can be seen that for Benson Beach (box 2), the fraction of breaking waves from the radar data is smaller (probably half) than the SWAN projections. However, the correlation between both data sets is rather high (R^2=0.71). At the same time, the radar data is showing the opposite trend in the channel, where the radar data is showing breaking when SWAN is not.



When the peak period is used, the fraction of breaking waves is increased as expected (due to a smaller number of waves per run). Benson Beach results seem to approximate better the SWAN results, and correlation remains about the same (not a surprise, since we only scaled the time series differently). Obviously, the opposite result is obtained at the channel, where the fraction of breaking waves is also enhanced, departing more from the SWAN data.



Assuming that SWAN is doing a good job, the behavior in the channel suggests that the threshold value used is relatively low, but at the same time, increasing the threshold would affect negatively the results at Benson Beach, which at the moment are quite promising. Other causes could be the presence of boats (increasing mean and variance values), or the effect of currents. I am going to explore all these options now.

Regarding wave heights and their comparison with the SNR value, well....they don't agree. I am leaving this part for later.

Good news?

So after many things failed, and noticing that the angles were always messed up, I tried to run NM by changing the orientation of the waves, by defining the x-axis as pointing offshore. This led to high skill values in my test box, so I went ahead and ran the whole window. It took roughly 7 hours, because I subdivided the domain in smaller windows (84 x 84 m), with 3 iterations per frequency (to update the previous pass), retaining 5 frequencies in the band f=1/20-1/5 Hz but with a frequency resolution of 1/60 Hz (so more frequency bands were analized). These values could be changed to a faster configuration, but let's go step by step.

So here are the results, of the quilted map of the 196 windows used, for the two frequencied that yielded more results. No quantitative analysis of the relative importance of each one at this point.





Several things can be noticed:

1)Jetty influence: Clearly the jetty affects the analysis, because it is a constant signal, that does not correlate well with the wave data. Aditionally, the jetty casts a long shadow in which no radar signal is present, that explains the low skill in that area. This was something I expected to happen but it didn't show up in my previous runs.
2) Speeds: These are my main concern at the moment. Although they exhibit the associated deacceleration due to shoaling, the magnitude of the speed is rather high. I made a manual estimation for them to be near 5 m/s at x=800 m, but it seems that we are doubling that value. Something to dig in, definetely.
3)Angles: What I expected. Close to shorenormal for most of the domain, with a big patch of obliquely incidedent waves offshore of the jetty and in the channel, probably an influence of the jetty.

Here is a portion of a movie from that run (190905). Note that Benson Beach is at the top and not at the right, as it should be.

MCR Breaking stuff

Here goes the new attepmt to correlate the fraction of breaking waves coming out of SWAN with the radar signal. What is new here is the computation of the signal-to-noise ratio of the radar signal. To do this, we assumed that the wave signal is all the variance in a given frequency band (here I'm showing f = 0.01 - 0.15 Hz, chosen arbitrarely), and noise is the remaining. Hence, we integrated the grayscale intensity power spectrum over those frequency bands, and computed the ratio SNR=wave/noise. In order to compare with the fraction of breaking waves, I took the square root {SNR}. Results (blue lines) are in the ballpark, but the agreement with SWAN data (black) is visually less than that of the normalized standard deviation (green) I calculated before. The initial comparison set, (normalized mean intensity) is shown in red for completeness.



I also computed the correlation coefficient between each of the radar series and SWAN data (box 2 @ Benson Beach). They are

Normalized mean : R^2 =0.1541
Normalized std : R^2 =0.7345
sqrt(SNR) : R^2 =0.6345

This suggests that the SNR is still well correlated, but not as good as the normalized standard deviation. However, the former has a better physical background than the latter.

In addition, in order to check that I was comparing apples to apples, I compared the total variance on each box computed in the time domain (i.e., the non-normalized green data squared) versus the sum of the signal and the noise from spectral analysis, for each box. They lie pretty much on the 45 degree line, so I'm confident of the calculations.

Timestacks

As requested:

Here are two sample timestacks from the same MCR data (190905), taken at the cross-shore and alongshore transects I used yesterday. They look as expected.

To boost morale

I've decided to take a mini break from MCR and spend a little time at Duck. I focused on studying if the video and radar signals
were coherent or not.

To do this, I subdivided my domain in small windows (9x9 pixels each, that is 27x27 m), and computed the cross-spectrum at each of the 81 radar-video pairs, and then average all the statistics (auto spectra, cross-spectrum). Then, for each of them i found its maximum, and the frequency associated with it. For example, for the cross-spectrum, I found the maximum of the coherence, stored that value, and also stored the frequency associated with it. Ditto for each auto spectrum.

This analysis yields a map of maximum value of coherence, and maximum spectral power, as a function of spatial location. This would give us an idea if:

-where the signals are coherent.
-if the maxima in coherence corresponds to a frequency associated with large power in both signals. This would be the ideal case.

And here are the results for our pet run (2711215)

Maps: In the image below, the left column are the maps of peak coherence, peak radar power and peak video power. In the right colum are the corresponding peak frequencies. Several features can be seen:

1) In general, we have very coherent signals, but coherence values fall below 0.5 in two areas. The first one is the oblique linear feature, present in the coherence map but also in the frequency map associated with video autospectrum and cross-spectrum. It seems that this correspond to a camera boundary, where a strong gradient in pixel intensity exists., which affects the whole analysis.
Of major concern is the second area which is located offshore of the bar. It seem that radar and video decorrelate, probably due to the change in their respective MTFs, in particular that of viedo, which goes frm specular to breaking related. I need to try to compare this with the time exposures and variance maps.

2)Frequencies seem to be very uniform throughout the window. Noticeable is the patch associated to the video boundary.

3) Both autospectra show the increase in power in the surfzone, where breaking begins to dominate. Perhaps nother way to estimate surf zone witdths.




In order to see if the retrieved frequencies were the same for the three sets, we plot them as a function of cross-shore location. It can be seen that the peak at video spectrum and the pek cohernce overlap, suggesting that video is dominating the coherence maps.

Updated posts

Due to popular request, I forced the axis on the snapshots to be equal and to cover the same footprint (1.3 km cross-shore). Additionally, the frequency plots are restricted to f=0.04-0.3 Hz.

Here they are:

Frequency plot at Peak Coherence

MCR, 190905


Duck, 2711215


Frequency plot at peak autospectra

MCR, 190905


Duck, 2711215


Timestacks will be available tomorrow afternoon (ora pro nobis)

More plots

A couple of new plots. In this case, rather than selecting the peak frequency at the center of the domain, I selected the frequency of maximum coherence from the cross-spectrum. As expected, this procedure results in a map of high coherence values (pretty much all close to 1), but a messy frequency map.

Here they are:

MCR, 190905


Duck, 2711215

uh-oh

Since the wave direction analysis with the MCR data is not doing what I expected, I went back to calculate somevery basic parameters.
First, I estimated visually the speed of several waves in the surfzone for my test run (190905),  and that yield speeds between 4 and 2 m/s. The results given by the NM where on the order of 20-30 m/s. 

Then I looked at the directions, and NM was telling me that the waves moved alongshore, paralell to the beach.

Obviously something is not right.

So what I did is the following.

1. Compute the spectra along cross shore and alongshore transects, to see how much the spectra varied and how different were the peak frequencies obtained.

It turned out that for the MCR data, eventhough the waves look visually fairly clean, the spectrum varied significantly,  with the peak bouncing back and forth.

That could indicate that adjacent series won't be as correlated (coherent) as we would expect.

Here are the plots:

First, a cross-shore array, where it can be seen that the frequency changed quite a bit over short lenghscales.



Now an alongshore array, centered at the same point. In this case, a broad range of frequencies seem to be quite energetic, and the peak bounces back and forth.


Just for comparison, I did a similar analysis for one f our cleanest Duck runs (2711215). Here are the plots:

Cross-shore, where the peak frequency appears to be more stable, but power changed significantly.



Along-shore, where again somewhat broaf spectra is present, but the peaks appear to be more confined.

 
2. Next I did a similar analysis, but now estimating the coherence. To do this, I choose a location, and then computed the cross-spectra with all its surrounding points, and estimated the coherence at the peak frequency of the central point. Aditionally, I computed the peak frequency of the surrounding series. This yield two maps, one for the coherence, and one for the peak period.

For the MCR case it can be seen, quite surprisingy, that coherence is relatively high over a large area, but the periods show a significant amount of scatter. This is somewhat intriguing, because we would expect that if the signals are coherent, the peak periods should be similar.


Incidentally, the Duck record shows less coherent regions, but a significantly more uniform peak frequency map. It must be noted that this Duck run yield very good results (high skill) for Nathaniel's method.



Hence I am a bit lost, at the moment. Comments are welcome.