Effective Incidence Angles

Two things regarding my previous posts:

1. I used a generic location for the radar location just for debugging.
   
2. I used wave conditions that were fairly small.

Now, in order to get more representative results, I am running a more chllenging wave condition, that of August 11, 2007, at 7 am (Hmo=1.38 m, Tp=7.5 s). I also put Funwave to high strain, since I decreased my grid size to 1 m, rather than 3. The only thing lingering in the air is the actual antenna height, which I don't have because the GPS readings were crazy (it reported 0 m at the antenna base), so I am using a conservative value of 5 m.

Since the antenna is further inland, the grazing angle is even smaller, and that combined with the steeper waves gave a lot of shadowing, which was something expected for nearshore applications.

But the other interesting thing was to compute the actual surface slope, which combined with the local grazing angle (et every grid point), can be used to get the effective incidence or grazing angle. This is relevant, because LGA effects are usually considered to be relevant in the range GA<20 Traditional Bragg theory seems to be best suited to the range 20 < GA < 70. One of the results of our preliminary analysis of field data showed that large waves (steep) yield increased returns, especially under the appropiate water and wind conditions. Wave breaking on the other hand, was rather insensitive to the ambient conditions (although calibrated data is needed to corrboarate this). So, we speculated that the high returns were due to Bragg-like effects. Now the question is wheter the effective angles are compatible with the range were Bragg is known to be the main mechanism.

And it seems that they are.

Here is a snap fo the free surface for this scenario. The upper panel is the free surface, where wave peaks esceeded 1 m. The second panel is the surface slope, with zero value being a horizontal surface (thus crests have zero angle). The third panel is the effective grazing angle, which is the result of the sum of the local grazing angle and the slope. It can be seen that in the front of the waves, local grazing angles can reach 20 deg for these waves, thus no longer being in the LGA regime. Moreover, the choice of z_radar=5 m is a conservative one, since a higher elevation value would shift all the values even more outside the LGA regime.



Hence, it looks reasonable that Bragg mechanisms are responsible for large returns in the front of the waves.  The confirmation of this requires modeling the radar return and compare it with the measurements.

Lags

Though in the images a good agreement can be observed, once we zoom into the details only a few events can be used for the analysis. In particular, I was able to extract only 14 events lasting about 25 seconds each, which correspond to changes in direction associated with the doughnuts we traced with the gator.

I extracted them, and then fitted a second order polynomial to both the video and radar record. The polynomial parameters were applied to a finer resolution temporal grid, from where I extracted the time of the maxima, and computed the difference.

In general, results are very consistent between the runs, as can be seen here:

Run #     Number of Events            Mean(lag) sec              Std Dev (lag) sec
1100                     5                                           -0.29                                    0.41
1200*                   2                                           -0.46                                    0.23
1230                     4                                           -0.25                                    0.15
1400                     3                                           -0.20                                    0.29
Overall                14                                           -0.28                                   0.28

Although 14 events (that is, about 300 seconds, 5 minutes) is hardly statistically robust, the results are in general pretty consistent, showing a small lag, less than a second.

The asterisk denotes a run where the Radar computer clock was reset using the gps. Sadly, only two events where identified in that run. Hence, at this point, we don't know if drift in the windows clock is going to be significant.

Synch

Hi,

The following plots show the paths of the gator as obtained by particle tracking using the video (blue), radar (red) and the garmin GPS we used on the gator.

Radar is quite noisy because the "beach" was very narrow, so it may happen that sometimes the PT algorithm picked a breaking wave or it got confused with the dune. I used the GPS points as predictor-corrector data, in the sense that if a tracked point position was more than 15 m. away from the closest GPS point, it will reset the position of the track to the GPS position for the next iteration. GPS points are thus not included in either the video or radar tracks.

The following plots correspond to the 5 runs where we have all the data. The left panel is time (seconds since midnight) against FRF x coordinate, and the right panel is time against FRF y.

Gaps in the video record typically correspond to areas where data was missing. Most noticeable is that Camera 0 died, but I am expecting to use the full frame to fill this gap. However, it seems that most of the data in that camera will be straight lines, with little structure to quantify the lag.

Gaps in the radar record are areas where a clear identification of the target was not possible.

It must be noted that the video is subject to misregistration, since the pixels were defined at z=0, but the gator traveled at higher elevations. Since we can not correct for this, more important than the actual positions are the relative motions.

Radar 2211031-Video 1186669740- Aug 09, 2007- 14:30 GMT (1030 EST)



Not too much structure in this one. We basically made alongshore transects, and we turned south of the pier. It didn't help that the radar started 1 minute or so later.

Radar 2211100-Video 1186671480- Aug 09, 2007- 15:00 GMT (1100 EST)



One of the best for our purposes. We did doughnuts near the pier, which can be seen as the periodic signal in bot x and y. The idle times near y=900 are affected by misregistration of the video. The second idle time can be used for synch as well as a "start" signal.

Radar 2211200-Video 1186675140- Aug 09, 2007- 16:00 GMT (1200 EST)



Pretty much useless. I can try to fine tune the radar track, but we made doughnuts near the radar antenna, where the dune grass affected significantly.

Radar 2211230-Video 1186677000- Aug 09, 2007- 16:30 GMT (1230 EST)



Another possibly good one. It has the added benefit that the radar gps was not reset, so it can give us a hint of the drift (if any).

Radar 2211400-Video 1186682340- Aug 09, 2007- 18:00 GMT (1400 EST)



Possibly good. Lots of radar noise, but clear y signal with doughnuts.

Now I am going to try to quantify the lag / drift/

FBB 2: The real test

Since the FBB code seemed to be working, it is time for a real test. To use it under a field condition, where the amount of remaining foam is usually larger. We used our full frame video data (though monochrome) from the Duck Mini experiment, in particular our pet run #2701215.

Here is the timex of that stack (note that the coordinates are pixels and not FRF distances).



The boxes correspond to the two test areas for the code. The magenta box (north of the pier), shows a greater contrast between the breaking areas and the offshore region, whereas for the south box (yellow), the contrast is less. Possible causes are a difference in gain or sun glare. In this case, it seems that it corresponds to a dark colored water (sediment in suspension?) as seen here



North Box Results
For the North Box, it seems that the code was doing a decent job, although it seems that we are missing some events. A sample movie of the approaching waves with the detected events marked as blue dots can be seen here:



However, if we count the total number of events, the spatial structure of the bar seems to be well recovered, as shown in the following figure. The left panel is the number of breaking events, the middle panel is the time exposure and the right panel is the overlay between the two other panels. In this kind of figure, we are,looking for areas with a good agreement which will show up with a white color. Red (magenta) areas correspond to areas where the time exposure signal is brighter than the FBB one. This suggests that tracking the FBB will "move the bar" further onshore, as opposed to the expected behavior of moving it offshore (since the effect of persistent foam would be reduced). Is worth noticing though, that we are comparing time exposures (ie brightness) against number of events. Perhaps a better comparison would be to identify the location of the events, and extract its brightness information.



Another possible way to look the results is using a timestack. I randomly selected the pixel 120 (alongshore) and here is the result of the detection process:



It can be seen that we are doing a pretty good job in the surf, but not so good in the swash. However, our area of interest is the surf. A few number of events were missed near t=600 samples, probably due to the area requirements imposed in the code. Additionally, a very bright patch of foam is responsible for the no detection of events near t=1150 and t=1700 (t is the abcissa).


South Box Results
Here we found more problems. It seems that the persistent foam was very bright, therefore identification of the breaking events was much more difficult. The time exposure give us a hint, because it can be seen that the bright area is very broad (cross-shore wise), whereas for the north area is relatively narrow.

Here is the movie, where we can see that several events are missed.


And here is the FBB, timex and overlay.


To do

Check the overall effect of area requirements in the code, when we are working with lof sampling rates and coarse resolution. It seems that for the lab experiments, where we had Delta x=25 cm and Delta t=0.1 s, that wasn't a problem. Here, the sampling rate was 2 Hz and the resolution was 3 m.

Spectra

This is a comparison of the directional spectra for our pet run at Duck (2711215), from both video and radar against the 8m spectrum.

Very good agreement (though clearly with coarser resolution), even at high frequencies the directions seem to be quite reasonable. Radar still has a lot of power at zero angle though.



However, this is encouraging

To boost morale

I've decided to take a mini break from MCR and spend a little time at Duck. I focused on studying if the video and radar signals
were coherent or not.

To do this, I subdivided my domain in small windows (9x9 pixels each, that is 27x27 m), and computed the cross-spectrum at each of the 81 radar-video pairs, and then average all the statistics (auto spectra, cross-spectrum). Then, for each of them i found its maximum, and the frequency associated with it. For example, for the cross-spectrum, I found the maximum of the coherence, stored that value, and also stored the frequency associated with it. Ditto for each auto spectrum.

This analysis yields a map of maximum value of coherence, and maximum spectral power, as a function of spatial location. This would give us an idea if:

-where the signals are coherent.
-if the maxima in coherence corresponds to a frequency associated with large power in both signals. This would be the ideal case.

And here are the results for our pet run (2711215)

Maps: In the image below, the left column are the maps of peak coherence, peak radar power and peak video power. In the right colum are the corresponding peak frequencies. Several features can be seen:

1) In general, we have very coherent signals, but coherence values fall below 0.5 in two areas. The first one is the oblique linear feature, present in the coherence map but also in the frequency map associated with video autospectrum and cross-spectrum. It seems that this correspond to a camera boundary, where a strong gradient in pixel intensity exists., which affects the whole analysis.
Of major concern is the second area which is located offshore of the bar. It seem that radar and video decorrelate, probably due to the change in their respective MTFs, in particular that of viedo, which goes frm specular to breaking related. I need to try to compare this with the time exposures and variance maps.

2)Frequencies seem to be very uniform throughout the window. Noticeable is the patch associated to the video boundary.

3) Both autospectra show the increase in power in the surfzone, where breaking begins to dominate. Perhaps nother way to estimate surf zone witdths.




In order to see if the retrieved frequencies were the same for the three sets, we plot them as a function of cross-shore location. It can be seen that the peak at video spectrum and the pek cohernce overlap, suggesting that video is dominating the coherence maps.

Updated posts

Due to popular request, I forced the axis on the snapshots to be equal and to cover the same footprint (1.3 km cross-shore). Additionally, the frequency plots are restricted to f=0.04-0.3 Hz.

Here they are:

Frequency plot at Peak Coherence

MCR, 190905


Duck, 2711215


Frequency plot at peak autospectra

MCR, 190905


Duck, 2711215


Timestacks will be available tomorrow afternoon (ora pro nobis)

More plots

A couple of new plots. In this case, rather than selecting the peak frequency at the center of the domain, I selected the frequency of maximum coherence from the cross-spectrum. As expected, this procedure results in a map of high coherence values (pretty much all close to 1), but a messy frequency map.

Here they are:

MCR, 190905


Duck, 2711215

uh-oh

Since the wave direction analysis with the MCR data is not doing what I expected, I went back to calculate somevery basic parameters.
First, I estimated visually the speed of several waves in the surfzone for my test run (190905),  and that yield speeds between 4 and 2 m/s. The results given by the NM where on the order of 20-30 m/s. 

Then I looked at the directions, and NM was telling me that the waves moved alongshore, paralell to the beach.

Obviously something is not right.

So what I did is the following.

1. Compute the spectra along cross shore and alongshore transects, to see how much the spectra varied and how different were the peak frequencies obtained.

It turned out that for the MCR data, eventhough the waves look visually fairly clean, the spectrum varied significantly,  with the peak bouncing back and forth.

That could indicate that adjacent series won't be as correlated (coherent) as we would expect.

Here are the plots:

First, a cross-shore array, where it can be seen that the frequency changed quite a bit over short lenghscales.



Now an alongshore array, centered at the same point. In this case, a broad range of frequencies seem to be quite energetic, and the peak bounces back and forth.


Just for comparison, I did a similar analysis for one f our cleanest Duck runs (2711215). Here are the plots:

Cross-shore, where the peak frequency appears to be more stable, but power changed significantly.



Along-shore, where again somewhat broaf spectra is present, but the peaks appear to be more confined.

 
2. Next I did a similar analysis, but now estimating the coherence. To do this, I choose a location, and then computed the cross-spectra with all its surrounding points, and estimated the coherence at the peak frequency of the central point. Aditionally, I computed the peak frequency of the surrounding series. This yield two maps, one for the coherence, and one for the peak period.

For the MCR case it can be seen, quite surprisingy, that coherence is relatively high over a large area, but the periods show a significant amount of scatter. This is somewhat intriguing, because we would expect that if the signals are coherent, the peak periods should be similar.


Incidentally, the Duck record shows less coherent regions, but a significantly more uniform peak frequency map. It must be noted that this Duck run yield very good results (high skill) for Nathaniel's method.



Hence I am a bit lost, at the moment. Comments are welcome.

Round 3: Little Update

I solved a little problem I had with the other 16 runs. Now the set is complete and can be seen here

Round 2

I've been able to run the code succesfully in 36 out of 51 runs.  Results are shown here. 
Some of them show funky maps. The reason is that

1. I removed NaNs.
2. I used only the data with a skill greater than 0.7 (again, an output from NM).

The last value was set arbitrarely, so I will explore other values. I hope.

Nathaniel vs CEOF. First round

After a little of muscling and wrestiling, I've been able to make Nathaniel's method to work (henceforth NM), but it came with a somewhat important computational cost.

In the process, I've found that the method is pretty robust, but it has some problems when the signal has little coherence. Nothing new here, because that is precisely the strength of the method, but this means that in practice, we can not extend the region of analysis very far offshore for most of the cases, because the radar signal is dominated by wave groups and large events, that are not coherent in the frequency range of analysis (f= 1/20-1/5 Hz). One may wonder why this did not pose a problem for the CEOF analysis. I think this is due to the fact that the CEOF method tries to find a signal that is valid throughout the domain. Once it finds a dominant signal in a certain region, implictly is extended to the rest of the domain. If the signal is not actually there, the variance of the mode would not be large. We must remember now that CEOF variances for the first mode obtained in our previous analysis were of the order 10-20 %, which is not a large value.

Hence, what I did is the following.

1. Use a similar domain as before. However, the limitation explained above showed that we don't get results seaward of x=400 m (FRF coordinates), and at the same time we increased the computational effort significantly. For the results I show here then, the domain definition is as follows xgrid=60-564 m, ygrid=580-1420 m,


2. I used 15 windows, of 168 x 168 m each to do the whole domain, and applied NM to each one of them, yielding independent results.


3. Collect all the wavenumber results for a given frequency, calculate the phase speed and interpolate them to a uniform grid. Ditto for wave angle and skill.

The following plots correspond to a comparison of this approach with our previous data set submitted to the BeachWizard community.

The upper panel is the phase speed. Green pluses correspond to the raw data used before for BW. The red points are the raw data coming from NM, where it can be seen that the new method can only extend to x=560 m. The blue line is the alongshore mean of the green pluses, and shows a very good correlation with the data from NM, which in turn exhibits significantly less noise.

Lower panel are the phase speed maps, with the same colormap. Left panel is NM after interpolation to a uniform grid, and the right data is the data I provided to BW before. They look quantitatively very similar, but NM shows less blank spaces due to the interpolation. On top of the left panel si the local wave direction, obtained from NM.

(Below are the same plots for other runs. These are the only 4 runs I have finished so far. The first one shows a condition were both methods fail catastrophically.)

Those plots have been updated by the next post in this blog. See above

Comparison Update

I also included the variance images

Just a quali comparison

In this messy gallery it is possible to take a quick look at the first comparison between radar and video timex. Nothing fancy yet, but the similarity between the records is very encouraging.

More of this waffle here

Two step forward, take one back

This is a summary of this week's events.

After running the code over the weekend to rectify the video, and doing the same for the radar, I found that my images were not aligned. Misregistration anyone? So I had to go back and figure out what the problem was (or were).

I think I solved all of them, so here is the summary:

VIDEO

1. First, it was pretty obvious that the GCPs were not on the right spot on the rectified images. Then it struck me that they are all at different Z-levels. Once I included this correction, the GCPs were they should be, when looking in the plan view (Euclidean Plane).
2. Which lead me to the idea to include the tide data in order to rectify at the right Z-level.
3. I had forgot that we have 5 cameras, so now I included the south of the pier as well. Sadly, it seems that Camera 2 does not have geometry solutions, so it will not provide data. But Camera 4 is ok.
4. I recoded my functions and they are running. Probably it will take the whole weekend.

RADAR

This baby was lingering in space with no other atachment point than the pier, which is a radial feature, which makes things more complicated.

Since the cartesian data (the one rectified using the dungle) has a somewhat arbitrary orientation, I used the pier as reference to find the zero-heading of the antenna and the doughnut size. I think I achieved a good agreement using heading=-7 deg and a doughnut of 88 pixels.

This is a sample image, from run 2701215, showing the timex of the first 14 seconds (10 frames), with the pier oriented almost along a uniform y value. Also, it seems that the GCP at the end of the pier is well matched.




The cubes are going to be reconstructed using the binary files and taking care to interpolate to a uniform grid using the actual azimuths recorded by the A*.TXT file.

We will see how it goes come monday, I hope.

Video Rectification is on the way

After many climax and anticlimax episodes, it seems that video rectification is on the way. My plan is to code the batch file by the end of tomorrow, so it can run during the weekend.

Maybe I can start video-radar comparisons soon. (*fingers and toes crossed*)