Comments on the current tsunami benchmark problems

benchmark site: nctr.pmel.noaa.gov/benchmark/

We have encountered a number of difficulties in performing the benchmark tests described at the above website and in the references below, either from incompletely specified data or from ambiguities in the description of the target solution.

If you have data that might help in this process, please contact Randy LeVeque (rjl at uw.edu) or Frank González (figonzal at uw.edu).

Specific comments on each of the benchmark problems can be found on the webpages we have created for each example, see the index.

We also believe that additional benchmark problems are needed to allow better verification, validation, and comparison of tsunami codes in the future. [Suggested problems].

General comments on the current problems:

The section numbers below refer to PMEL Report 135 (link below).

• In Section 2.1 it is stated that conservation of mass should be confirmed by checking that the total displaced volume at the end of the computation should be within 5% of the initial total displaced volume. It is not clear that this would be sufficient for many of these problems, assuming that conservation is important.

• In Section 2.1 it is stated that convergence should be tested by checking that the numerical predictions of extreme runup and rundown converge to "certain value, and further reductions in step sizes should not change the computed results." This is not very well defined, since for most problems the topography is two-dimensional and the extreme runup and rundown will be spatially varying. This is also no guarantee that the value a code converges to is the correct value. For a well-defined benchmark, particular points should be specified and an agreed-upon correct value provided to compare against. Perhaps this test is intended to apply only to the one-dimensional problems, but this is not stated.

• In Section 2.2 various one-dimensional results are supposed to agree with the analytic solution to within 5%. There is no specification of what grid resolution should be used. Presumably any code that converges to the correct solution of the shallow water equations must agree to within 5% on a sufficiently fine grid (by definition of convergence), but the more interesting question is whether it agrees to within a certain tolerance on a specified grid, or conversely how fine the grid must be chosen to obtain a given accuracy.

• The maximum runup will be very sensitive to changes in friction. Of course convergence to the frictionless shallow water equations can be expected only if there is no bottom friction, but some codes may require adding friction for stability. If so, how should this be handled?

• In Section 2.1 when discussing extreme inundation, it is stated that "it has been observed that even with numerical codes that use friction factors within reasonable limits, the predictions are not sensitive to the first order." However, choice of friction factor does have some effect on results and so friction factors (and form of the friction term) should be specified for benchmark problems so that all codes use the same thing. This is not generally done.

• In Section 2.3 in the discussion of the conical island, it is stated that "Predictions of the runup on the back of the island where the two fronts collide should not differ by more than 20% from the laboratory measurements." Why 20% here whereas most tolerances are specified to be much smaller? In the discussion of Monai Valley it is stated that "Comparison of results from different codes has shown that the maximum runup in these experiments can be calculated within 10%, which is thus the standard." Can similar statements be made about the conical island?

• To be continued...

References:

• PMEL Report 135:
  Synolakis, C.E., E.N. Bernard, V.V. Titov, U. Kânoğlu, and F.I. González (2007): Standards, criteria, and procedures for NOAA evaluation of tsunami numerical models. NOAA Tech. Memo. OAR PMEL-135, NOAA/Pacific Marine Environmental Laboratory, Seattle, WA, 55 pp. [PDF Version]

• Synolakis, C.E., E.N. Bernard, V.V. Titov, U. Kânoğlu, and F.I. González (2008): Validation and Verification of Tsunami Numerical Models, Pure and Applied Geophysics Volume 165, Numbers 11-12, 2197-2228, DOI: 10.1007/s00024-004-0427-y [link]