Comments on the current tsunami benchmark problems
benchmark site:
nctr.pmel.noaa.gov/benchmark/
We have encountered a number of difficulties in performing the benchmark
tests described at the above website and in the references below, either
from incompletely specified data or from ambiguities in the description of
the target solution.
If you have data that
might help in this process, please contact Randy LeVeque (rjl at uw.edu)
or Frank González (figonzal at uw.edu).
Specific comments on each of the benchmark problems can be found on the
webpages we have created for each example, see the
index.
We also believe that additional benchmark problems are needed to allow
better verification, validation, and comparison of tsunami codes in the
future.
[Suggested problems].
General comments on the current problems:
DRAFT Version
The section numbers below refer to PMEL Report 135 (link below).
- In Section 2.1 it is stated that conservation of mass should be
confirmed by checking that the total displaced volume at the end of the
computation should be within 5% of the initial total displaced volume. It
is not clear that this would be sufficient for many of these problems,
assuming that conservation is important.
- In Section 2.1 it is stated that convergence should be tested by
checking that the numerical predictions of extreme runup and rundown
converge to "certain value, and further reductions in step sizes should not
change the computed results." This is not very well defined, since for
most problems the topography is two-dimensional and the extreme runup and
rundown will be spatially varying. This is also no guarantee that the
value a code converges to is the correct value. For a well-defined
benchmark, particular points should be specified and an agreed-upon correct
value provided to compare against. Perhaps this test is intended to apply
only to the one-dimensional problems, but this is not stated.
- In Section 2.2 various one-dimensional results are supposed to agree
with the analytic solution to within 5%. There is no specification of what
grid resolution should be used. Presumably any code that converges to the
correct solution of the shallow water equations must agree to within 5% on a
sufficiently fine grid (by definition of convergence), but the more
interesting question is whether it agrees to within a certain tolerance on a
specified grid, or conversely how fine the grid must be chosen to obtain a
given accuracy.
- The maximum runup will be very sensitive to changes in friction. Of
course convergence to the frictionless shallow water equations can be
expected only if there is no bottom friction, but some codes may require
adding friction for stability. If so, how should this be handled?
- In Section 2.1 when discussing extreme inundation,
it is stated that "it has been observed that even with
numerical codes that use friction factors within reasonable limits, the
predictions are not sensitive to the first order." However, choice of
friction factor does have some effect on results and so friction factors
(and form of the friction term) should be specified for benchmark problems
so that all codes use the same thing. This is not generally done.
- In Section 2.3 in the discussion of the conical island, it is stated
that "Predictions of the runup on the back of the island where the two
fronts collide should not differ by more than 20% from the laboratory
measurements." Why 20% here whereas most tolerances are specified to be
much smaller? In the discussion of Monai Valley it is stated that
"Comparison of results from different codes has shown that the maximum runup
in these experiments can be calculated within 10%, which is thus the
standard." Can similar statements be made about the conical island?
- To be continued...
References:
- PMEL Report 135:
Synolakis, C.E., E.N. Bernard, V.V. Titov, U.
Kânoğlu, and F.I. González (2007): Standards, criteria, and
procedures for NOAA evaluation of tsunami numerical models. NOAA Tech. Memo. OAR
PMEL-135, NOAA/Pacific Marine Environmental Laboratory, Seattle, WA, 55 pp. [PDF Version]
-
Synolakis, C.E., E.N. Bernard, V.V. Titov, U.
Kânoğlu, and F.I. González (2008): Validation and Verification of
Tsunami Numerical Models, Pure and Applied Geophysics
Volume 165, Numbers 11-12, 2197-2228, DOI: 10.1007/s00024-004-0427-y
[link]