The Accuracy of WSR-88D Precipitation Estimates

For Convective Rainfall Events Across Utah

By: Ed Carle and Randy Graham

INTRODUCTION

The addition of the KMTX and KICX WSR-88D radars in Utah has given forecasters a valuable new tool for use in flash flood forecasting. In particular, rainfall estimates produced by the 88D precipitation algorithms present the opportunity for a substantial increase in the coverage of quality data available to the forecaster. However, before this data can be used effectively we must develop an appropriate level of confidence (or lack thereof) in the precipitation estimates. The goal of this study was to determine the accuracy of the 88D precipitation algorithms by comparing radar estimates to ground truth reports for convective rainfall events.

METHODOLOGY

Rainfall reports of .70" or greater in approximately one hour or less on days that we had archived radar data were catalogued for the convective seasons of 1996 and 1997. If these spotter reports of "heavy" rain were within approximately 75 nm of the radar they were compared to the radar estimates for the same time period. Using these criteria a total of 16 cases were examined, 13 from KMTX and 3 from KICX.

The following definitions were developed concerning the accuracy of the radar estimates in comparison to the spotter reports:

a. Close: difference of 15 percent or less.

b. Under/Over: difference of greater than 15 percent but less than or equal to 30 percent.

c. Significant Under/Over: difference of more than 30 percent.

d. Reasonably Accurate: difference of 30 percent or less.

e. Good look at the storm: no reason (such as beam blockage or slow-moving cells over mountainous terrain where more clutter suppression occurs etc.) to assume that radar estimate should be more than 30 percent different from spotter report.

Several assumptions had to be made in comparing the precipitation estimates and the spotter reports. One particularly important assumption was that we were able to identify the exact locations of the spotter reports. Interpreting the specific location of spotter reports was not always an easy task (i.e., does spotter X who is listed as living in Tooele live on the east side of town, the south side of town, or 3 miles outside of town?). If there was debate as to the location of a report, phone calls were made to the spotters to help us ascertain their exact location. An even more difficult task was translating the spotter location to the radar map background. We assumed that the 88D map backgrounds were reasonably accurate (there appeared to be a few areas where the 88D background maps were inaccurate, including Salt Lake County). Cases in which hail contamination or beam blockage appeared to affect the radar precipitation estimates were given special attention or thrown out entirely.

Other considerations included the difference between the way the radar calculates the precipitation estimates versus a spotter reporting a point value. Since the radar gives an areal estimate of precipitation (in 1.1 nm mile by 1 degree bins) and the spotter reports are for a point location some data "blurring" was done to come up with the radar estimates. For example, when comparing the radar data to a spotter report we did not use a single pixel of radar data. Instead we "blurred" the data by averaging the pixels around the location where we believed the spotter report was from (this process was somewhat subjective). This yielded more of an average precipitation estimate for a small area. In several cases the areal averages yielded less accurate results than if we had used the single pixel where we believed the spotter was located. This was done due to the uncertainty of the exact location of the spotters although we are quite confident that we were able to put them within 1-2 nm of their exact location.

After "blurring" the radar estimates by averaging adjacent pixels we generated an "exact" amount by taking the midpoint of the specific color group on the 88D color table. For example, if the spotter location fell right in the middle of a large area of light green (marking an estimate of at least .50") on the one hour precipitation product we would assign a value of .62" to that color. This was done in that the next break point on the 88D color curve is at .75" and the average of .50" and .75" is .625". As you can see several significant assumptions and considerations affected the values which we assigned the 88D estimates.

RESULTS

Using the above methodology, 16 cases were examined:

5 were considered "close" (error of 15% or less)

5 were considered "underestimates" (under by 16-30%)

2 were considered "overestimates" (over by 16-30%)

3 were "significant underestimates" (under by more than 30%)

1 was a "significant overestimate" (over by more than 30%)

(Please see Table 1 for case by case results)

We found that for these 16 cases the KICX and KMTX precipitation estimates were accurate within 30 percent of the reported rainfall amounts 75 percent of the time (for amounts in the .75 - 1.25 inch per hour range). We defined this as being "reasonably accurate." It was also noted that when there was more than a 15 percent discrepancy between the radar and the spotter report the radars were more likely to underestimate amounts (8 cases or 73 percent of the time) than they were to overestimate them (3 cases or 27 percent of the time). In addition, two of the three cases where the radar overestimated the rainfall amounts the estimates were clearly affected by hail contamination.

A closer look at several of the cases will better demonstrate our interpretation techniques. The first example is a case from the Brigham City area. The radar estimated amounts between .67" and .87" across much of the Brigham City area. A spotter reported .90" in just over an hour in a location which was encompassed by radar estimates of .87" (figure 1...X marks spotter location). With a radar estimate of .87" versus a spotter report of .90", this case clearly falls into the "close" category.

Another example is an event from northern Weber county. The radar was estimating amounts of 1.75" (figure 2...X marks spotter location) across portions of northern Weber county. Spotter reports from the same area indicated only .75" of rain fell during this time. This was the one significant overestimate in our data set. However, there were also numerous reports of hail in this area ranging from pea-size to one inch in diameter. The number of hail reports that were received suggests a strong likelihood of hail contamination in this area. So, in this case the poor precipitation estimates are not a big problem in that an alert radar operator would be aware of the potential hail contamination problem (assuming that he/she had received the hail reports).

CONCLUSIONS AND RECOMMENDATIONS

Sixteen cases were examined from the convective seasons of 1996- 1997. For these cases we had accurate spotter reports of significant rainfall occurring in approximately one hour or less as well as archived radar from the KICX and KMTX radars. The radar precipitation estimates were accurate within 30 percent of reported rainfall amounts 75 percent of the time in the .75" - 1.25" per hour range. For these 16 cases the radars were more likely to underestimate precipitation amounts than to overestimate them. Overall the radar estimates compared favorably to ground truth reports.

How can data from this study be utilized by NWSFO SLC forecasters? In a moist and unstable airmass during the warm season, forecasters can probably assume that KMTX and KICX are reasonably accurate (with a tendency to underestimate) when estimating rainfall amounts in the .75" - 1.25" per hour range. If it appears that the radar does not have a "good look" (see definition in Methodology section)at the storm the underestimates could be more significant. Of course, if hail contamination is suspected then corresponding overestimates in the algorithm output are likely.

Rainfall amounts of 1.00 inch or greater in 20 minutes or 1.50 inches or greater in 1 hour have historically been considered as flash flood producing rainfall by the Salt Lake City office. Based on this study, forecasters can probably conclude that if the radar estimates meet these thresholds (or are even a little less) then the issuance of an Urban and Small Stream Flood Advisory or a Flash Flood Warning (if it is a life threatening situation) would be appropriate. This assumes, of course, that other conditions are favorable for flash flooding (e.g., flash flood prone area, deep moisture, etc,).

The study included no spotter reports with amounts greater than 1.50 inches of rain (our highest report was 1.49 inches) and there was only one case of radar estimated rainfall greater than 1.50 inches (1.75 was the highest). However, there does not appear to be any reason to assume that radar estimates of greater than 1.50 inches of rain are any less reliable than our findings for amounts in the .75" - 1.25" range.

This is a very small data set and additional cases would no doubt yield improvements to the results of this study. In particular more comparisons need to be done with data from the KICX radar. Flash flooding is much more likely under the KICX radar umbrella than it is within the reaches of KMTX. Yet, we did not have many cases of significance in southern Utah either because of a lack of radar data or a lack of spotter reports. Despite the reasonable success of the 88D for these 16 cases a great deal of interpretation is required by the radar operator to determine the accuracy of the 88D estimates. Overall, the algorithm performed rather well with the majority of the poor estimates clearly caused by mitigating circumstances such as hail contamination or beam blockage.