Generating Real-time Heat Maps for the Middle East region

How we used lapse rate equations to interpolate a sparse geo-spatial weather information grid in real-time

alt

Live heat-map of the Middle East showing 2m temperature interpolated from our distributed weather stations (shown on the map as blue dots). The x and y axes represent longitude and latitude, and the color scale on the right represents the temperature.

Almost a year ago, ArabiaWeather began deploying a number of small weather stations around the Middle East region, all of which were connected to the Internet of Things and sent back real-time data to our servers at an update rate of about 5 seconds. One of the goals of this project was to create a better understanding of the real-time weather of the region. The problem that we faced before was the fact that weather information could only be gathered from METAR and SYNOP reports that came from very sparsely located airports around the Middle East and at a very slow update interval (usually around 1 hour for each report). To get anything closer to real-time, we would resort to using the last available forecast for the region at the current time; this creates many obvious problems, the biggest of which is the accuracy of the readings that were provided from this method.

Our proposed solution to this problem was to use the fast-updating and widely dispersed weather stations around the Middle East to create a sparse geo-spatial grid of weather information that could be then interpolated to fill in the gaps in the grid that had no weather stations.

What is interpolation?

In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within a range of known data points. Specifically in our case, we have a sparse geo-spatial grid that we need to interpolate. A geo-spatial grid is a grid that covers a large geographical area, and a sparse geo-spatial grid means that the known points in the grid are very few, have vast amounts of empty space between them, and are distributed unevenly. This makes interpolation a challenging task for anyone to program in a way that is both efficient, accurate, and performant.

Our approach to geo-spatial interpolation

The way we approached interpolation was by overlaying a high-resolution elevation grid onto the sparse geo-spatial grid in such a way that, given any point on the grid, we could know its elevation and the elevation of all points surrounding it. What this allows us to do is use lapse rate equations to figure out unknown weather information if the surroundings are known. For a heat map (a map that shows the real-time 2 meter temperature of a particular region) we are interested in interpolating temperature and this is done using a mix of the dry lapse rate and the wet lapse rate. This mixing ratio between dry and wet is defined by the relative humidity of that location (or the average relative humidity of the surrounding locations if the current location has unknown information).

alt

All grids are produced using a digital elevation map which is 500m by 500m in resolution, and we overlay the grid with a land-sea mask to show the results properly to the user

The workflow of the algorithm is somewhat simple; given a point with unknown weather information, iterate over all its surrounding points and use the average difference in elevation, along with the average calculated lapse rate and temperature of the surrounding points, to calculate the new temperature of the unknown point. We also interpolate other variables, such as the dew point and the mean sea level pressure, because they are used in the wet lapse rate calculation. All new known points now become useful in the next iteration over the grid’s unknown points because we now have more information to interpolate from.

What is worthy to note is that even if a point is known, it is still interpolated in every iteration. This is done to “smooth” the grid and relate all information for a certain point to all other known points in the grid. For example, considering the Middle East region, a point in Southern Jordan is mainly affected by information interpolated from weather stations in Amman, but is also slightly affected by information interpolated from weather stations in Qatar and Dubai. This means that the more stations we add to cover more areas of the Middle East region, the more accurate our interpolations become.

How well does our method work?

To demonstrate the accuracy of our algorithm, we tried obscuring data from a certain weather station, and letting the algorithm interpolate for that point using the data from all other weather stations. We found that this produces a 2 meter temperature with a maximum error of ±2 degrees Celsius (for example, when hiding all our weather stations in Qatar). In most cases, the average error was ~0.5 degrees Celsius when interpolating over short distances (this was done by hiding one weather station in Amman or Dubai, given the presence of other stations in that area). This leads us to believe that our interpolation methods produce a very accurate heat-map for the Middle East region, which can easily be extended to other kinds of maps, provided they have lapse rate equations. With the addition of more and more weather stations to the Middle East region, ArabiaWeather will be able to view in real-time a very accurate heat-map of the region, to an extent of accuracy that, so far, no other weather company can achieve.

alt

The algorithm interpolates well over large distances (even when grid-points are separated by the sea) and the accuracy can be easily increased as we add more and more stations across the Middle East region. The heatmap shown above shows the regions of UAE and Oman, and it is possible to predict the temperature on the peaks of the Al Hajar Mountains by interpolating from stations in Dubai.