» 2016 » elokuu

It’s all about I/O

2.8.2016 by r@roopetervo.com

In previous posts I’ve been talking about data dissemination — how crucial data post-processing and exporting the data is. Producing huge amount of data is an useless effort if it can’t be ennobled to and delivered as information. But a weather model is not the source of all information. Weather models are based on many kinds of observations about atmosphere. Before running the model, those observations need to be processed and read in. It is far from trivial task.

Few months ago there was a fuzz about Panasonic’s new weather model¹. The TV Maker has been developing it’s own weather model as side of it’s TAMDAR² (Tropospheric Airborne Meteorological Data Reporting) services (a weather observation system attached to airplanes). While providing weather services for airlines they are gathering a large amount of useful data for weather forecasting. Instead of selling all data away, Panasonic took GFS³ weather model and developed their own one to take full advantage of observations they had.

The first time private company has been able to develop and run a global weather model that can compete with governments’ models. I don’t find the news themselves very surprising. Creating a well functioning weather model core sure needs huge amount of research. But this physics is commonly known. And many global weather models are open source code which gives private companies a good starting point for further development.

But there’s an other aspect which is notable in the article. One of the main advantages Panasonic says it has is a large amount of TAMDAR observations. Still, it’s not that governments’ modellers hadn’t had enough observations available. They just haven’t been able to use them. Weather forecasting has been a driving force in IT development for decades. It has created global telecommunication system (GTS)⁴ to share weather observations before World Wide Web⁵. Today, the most powerful computers are used for weather and climate modelling⁶. But long and glorious history has it’s disadvantages as well. Models are typically written in Fortran⁷ which has a great mathematical power but lacks of modern data I/O capabilities. Model developers are traditionally physicists, not IT specialists, who have more interest and skills in models’ scientific performance than creating the most sophisticated data flow. Public sectors’ weather modellers are also typically running and developing an operational system with large amount of legacy which slows down the development.

But regardless of partly inevitable disadvantages that traditional public sector weather forecasters have, the article emphasises how crucial data assimilation (reading data in) is. Having a good data assimilation process⁸ is also one reason why ECMWF model is better than GFS⁹. (These training course lecture notes¹⁰ give an idea how complicated data assimilation process can be.) Having the best available IT resources for creating fluent data I/O may also be one advantage that companies like Panasonic has.