PPoPP 2019
Sat 16 - Wed 20 February 2019 Washington, DC, United States
Mon 18 Feb 2019 10:00 - 10:25 at Salon 12/13 - Session 1: Big Data Chair(s): Roberto Palmieri

Ensemble Kalman filter (EnKF) is one of the most important methods for data assimilation, which is widely applied to the reconstruction of observed historical data for providing initial conditions of numerical atmospheric and oceanic models. With the improvement of data resolution and the increase in the amount of model data, the scalability of recent parallel implementations suffers from high overhead on data transfer. In this paper, we propose, S-EnKF: a scalable and distributed EnKF adaptation for modern clusters. With an in-depth analysis of new requirements brought forward by recent frameworks and limitations of current designs, we present a co-design of S-EnKF. For fully exploiting the resources available in modern parallel file systems, we design a concurrent access approach to accelerate the process of reading large amounts of background data. Through a deeper investigation of the data dependence relations, we modify EnKF’s workflow to maximize the overlap of file reading and local analysis with a new multi-stage computation approach. Furthermore, we push the envelope of performance further with aggressive co-design of auto-tuning through tradeoff between the benefit on runtime and the cost on processors based on classic cost models. The experimental evaluation of S-EnKF demonstrates nearly ideal strong scalability on up to 12,000 processors. The largest run sustains a performance of 3x-speedup compared with P-EnKF, which represents the state-of-art parallel implementation of EnKF.

Mon 18 Feb

Displayed time zone: Guadalajara, Mexico City, Monterrey change

09:35 - 10:25
Session 1: Big DataMain Conference at Salon 12/13
Chair(s): Roberto Palmieri Lehigh University
09:35
25m
Talk
Beyond Human-Level Accuracy: Computational Challenges in Deep Learning
Main Conference
Joel Hestness Baidu Research, Newsha Ardalani Baidu Research, Gregory Diamos Baidu Research
DOI
10:00
25m
Talk
S-EnKF: Co-designing for Scalable Ensemble Kalman Filter
Main Conference
Junmin Xiao , Shijie Wang Institute of Computing Technology, Chinese Academy of Sciences, Weiqiang Wan Institute of Computing Technology, Chinese Academy of Sciences, Xuehai Hong Institute of Computing Technology, Chinese Academy of Sciences, Guangming Tan Chinese Academy of Sciences(CAS)
DOI