|
Structural Monitoring and Maintenance Volume 7, Number 4, December 2020 , pages 345-365 DOI: https://doi.org/10.12989/smm.2020.7.4.345 |
||
|
Big data platform for health monitoring systems of multiple bridges |
||
Manya Wang, Youliang Ding, Chunfeng Wan and Hanwei Zhao
|
||
| Abstract | ||
| At present, many machine leaning and data mining methods are used for analyzing and predicting structural response characteristics. However, the platform that combines big data analysis methods with online and offline analysis modules has not been used in actual projects. This work is dedicated to developing a multifunctional Hadoop-Spark big data platform for bridges to monitor and evaluate the serviceability based on structural health monitoring system. It realizes rapid processing, analysis and storage of collected health monitoring data. The platform contains offline computing and online analysis modules, using Hadoop-Spark environment. Hadoop provides the overall framework and storage subsystem for big data platform, while Spark is used for online computing. Finally, the big data Hadoop-Spark platform computational performance is verified through several actual analysis tasks. Experiments show the Hadoop-Spark big data platform has good fault tolerance, scalability and online analysis performance. It can meet the daily analysis requirements of 5s/time for one bridge and 40s/time for 100 bridges. | ||
| Key Words | ||
| structural health monitoring; bridge; big data; Hadoop-Spark platform; data processing | ||
| Address | ||
| Manya Wang, Youliang Ding, Chunfeng Wan and Hanwei Zhao: Key Laboratory of C&PC Structures of the Ministry of Education, Southeast University, Nanjing 210096, China; Department of Civil Engineering, Southeast University, Nanjing 210096, China | ||
