When we do data analysis, we need to export data from the data warehouse according to the needs of the business, and then download the results. Of course, we can do it ourselves, and complete our work. However, as business iterations incrementally complicate, we need to have a more systematic data self-help platform to complete. It has diverse and multi-dimensional functions such as task submission, task management, monitoring, cluster resource management and monitoring.
There are still a lot of good open source systems in the data self-service platform. We got up early, using Apache, Zeppelin, Hue and so on. However, with the rapid development of the business and some specific requirements of Internet Co, the existing open source system has been unable to meet the task management, cluster monitoring, expansion and other functions.
Therefore, we in the past time, starting from the needs of the Internet Co, starting from the DEVS you use experience and feedback, combined with some of the industry's large open source data message monitoring&manager, starting with some thoughts on monitoring&manager, design and development of the monitoring&manager system now : Hive Cube.
The goal of Hive Cube is to make data export easier, Hadoop cluster resource monitoring is more convenient.