Hadoop CDC and processes notification with Apache Falcon, Apache ActiveMQ, and Apache Camel
Some weeks (months ? ;)) ago, I started to work on Apache Falcon. First of all, I would like to thanks all Falcon guys: they are really awesome and do a great job (special thanks to Srikanth, Venkatesh, Swetha). This blog post is a preparation to a set of “recipes documentation” that I will propose in Apache Falcon. Falcon is in incubation at Apache. The purpose is to provide a data processing and management solution for Hadoop designed for data motion, coordination of data pipelines, lifecycle management, and data discovery. Falcon enables end consumers to quickly onboard their data and its associated processing and management tasks on Hadoop clusters. A interesting feature provided by Falcon is notifications of the activities in the Hadoop cluster “outside” of the cluster 😉 In this article, we will see how to get two kinds of notification in Camel routes “outside” of the Hadoop cluster: a Camel route will be notified and triggered when a process is executed in the Hadoop cluster a C