{"id":2533,"date":"2015-06-02T10:53:31","date_gmt":"2015-06-02T10:53:31","guid":{"rendered":"https:\/\/www.happiestminds.com\/blogs\/?p=2533"},"modified":"2026-03-06T06:18:03","modified_gmt":"2026-03-06T06:18:03","slug":"streaming-big-data-analytics-with-spark-kafka-and-cassandra","status":"publish","type":"post","link":"https:\/\/www.happiestminds.com\/blogs\/streaming-big-data-analytics-with-spark-kafka-and-cassandra\/","title":{"rendered":"Streaming Big Data Analytics with Spark, Kafka and Cassandra"},"content":{"rendered":"
<\/div>
\n

Analytics has become important for every business to run successfully. Understanding your customer\u2019s requirement, predicting what they want\u00a0at the right time has become more and more important. Time to react<\/strong>\u00a0has become the key to success for many business lines, particularly for the e-commerce, automobile, healthcare, advertising industry and others. Data being\u00a0gold for the business has to be mined as it comes in. Gone are the days where you mine data to find HISTORICAL INSIGHTS.<\/p>\n

ACTION BASED INSIGHTS\u00a0need to take place\u00a0in\u00a0real-time<\/strong>\u00a0to make meaningful impacts.<\/p>\n

We will\u00a0explore a set of tools that have been successfully used across the industry to implement a system that helps\u00a0business to ingest actions based on data in near real-time.<\/p>\n

The present day business challenges:-<\/p>\n

    \n
  1. Huge volumes of data and a variety of data, making it difficult to process using traditional systems.<\/li>\n
  2. Large number of tools are available. Hadoop<\/a>, Map-Reduce, Hive, Pig, Impala, Drill, SparkSql, Cassandra, MongoDB<\/a>, Hbase, Spark, Storm, Spark Streaming, Flume ,kafka, Mahout, Mllib, Oryx and the list goes on. What should we\u00a0choose from the above to build a successful platform?<\/li>\n
  3. Involves a huge talent pool to address \/ build the system as there are a lot of dependencies between different systems.<\/li>\n
  4. Making the analytics available on time using the right streaming framework.<\/li>\n<\/ol>\n

    Possible Solutions:<\/strong><\/h3>\n
      \n
    1. Using a technology stack which has a common platform and different abilities to solve my business problem, by providing a way to stream, process and analyze the data with a little learning curve for the developer teams, thus reducing the cost to the business.<\/li>\n
    2. Spark provides exactly what we want by providing a better map-reduce implementation, richer functionality, support for machine learning, streaming and ease to implement in three different and powerful languages, which are java, scala, python.<\/li>\n
    3. There is a tough challenge over the NOSQL area. Cassandra is a\u00a0clear winner and has tight integration with apache spark.<\/li>\n
    4. Kafka perfectly fits today\u2019s real world streaming applications by providing scalability, data partitioning and ability to handle a large number of diverse consumers, all which is currently used across various companies.<\/li>\n<\/ol>\n

      Sign-off Note:<\/p>\n

      We will be watching where\u00a0these technology fits in to build a streaming model\u00a0that resembles the architecture required to solve the above business problem. Our model will follow the below architecture:<\/p>\n

      \"big-data-casandra\"<\/p>\n<\/div>\n

      \r\n
      \r\n \r\n <\/i>\r\n <\/a>\r\n 0 <\/span>\r\n<\/div><\/div>","protected":false},"excerpt":{"rendered":"

      Analytics has become important for every business to run successfully. Understanding your customer\u2019s requirement, predicting what they want\u00a0at the right time has become more and more important. Time to react\u00a0has become the key to success for many business lines, particularly for the e-commerce, automobile, healthcare, advertising industry and others. Data being\u00a0gold for the business has […]<\/p>\n","protected":false},"author":25,"featured_media":1279,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[225,312,330,123,661],"tags":[201,277,262,818,1022,1070,263,1130,1136,1162,1260,1281,1286,1306,331,1390,1426,1457,1592,1594,1595,1600],"class_list":["post-2533","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analytics","category-big-data","category-hadoop","category-technology","category-tools","tag-big-data","tag-big-data-analytics","tag-big-data-hadoop","tag-cassandra","tag-drill","tag-flume","tag-hadoop","tag-hbase","tag-hive","tag-impala","tag-kafka","tag-mahout","tag-map-reduce","tag-mllib","tag-mongodb","tag-oryx","tag-pig","tag-real-time-analytics","tag-spark","tag-spark-streaming","tag-sparksql","tag-storm"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t