Big data is a combination of structured, semi structured and unstructured data collected by organizations that can be mined for information and used in machine learning projects, predictive modelling and other advanced analytics applications.
Systems that process and store big data have become a common component of data management architectures in organizations, combined with tools that support big data analytics uses. Big data is often characterized by the three V’s:
- the large volume of data in many environments;
- the wide variety of data types frequently stored in big data systems; and
- the velocity at which much of the data is generated, collected and processed.
Big data is a great quantity of diverse information that arrives in increasing volumes and with ever-higher velocity.
Big data can be structured (often numeric, easily formatted and stored) or unstructured (more free-form, less quantifiable).
Nearly every department in a company can utilize findings from big data analysis but handling its clutter and noise can pose problems.
Big data can be collected from publicly shared comments on social networks and websites, voluntarily gathered from personal electronics and apps, through questionnaires, product purchases, and electronic check-ins.
Big data is most often stored in computer databases and is analysed using software specifically designed to handle large, complex data sets.
Topics Covered in these course are:
- Big Data Enabling Technologies
- Hadoop Stack for Big Data
- Hadoop Distributed File System (HDFS)
- Hadoop MapReduce
- MapReduce Examples
- Spark
- Parallel Programming with Spark
- Spark Built-in Libraries
- Data Placement Strategies
- Data Placement Strategies
- Design of Zookeeper
- CQL (Cassandra Query Language)
- Design of HBase
- Spark Streaming and Sliding Window Analytics
- Kafka
- Big Data Machine Learning
- Machine Learning Algorithm K-means using Map Reduce for Big Data Analytics
- Parallel K-means using Map Reduce on Big Data Cluster Analysis
- Decision Trees for Big Data Analytics
- Big Data Predictive Analytics
- PageRank Algorithm in Big Data
- Spark GraphX & Graph Analytics
- Case Studies of big companies and how they operate.