EECS E6893: Big Data Analytics ; EECS E6895: Advanced Big Data Analytics

EECS E6893: Big Data Analytics


    • Students will gain knowledge on analyzing Big Data. It serves as an introductory course for graduate students who are expecting to face Big Data storage, processing, analysis, visualization, and application issues on both workplaces and research environments.
    • Gain knowledge on this fast-changing technological direction. Big Data Analytics is probably the fastest evolving issue in the IT world now. New tools and algorithms are being created and adopted swiftly. Get insight on what tools, algorithms, and platforms to use on which types of real world use cases.
    • Get hands-on experience on Analytics, Mobile, Social and Security issues on Big Data through homeworks and final project

Dr. Ching-Yung Lin is the Manager and Founder of the Network Science and Big Data Analytics Department in IBM T. J. Watson Research Center. He is also an Adjunct Professor in Columbia University (since 2005) and New York University (since 2014). His interest is mainly on fundamental research of large-scale multimodality signal understanding, network graph computing, and computational social & cognitive sciences, and applied research on security, commerce, and collaboration. Since 2011, he has been leading a team of more than 40 Ph.D. researchers in worldwide IBM Research Labs and more than 20 professors and researchers in 9 universities (Northeastern, Northwestern, Columbia, Minnesota, Rutgers, CMU, New Mexico, USC, and UC Berkeley). He is currently the Principal Investigator of three major Big Data projects: DARPA Anomaly Detection at Multiple Scales (ADAMS), DARPA Social Media in Strategic Communications (SMISC), and ARL Social and Cognitive Network Academic Research Center (SCNARC). He leads a major IBM R&D initiative on Linked Big Data called IBM System G. Dr. Lin was the first IEEE fellow elected for contribution to Network Science. His team recently earned the Best Paper Awards on ACM CIKM 2012 and IEEE BigData 2013.

This will be a hands-on course. Students need to know at least one or more programming languages: C, C++, Java, Perl, Python, and/or Javascript to finish homeworks and final project.

With the advance of IT storage, pcoressing, computation, and sensing technologies, Big Data has become a novel norm of life. Only until recently, computers are able to capture and analysis all sorts of large-scale data from all kinds of fields — people, behavior, information, devices, sensors, biological signals, finance, vehicles, astronology, neurology, etc. Almost all industries are bracing into the challenge of Big Data and want to dig out valuable information to get insight to solve their challenges.
This course shall provide the fundamental knowledge to equip students being able to handle those challenges. This discipline inherently invoves many fields. Because of its importance and broad impact, new software and hardware tools and algorithms are quickly emerging. A data scientist needs to keep up with this ever changing trends to be able to create a state-of-the-art solution for real-world challenges.
This Big Data Analytics course shall first introduce the overview applications, market trend, and the things to learn. Then, I will introduce the fundamental platforms, such as Hadoop, Spark, and other tools, such as IBM System G for Linked Big Data. Afterwards, the course will introduce several data storage methods and how to upload, distribute, and process them. This shall include HDFS, HBase, KV stores, document database, and graph database. The course will go on to introduce different ways of handling analytics algorithms on different platforms. Then, I will introduce visualization issues and mobile issues on Big Data Analytics. Students will then have fundamental knowledge on Big Data Analytics to handle various real-world challenges.
Afterwards, the course will zoom in to discuss large-scale machine learning methods that are foundations for artificial intelligence and cognitive networks. The course will discuss several methods to optimize the analytics based on different hardware platforms, such as Intel & Power chips, GPU, FPGA, etc. The lectures will conclude with introduction of the future challenges of Big Data, especially on the onging Linked Big Data issues which involves graphs, graphical models, spatio-temporal analysis, cognitive analytics, etc.
Students will choose the topics of their own for a final project. The application domain can be based on the students’ own interest. This will be a good opportunity for students to apply what’s learned in the class for their needs, either for the future work requirements or for the research problems at hand.

Course Outline
Class Date Class 
Topics Covered Assignment Due
09/04/14 1 Introduction to Big Data Analytics HW #0 (Download Hadoop)
09/11/14 2 Big Data Analytics Platforms
09/18/14 3 Big Data Storage and Processing HW #1 (Data Store & Processing — Pig, HBase, Hive, and Oozie)
09/25/14 4 Big Data Analytics Algorithms — I
10/02/14 5 Big Data Analytics Algorithms — II (recommender) HW #1 Demo (updated) HW #1
10/09/14 6 Big Data Analytics Algorithms — III (clustering) HW #2 (Recommendation & Clustering); Demo HW #1 (extended)
10/16/14 7 Big Data Analytics Algorithms — IV (classification)
10/23/14 8 Big Data Analytics Algorithms — V (classification) HW #2
10/30/14 9 Linked Big Data: Graph Computing I (Graph DB) HW #3 (Classification & Graph DB); Classification Demo
11/06/14 10 Linked Big Data: Graph Computing II (Graph Analytics)
11/13/14 11 Linked Big Data: Graphical Models and Hardware Platform Issues HW #3
11/20/14 12 Final Project Proposal Presentations
11/27/14 NO CLASS — Thanksgiving Holiday
12/04/14 13 Big Data Visualization
12/09/14 & 12/11/14 14 Full-Day Big Data Analytics Workshop Final Project Slides



EECS E6895: Advanced Big Data Analytics



Class Date Class 
Topics Covered Assignment Due
01/22/15 1 Introduction to Advanced Big Data Analytics
01/29/15 2 Big Data Analytics Case Study
02/05/15 3 Spark and Data Analytics HW #1
02/12/15 4 Data Store
02/19/15 5 Social and Cognitive Analytics HW #1
02/26/15 6 Social and Cognitive Analytics II HW #2
03/05/15 7 Social and Cognitive Analytics III
03/12/15 8 Social and Cognitive Analytics IV HW #2
03/26/15 9 Encrypted Domain Data Mining HW #3
04/02/15 10 Parallel Computing, GPU, and Graphs
04/09/15 11 Final Project Proposal Presentations HW #3
04/16/15 12 GPU, CUDA, and Cognitive Security
04/23/15 13 GPU Examples and GPU on iOS devices
04/30/15 14 Mobile Big Data Visualization
05/18/15 15 Final Project Presentations Final Project Slides


You may also like...