Real-time Insights from Social Media Data

Loaded and inspected data for topics that were hot worldwide (WW) and in the United States (US) at the moment of query Used json.dumps() method to have the data formatted as a pretty JSON string Used python’s set datastructure to find the common trends between “World Trends” and “US Trends” Digged deeper to explore the common hot trend “#WeLoveTheEarth” Performed frequency analysis to get the sense of the data and Extracted useful information from retweets Manipulated and Visualized the data in a better and richer way Analysed languages used in tweets Link to GitHub Repository

May 19, 2020

Uncovering The Hottest Topics in Machine Learning

Explored the NIPS Papers data, to determine what type of data it is and how it is structured Prepared the data by removing all the columns that do not contain useful text information Visualized the number of publications per year, to understand the extent of the machine learning ‘revolution’! Modified the text data by applying regular expression to remove any punctuation in the title to make it more amenable for analysis Created a word cloud of the titles of the research papers using Andreas Mueller’s word cloud library.

May 18, 2020

Book Recommendations from Charles Darwin

Explored the books to be used in recommendation system, and loaded the contents of each book Pre-processed the data to facilitate the downstream analysis Referred Darwin’s most famous book: “On the Origin of Species.” for consistency of the analysis Transformed the Corpus (collection of words) into a format that is easier to deal with for the downstream analyses, i.e., transform each text into a list of the individual words (called tokens) Implemented Stemming Process to group together the inflected forms of a word so they can be analyzed as a single item: the stem Loaded the final result from a pickle file to make the process faster, as stemming algorithm takes several minutes to run Created universe of all words, i.