https://www.zzjharry.com/tag/deep-learning/Deep LearningWowchemy (https://wowchemy.com)en-usSat, 18 Dec 2021 00:00:00 +0000https://www.zzjharry.com/media/icon_hu8617b2913270b57d2d49c0a56c925c16_905369_512x512_fill_lanczos_center_3.pnghttps://www.zzjharry.com/tag/deep-learning/https://www.zzjharry.com/project/datasifter/Sat, 18 Dec 2021 00:00:00 +0000https://www.zzjharry.com/project/datasifter/<p>Data Sifter - TODO</p>https://www.zzjharry.com/project/showtellbeyond/Sat, 18 Dec 2021 00:00:00 +0000https://www.zzjharry.com/project/showtellbeyond/<p>Show, Tell, and Beyond - TODO</p>https://www.zzjharry.com/project/where2travel/Sat, 18 Dec 2021 00:00:00 +0000https://www.zzjharry.com/project/where2travel/<p>Where2Travel - TODO</p>https://www.zzjharry.com/project/extremeweathereventsdashboard/Mon, 01 Jan 0001 00:00:00 +0000https://www.zzjharry.com/project/extremeweathereventsdashboard/<p>During the summer of 2023, I worked as a data science intern at ClimateAi, a company dedicated to assisting businesses and organizations in formulating long-term climate resilience strategies through various AI-based services.</p> <p> <figure > <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="Project Overview" srcset=" /project/extremeweathereventsdashboard/ClimateAiInternProjectOverview_huc03b481c742eeb108cd284260239cbd6_550095_f1d96add90dec43e2126db1f9d67446c.png 400w, /project/extremeweathereventsdashboard/ClimateAiInternProjectOverview_huc03b481c742eeb108cd284260239cbd6_550095_906cb9c0f7c93125eee5e5b7fa19248a.png 760w, /project/extremeweathereventsdashboard/ClimateAiInternProjectOverview_huc03b481c742eeb108cd284260239cbd6_550095_1200x1200_fit_lanczos_3.png 1200w" src="https://www.zzjharry.com/project/extremeweathereventsdashboard/ClimateAiInternProjectOverview_huc03b481c742eeb108cd284260239cbd6_550095_f1d96add90dec43e2126db1f9d67446c.png" width="760" height="391" loading="lazy" data-zoomable /></div> </div></figure> </p> <p>I built a comprehensive extreme weather events dashboard, which contains various key attributes (e.g., type of extreme weather events, location, time, crops impacted, quantitative impact) of heatwaves, droughts, hurricanes, and other kinds of extreme weather events happening worldwide.</p> <p>There are four main components in the end-to-end pipeline: Web Scraping ⇒ Article Filtering ⇒ Information Extraction ⇒ Creating the Dashboard.</p> <h2 id="stage-1---web-scraping">Stage 1 - Web Scraping</h2> <ul> <li>This stage is to scrape publicly available articles from the internet.</li> <li>Data Source: 80,000+ articles (e.g., United Nations).</li> <li>Tech Stack: BeautifulSoap and Selenium.</li> </ul> <h2 id="stage-2---article-filtering">Stage 2 - Article Filtering</h2> <ul> <li>This stage is to filter out the irrelevant articles.</li> <li>Article Filtering Approaches <ul> <li>Approach 1 - Stemming and Keywords Matching</li> <li>Approach 2 - Directly Use Pretrained NLP Models</li> <li>Approach 3 - Sentence Embedding &amp; Similarity [Preferred] <ul> <li>Accuracy - 95%</li> </ul> </li> </ul> </li> </ul> <h2 id="stage-3---information-extraction">Stage 3 - Information Extraction</h2> <ul> <li>This stage leverages OpenAI LLMs API and prompt engineering to extract the key attributes from the structured textual data into relatively structured insights.</li> <li>OpenAI API - GPT-3.5-turbo</li> </ul> <h2 id="stage-4---creating-the-dashboard">Stage 4 - Creating the Dashboard</h2> <ul> <li>This stage is the final stage for the creation of the final dashboard, including concatenating the smaller dashboards from different data sources, cleaning &amp; structuring the dashboard, and adding new features for better visualization (e.g., filters based on specific values of an attribute).</li> </ul>