Release V0.3

External Project Contribution

Background Knowledge

On October 19th 2020, during my Release v0.2, I worked on a web scraper. Web scraping is extracting data from websites, such as Google. While I was browsing on Github, I found this issue. I have used web data extraction before when I was harvesting urls for a website. The repository owner for that weeks project, was looking to purchase some weights for at home workouts during quarantine. The repository owner wanted a notification sent to his personal email when the scraper found weights in his price range. You can check out the implementation of SendGrid here. 

This week, I thought it would be cool to try and implement a web scraper myself! I found this issue while browsing on Github called realpython/python-scripts. I always thought these repositories were about getting a commit into a repository for Hackoctober Fest, also known as cheating. It turns out these repositories are actually pretty cool! The owner of the repository created an issue saying it would be nice to have a script that scrapes all the events from the list of Hackoctober events, and store them in a CSV file. I thought this would be a great challenge to test my abilities in Python. 

During this Pull Request, I got some exposure to the library called Pandas. Pandas is a software library for data manipulation. Although I didn’t get to work with Pandas very much, I got to use DataFrames one of the methods that this library includes. BeautifulSoup is another library that I had the pleasure to gain experience with. BeautifulSoup is a package for parsing HTML documents and allows you to find HTML attributes efficiently. With the use of BeautifulSoup, I was able to find the HTML elements that I wanted to manipulate rather than waste time parsing any other text that I did not want. I simply was able to append this data to a list, that I would later use to create my comma separated value list.

 




Comments

Post a Comment

Popular posts from this blog

Working with others

     This week our course was given the task of creating a cli command, which included our code being reviewed by one of my classmates. Initially, this gave me some anxiety as sharing my work with others is not my strongest suit. Nonetheless, I created a post asking if anyone was looking for a partner and Michael Appeared.      It turns out that having someone look over my work was much less scary than I had initially anticipated. Working with Michael was actually fun, and having the opportunity to look over someone else's work was a great learning experience for me.       With another pair of eyes looking at my code, Michael noticed errors and instructions that I was not addressing. Some instructions I had forgot to mention include the fact that my program requires bash. It did not occur to me that most of us live in a Windows Operating System. Another missing component was my Blog link in my READme. I had forgotten to include a link to my blog! As soon as Michael notified me he w
Read more

The Progress

This week I decided on a repository to work on, the telescope repository. I was excited since this was a front end development issue. I had never created my own component before, however, I have some prior experience using React. I started by creating a bottomBar component and locating where I should add this component. I located the PageBase.js, the mobile application interface, and added the component to the bottom of the page. I started by creating a div with a red background to make sure I was on the right track. since it has been a couple months that I have not worked with React, I started with the most basic tasks, such as, creating a react component, a  div element, and rendering the component I made.     After confirming that I did not break anything, I started designing the layout of the desired BottomBar. The idea was created by  agarcia-caicedo,   proposed a bottom app bar t o improve mobile usability. If you're curious on the original idea you can check out his 
Read more

We are off! V0.1

Initial Release V0.1  Our first Assignment was to create a command-line tool to check and report dead links in a file. Status code 200 would be considered "Good", 400 and 404 is "Bad", and all other status codes are considered "Unknown." My first experience creating a release was nerve racking. With no instructions on how to approach this problem, I started to brain storm ideas, and eventually started questioning my abilities. Inevitability, I chose to do my assignment in bash since it would allow me to colour my code, as well as create a command line tool. Overall, the first assignment was fairly simple to use. Simply clone or fork the repository, open up bash, and run the command. Command : ./cliCommand <filename> NOTE: This project will need Bash installed on your machine. Mac OS has bash built in, however, Windows OS may need to install Bash. Please feel free to play with codes, if you have a crazy idea, then pull request :) Usage and Features Us
Read more