a database scraper created with mechanical soup and sqlite
author: Mariya Sha
This repository was created to support a YouTube tutorial:
https://youtu.be/MkGQmZoMuRM
scraper.py
is designed to webscrape a table from Wikipedia and store it in an SQlite database.
selector.py
is designed to test wether our database from scraper.py was properly created.
proxy scraper 🔎 Installation: git clone https://github.com/ebankoff/proxy_scraper Required pip libraries (pip install library name): lxml beautifulso
Introduction This is a crawler I wrote in Python using the APIs of Telethon months ago. This tool was not intended to be publicly available for a numb
Docker containerized Python Flask API that uses selenium to scrape and interact with websites
Google Scholar Web Scraping This is a python script that asks for a user to input the url for a google scholar profile, and then it writes publication
KKBA Intruoduction A low-code tool that generates python crawler code based on curl or url Requirement Python = 3.6 Install pip install kkba Usage Co
A Python Covid-19 cases tracker that scrapes data off the web and presents the number of Cases, Recovered Cases, and Deaths that occurred because of the pandemic.
Dictionary - Application focused on word search through web scraping, in addition to other functions such as dictation, spell and conjugation of syllables.
snallygaster Finds file leaks and other security problems on HTTP servers. what? snallygaster is a tool that looks for files accessible on web servers
Pixiv-spider A Pixiv spider module WARNING It's an unfinished work, browsing the code carefully before using it. Features 0004 - Readme.md updated, co
ChromiumJniGenerator - Jni Generator module extracted from Chromium project
GetTss python Package extract gene TSS site form gencode/ensembl/gencode database GTF file and export bed format file. Install $ pip install GetTss Us
scraper-dblp Get paper names from dblp.org and store them in a .txt file Useful for a related literature :) Install libraries pip3 install -r requirem
Proxy Scraper A simple proxy scraper that utilizes the requests module in python. Usage Depending on your python installation your commands may vary.
Scrape-Product-Aliexpress A tool can scrape product in aliexpress: Title, Price, and URL Product. Usage: 1. Install Python 3.8 3.9 padahal halaman ins
BiliComment A spider for BiliBili comment. Spider Usage Put config.json into config directory, and then python . ./config/config.json. A example confi
Introduction This was supposed to be a web scraping project, but somehow I've turned it into a spamming project.
Scrapy Overview Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pag
WaGpScraper A Python Oriented tool to Scrap WhatsApp Group Link using Google Dork it Scraps Whatsapp Group Links From Google Results And Gives Working
memey A simple reddit scraper to get memes (only images) from r/ProgrammerHumor. Note Only works if you have firefox installed (yet). Instructions foo
HITsz Daily Report 基于 GitHub Actions 的「HITsz 疫情系统」访问入口 定时自动上报脚本，开箱即用。 感谢 @JellyBeanXiewh 提供原始脚本和 idea。 感谢 @bugstop 对脚本进行重构并新增 Easy Connect 校内代理访问。
19 Dec 07, 2022
39 Dec 28, 2022
0 Jan 22, 2022
1 Dec 12, 2021
8 Sep 20, 2021
1 Nov 13, 2021
2 May 09, 2022
2k Dec 28, 2022
1 Nov 14, 2021
4 Jun 12, 2022
6 Nov 21, 2022
1 Dec 07, 2021
3 Sep 08, 2021
1 Dec 30, 2021
3 Jul 05, 2021
1 Jan 23, 2022
45.5k Jan 07, 2023
27 Dec 18, 2022
2 Nov 16, 2021
56 Nov 27, 2022