Skip to content

Latest commit

 

History

History
26 lines (22 loc) · 911 Bytes

README.md

File metadata and controls

26 lines (22 loc) · 911 Bytes

skrapy

AI powered web scrapper built on selenium

Installaion

  1. Clone the repository
git clone https://github.com/bakayu/skrapy.git
cd skrapy
  1. Run the following command to install the dependencies (using a virtual env/ conda env is recommended)
pip -r install requirements.txt

Note: Skrapy utilizes a local LLM for parsing data, ollama and an appropriate model should be installed on your system in order to use the web scrapper, update llm.py according to the model you want to use.

  1. Run the following command to start the web scrapper
streamlit run main.py

Usage

  1. Enter a url and click on Scrape site to start scrapping the website image1
  2. Enter instructions for extracting the data in the prompt section image2