Welcome Guest, Not a member yet? Register   Sign In
Scraping and Automation in CodeIgniter
#1
Lightbulb 
(This post was last modified: 09-03-2022, 02:47 AM by jason06.)

Hey There!

Basically, I want to make a website that can scrape data from other websites without using an API in the same way that cURL in PHP scrapes data from other websites. I would like to know if it is possible to do this type of programming in CodeIgniter. My final year project will be a CodeIgniter project that focuses on automation and that I wish to do in CodeIgniter. Would anyone please guide me in this regard.

Thanks.
Reply
#2

You will probably need some additional libraries like Guzzle to load a webpage to scrape and XPath to extract data to scrape. You can use these libraries in Codeiginiter and process the scraped data.
You'll find that you need to write specific code for each website you want to scrape, I haven't seen an automated scraper with a user interface.
There are some examples here:
https://www.freecodecamp.org/news/web-sc...web-pages/
Reply
#3

Thanks for your guidance!
Reply




Theme © iAndrew 2016 - Forum software by © MyBB