CodeIgniter Forums
Preventing site scraping? - Printable Version

+- CodeIgniter Forums (https://forum.codeigniter.com)
+-- Forum: Archived Discussions (https://forum.codeigniter.com/forumdisplay.php?fid=20)
+--- Forum: Archived Development & Programming (https://forum.codeigniter.com/forumdisplay.php?fid=23)
+--- Thread: Preventing site scraping? (/showthread.php?tid=44173)



Preventing site scraping? - El Forum - 08-06-2011

[eluser]cwscribner[/eluser]
Hi all.

Just out of curiosity, is there a mechanism in CI to prevent site scraping? I've been seeing a lot of buzz around people's sites being scraped and wanted to know if there was a protection method.


Preventing site scraping? - El Forum - 08-06-2011

[eluser]Ayeyermaw[/eluser]
As far as I know there is no effective way to stop people scraping your site.

Have a look at this: http://blockscraping.com/prevent-scraping.html


Preventing site scraping? - El Forum - 08-06-2011

[eluser]Jaketoolson[/eluser]
A lot of my job requires me to scrape public data made available on public websites. Every so often I come across a site that is 'nearly' impossible to scrape. This is usually when the data I want to scrape is 'appended' or 'inserted' into the DOM using Javascript <em>after</em> the page has loaded and is therefore not directly accessible.

A lot of sites use forms to display the data requested but when a hash or token is required for the form to be changed, this usually requires an additional step or two on the scrapers part.

It is pretty difficult though to prevent. Not only do I run daily scrapes, I also manage a website which we dont want scraped! Some times I simply output charts and raw data to the page in an image form using a php image class.


Preventing site scraping? - El Forum - 08-09-2011

[eluser]xjohnson[/eluser]
Hi, Jaketoolson -

You said:
Quote:I simply output charts and raw data to the page in an image form using a php image class.

Can you share with me how to do the same thing?



Warm Regards,

xjohnson