Welcome Guest, Not a member yet? Register   Sign In
Job Board Scraper/Sample Application
#1

I've developed a small application to assist jobseekers. One of the problems with job boards is that their search results are presented as web pages, which are not in a data-friendly format. I thought it would be nice to search for jobs and get the resulting job listings as a spreadsheet that you can then further manipulate. That's what this application does. The application offers an interface to manage job search URLs, letting you save, edit, delete, and execute them. Executing a job search runs the search and automatically downloads a CSV file with the contents of the search to your local computer.

Since websites are coded differently, the application must "know" the website's code. For this, there's a class file for each website. For now, there are only class files for Indeed and Simply Hired. Adding more job boards will require coding class files for the additional boards, and if Indeed or Simply Hired change their pages' HTML, their class files will have to be modified too. 

You can see the application running here. http://www.clicketyhome.com/jobs

You can download the application here: https://github.com/RobertHallsey/cijobs If you have not used GitHub before, look for the button to the right that says Download Zip. Click it to download the application.

The application is complex enough that it could provide a good step up from the tutorials for those just starting with CodeIgniter, but is not so complex that it should overwhelm anyone basically familiar with PHP and MySQL.

I hope someone finds it useful, and if you see any bad practices in it, please let me know!
Hey, don't work without a PHP debugger. Several free IDEs have this features built in. Two are NetBeans and CodeLobster. Without a debugger, it's like you're driving with a blindfold on -- you are going to crash!
Reply
#2

Just wanted to say this line is really good:

PHP Code:
$url = ((preg_match($reg_next_url$page$matches) == 1) ? self::SITE $matches[1] : ''); 

Good work!!

Best wishes

Paul.
Reply
#3

(This post was last modified: 11-25-2015, 06:36 PM by PaulD. Edit Reason: minor typo )

Although I would mention that for 'indeed' in their terms and conditions cite you will not use:

Quote:...any portion of the web pages that are part of the Website without Indeed’s explicit permission

Just to be a pain.

Paul
Reply
#4
Smile 

(11-25-2015, 06:18 PM)PaulD Wrote: Just wanted to say this line is really good:

Wow, thank you!  Smile

Quote:Although I would mention that for 'indeed' in their terms and conditions cite you will not use:

Quote: Wrote:...any portion of the web pages that are part of the Website without Indeed’s explicit permission

Just to be a pain.

Oh, no worries. We should respect sites' terms and conditions, but I don't think this application does anything a normal visitor wouldn't do, and surely they don't mean people need explicit permission to run a search.
Hey, don't work without a PHP debugger. Several free IDEs have this features built in. Two are NetBeans and CodeLobster. Without a debugger, it's like you're driving with a blindfold on -- you are going to crash!
Reply
#5

I've updated the application to use PHP's DOMDocument instead of regex to scrape web pages. Smile
Hey, don't work without a PHP debugger. Several free IDEs have this features built in. Two are NetBeans and CodeLobster. Without a debugger, it's like you're driving with a blindfold on -- you are going to crash!
Reply




Theme © iAndrew 2016 - Forum software by © MyBB