Welcome Guest, Not a member yet? Register   Sign In
One more question about User Agent Library - what to put in robots array
#1

[eluser]tinawina[/eluser]
If i get a robot that looks like this:

Code:
SapphireWebCrawler/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; [email protected])

what do i enter into my robots array in the /config/user_agents file? Would this work?:

'sapphire' => 'Sapphire Web Crawler using Nutch'

Thanks!




Theme © iAndrew 2016 - Forum software by © MyBB