Welcome Guest, Not a member yet? Register   Sign In
Errors when bot supplies its own session ID?
#1

(This post was last modified: 05-21-2016, 10:13 AM by skunkbad.)

I've got a new hook that logs my php and mysql errors, then emails them to me every ten minutes. It's only been in place for the last 12 hours, and I'm seeing Baiduspider causing some errors, but not sure what is going on. Take a look at the log entry:


Code:
THIS IS THE PHP ERROR WITH REQUEST HEADERS SHOWING BAIDUSPIDER:

#----
Request URI: /just-some-page
Request headers:
    Host = example.com
    User-Agent = Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
    Accept-Language = en-US
    Accept = */*
    Cookie = ci_session=fd13b9c5b8c8556eebefb8c8a6e7ed29abdacfc7;
#---
PHP WARNING #2 - Date/Time: 5/21/2016 08:33:19
File: /home/skunkbad/public_html/example.com/ci/system/core/Common.php
Line: 573
Message: Cannot modify header information - headers already sent by (output started at /home/skunkbad/public_html/example.com/ci/system/core/Output.php:528)
Error Count: 1
#--

THIS IS THE ASSOCIATED MYSQL ERROR:

Error Number: 1062
Duplicate entry 'fd13b9c5b8c8556eebefb8c8a6e7ed29abdacfc7' for key 'PRIMARY'
INSERT INTO `ci_sessions` (`id`, `ip_address`, `timestamp`, `data`) VALUES ('fd13b9c5b8c8556eebefb8c8a6e7ed29abdacfc7', '180.76.15.33', 1463844799, '__ci_last_regenerate|i:1463844798;')
Filename: libraries/Session/drivers/Session_database_driver.php
Line Number: 233

Now, I know in my application code that nowhere am I just blindly allowing people to supply their own session IDs, and entering them into the database as a valid session ID, so I'm wondering if this is the way CI sessions is supposed to work? In the header it appears that Baiduspider is supplying a session ID, but in the MySQL error it looks like the session already exists and is expired. In a normal browser the expired session would be dropped, and this problem wouldn't exist, but why does CI blindly want to enter the session into the sessions table?

In the database I see the session with matching Id, and it has a timestamp a second or two before these errors: 1463844797

Enlighten me ....
Reply
#2

I can reproduce this error by running this a few times via CLI:


PHP Code:
<?php
$request_headers 
= array();
$request_headers[] = 'Cookie: ci_session=5f090575bb8411bdf1626212fef8d23f4cbaba6d';

$c curl_init('https://example.com'); // Replace with actual site URL
curl_setopt($cCURLOPT_VERBOSE1);
curl_setopt($cCURLOPT_HTTPHEADER$request_headers);
curl_setopt($cCURLOPT_RETURNTRANSFER1);
$page curl_exec($c);
curl_close($c);

echo 
$page
Reply
#3

Brian, I can't address the internal function of CI, but this may be caused by baiduspider requesting a second load of your page. First request via your url and then a second request via some other function?

Just a guess. It is only this one spider that produces the error, right?
CI 3.1 Kubuntu 19.04 Apache 5.x&nbsp; Mysql 5.x PHP 5.x PHP 7.x
Remember: Obfuscation is a bad thing.
Clarity is desirable over Brevity every time.
Reply
#4

(05-24-2016, 09:16 AM)twpmarketing Wrote: Brian, I can't address the internal function of CI, but this may be caused by baiduspider requesting a second load of your page.  First request via your url and then a second request via some other function?

Just a guess.  It is only this one spider that produces the error, right?

No. Googlebot, bingbot, Yahoo slurp, Majestic, etc, etc.

The issue is that they keep reusing the same session ID, and they are bots so they are too stupid to update the session ID. When they send the same session ID twice, CodeIgniter can't handle that because for some reason CI thinks it needs to insert a new record in the database, but it already exists.

I can blacklist a bot from sessions, but maybe there is a better solution from Narf or somebody else.

This issue is only new for me because I started using a custom error handler that sees these errors on production. I'm kicking myself because this could have been happening for many months since I upgraded to CI3, or maybe even longer.
Reply
#5

Ok, I'll follow this and hope to learn what causes the glitch.
Thanks
CI 3.1 Kubuntu 19.04 Apache 5.x&nbsp; Mysql 5.x PHP 5.x PHP 7.x
Remember: Obfuscation is a bad thing.
Clarity is desirable over Brevity every time.
Reply
#6

Another thing that is strange is that it isn't happening on the dev server with PHP7.
Reply
#7

Is your dev server connected to the net? Mine is totally local, so I don't see outside traffic until I do the upload to a live server.
CI 3.1 Kubuntu 19.04 Apache 5.x&nbsp; Mysql 5.x PHP 5.x PHP 7.x
Remember: Obfuscation is a bad thing.
Clarity is desirable over Brevity every time.
Reply
#8

(05-24-2016, 10:40 AM)twpmarketing Wrote: Is your dev server connected to the net?  Mine is totally local, so I don't see outside traffic until I do the upload to a live server.

No, it's just local. I'm just running my script locally, and not sure if that's why there's a difference.

FWIW, the query that determines if there already is a row in the DB for a specific session ID is executed on line 166 of Session_database_driver.php. I don't really want to do testing on the production environment, but I might put the production website in maintenance mode super late tonight and give it a shot.
Reply
#9

I assume you are using your name-brand site for this test?

Is this occurring on any other of your live websites?

I know your new hook has not been available until recently, but it might be worth a test if you have another site that could be updated.

I know I'm not offering real solutions, but perhaps my questions may help... I get brain lockup sometimes, when I'm too tired to think straight<g>.
CI 3.1 Kubuntu 19.04 Apache 5.x&nbsp; Mysql 5.x PHP 5.x PHP 7.x
Remember: Obfuscation is a bad thing.
Clarity is desirable over Brevity every time.
Reply
#10

(This post was last modified: 05-24-2016, 03:41 PM by skunkbad.)

I own a sewing machine store, and the errors were coming from the store's production website. Since I implemented the new error reporting hook (a few days ago) I started getting emails associated with this error. So I started blacklisting the bots from the session, one by one. Today I thought I might try to track down what the error is, and I thought I had a good idea of what it might be, but now all the sudden I can't get the website to give me an error. You can imagine the frustration.

As a test, I fresh installed CodeIgniter 3.0.6 with Community Auth on an old domain of mine. No problems. Wen't to my store's website, played around with things, and the last thing I tried was renaming the session from ci_session to ciSess. Problem went away. I thought great, let me switch it back to verify that the problem comes back. Nope, still no problem. Seriously, I didn't do anything, and now I can't reproduce the problem.

Error reporting hook:


PHP Code:
<?php
defined
('BASEPATH') or exit('No direct script access allowed');

class 
Tvsc_error_handler {

    /**
     * Count of errors during execution
     * @var integer
     */
    private $error_count 0;

    /**
     * How many errors are allowed before stopping execution
     * @var integer
     */
    private $error_limit 25;

    /**
     * The human readable type of error that occured
     * @var string
     */
    private $error_type '';

    /**
     * What environments the error handler should be applied to
     * @var [type]
     */
    private $environments_handled = [
        //'development',
        'production'
    ];

    // -----------------------------------------------------------------------
    
    
/**
     * CodeIgniter pre_system hook calls this method, 
     * and this method sets up the error handler.
     */
    public function set_tvsc_error_handler()
    {
        ifin_arrayENVIRONMENT$this->environments_handled ) )
        {
            set_error_handler( [ $this'production_errors'], E_ALL & ~E_DEPRECATED );
        }
    }

    // -----------------------------------------------------------------------
    
    
/**
     * This is the callback for the error handler.
     */
    public function production_errors$e_number$e_message$e_file$e_line$e_vars )
    {
        $this->error_count++;

        // Start log entry
        $log_entry '';

        // We only want the request info once
        if$this->error_count == )
        {
            // Include request URI in log entry
            $log_entry .= PHP_EOL '#----' PHP_EOL 'Request URI: ' $_SERVER['REQUEST_URI'] . PHP_EOL;

            // Include request headers in log entry
            if$request_headers apache_request_headers() )
            {
                $log_entry .= 'Request headers:' PHP_EOL;

                foreach$request_headers as $k => $v )
                {
                    $log_entry .= "\t" $k ' = ' $v PHP_EOL;
                }
            }

            // Include post vars in log entry
            if( isset( $_POST ) && ! empty( $_POST ) )
            {
                $log_entry .= 'POST vars:' PHP_EOL;

                foreach$_POST as $k => $v )
                {
                    $log_entry .= "\t" $k ' = ' $v PHP_EOL;
                }
            }

            $log_entry .= '#---' PHP_EOL;
        }
        else
        
{
            $log_entry .= '#---' PHP_EOL;
        }

        $this->set_error_type$e_number );

        // Include the error information
        $log_entry .= 'PHP ' $this->error_type ' #' $e_number ' - Date/Time: ' date('n/j/Y H:i:s') . PHP_EOL .
                    'File: ' $e_file PHP_EOL .
                    'Line: ' $e_line PHP_EOL .
                    'Message: ' $e_message PHP_EOL;

        // Include error count in log entry
        $log_entry .= 'Error Count: ' $this->error_count PHP_EOL;

        // Finish log entry
        $log_entry .= '#--' PHP_EOL;

        // Second param (3) says to store error in specified log file
        error_log$log_entry3APPPATH 'logs/php_errors/php_errors.log' );

        // If too many errors of any type, die()
        if$this->error_count $this->error_limit )
        {
            die('<h1>System Error X' $this->error_count '</h1>
                <p>A system error occurred. We apologize for the inconvenience.</p>'
);
        }

        // Continue to execute PHP internal error handler
        return FALSE;
    }

    // -----------------------------------------------------------------------

    /**
     * This method maps the error number to a human readable error type.
     */
    private function set_error_type$e_number )
    {
        switch$e_number )
        {
            case E_PARSE:
            case E_ERROR:
            case E_CORE_ERROR:
            case E_COMPILE_ERROR:
            case E_USER_ERROR:
                $this->error_type 'FATAL ERROR';
                break;
            case E_WARNING:
            case E_USER_WARNING:
            case E_COMPILE_WARNING:
            case E_RECOVERABLE_ERROR:
                $this->error_type 'WARNING';
                break;
            case E_NOTICE:
            case E_USER_NOTICE:
                $this->error_type 'NOTICE';
                break;
            case E_STRICT:
                $this->error_type 'STRICT';
                break;
            case E_DEPRECATED:
            case E_USER_DEPRECATED:
                $this->error_type 'DEPRECATED';
                break;
            default:
                $this->error_type 'UNKNOWN ERROR TYPE';
                break;
        }
    }

    // -----------------------------------------------------------------------


Reply




Theme © iAndrew 2016 - Forum software by © MyBB