CodeIgniter Forums
CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - Printable Version

+- CodeIgniter Forums (https://forum.codeigniter.com)
+-- Forum: Archived Discussions (https://forum.codeigniter.com/forum-20.html)
+--- Forum: Archived Development & Programming (https://forum.codeigniter.com/forum-23.html)
+--- Thread: CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with (/thread-55786.html)

Pages: 1 2


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Jeffrey McManus[/eluser]
I'm trying to use CodeIgniter to display some unicode data stored in MySQL. I am able to successfully query and display the data using PHP/mysql but the unicode data does not display correctly when I query it using CodeIgniter.

Here is the SQL I used to create the table:
Code:
CREATE TABLE `languages` (
  `id` char(2) NOT NULL DEFAULT '' COMMENT 'The ISO code for this language.',
  `name` varchar(40) NOT NULL COMMENT 'The name of the language in English.',
  `local_name` varchar(40) NOT NULL DEFAULT '' COMMENT 'The name of the language in that language',
  `sequence` int(11) DEFAULT '0',
  `created` datetime NOT NULL DEFAULT '0001-01-01 00:00:00',
  `modified` datetime NOT NULL DEFAULT '0001-01-01 00:00:00',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Here is the code I used to successfully query and output data from the table using PHP/mysql:
Code:
if($cn = mysql_connect('localhost', 'xxxx', 'xxxx')) {
echo "Connected to db.<br />";
} else {
die("Could not connect to db.");
}

mysql_select_db('mydb');
$result = mysql_query("SELECT local_name FROM languages;");

while($row = mysql_fetch_array($result)) {
echo $row['local_name'];
echo "<br />";
}
Here is the code I used to query and output the code using CodeIgniter:
Code:
function index() {
        echo "<!doctype html>\n";
        echo "&lt;html&gt;&lt;head>&lt;title&gt;test&lt;/title&gt;\n";
        echo "&lt;meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" /&gt;\n";
        echo "&lt;/head&gt;";
        echo "&lt;body&gt;";
        $langs = $this->db->get('languages')->result();
        foreach($langs as $lang) {
                echo $lang->local_name;
                echo "<br />";
        }
        echo "&lt;/body&gt;&lt;/html>";
}
My CodeIgniter config/config.php setting contains this setting:
Code:
$config['charset'] = 'UTF-8';
My CodeIgniter config/database.php contains these settings:
Code:
$db['default']['char_set'] = 'utf8';
$db['default']['dbcollat'] = 'utf8_general_ci';
I spent literally all day yesterday twiddling with various settings for MySQL, Apache, PHP, and CodeIgniter, but the fact that the output works using PHP/mysql and doesn't work with the equivalent code in CodeIgniter suggests to me that the problem exists at the CodeIgniter level. (Although I should add that I am only seeing the problem when I run the code on my Ubuntu testing machine -- when I run similar code on my OSX development machine the CodeIgniter version of the code does run correctly.)

Any ideas for something else to try to get this to work?




CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Narf[/eluser]
The answer is pretty simple - don't echo, use view files. Smile

Nowadays, setting the character set via meta tags isn't really effective - most browsers seem to just ignore it. CodeIgniter takes care of that in a real HTTP Content-Type header, using the value that you've set with $config['charset'], but when you echo data from your controller - it gets outputted before response headers are sent and that breaks the whole thing.


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Jeffrey McManus[/eluser]
I don't think the fact that I'm echoing code from a controller function is the problem. The problem happens when I use a view to output the data as well.

Also, the HTTP Content-Type header is actually being sent correctly (according to Chrome, it's "text/html; charset=utf-8" which is what I expected).


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Narf[/eluser]
While headers will be sent (and received by the browser), any output, including echoing from the controller, will cause a problem. If you have the same problem when using a view, then you probably have some white space characters outputted (preceeding a &lt;?php tag maybe?).

Alter your php.ini settings to set display_errors to 1 and error_reporting to E_ALL - if I'm correct, then you'll see numerous PHP E_WARNING (or E_NOTICE, not sure) messages saying that output is already sent when header() is executed.


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Jeffrey McManus[/eluser]
> any output, including echoing from the controller, will cause a problem

Why? Why is this a problem echoing from CI when it's not a problem echoing the identical data from PHP/mysql?

> you probably have some white space characters outputted

It's difficult to see how something like this could be the problem, since it's happening consistently with several separate files.

I set error_reporting to E_ALL and display_errors to 1. No change and no error messages.


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Narf[/eluser]
[quote author="Jeffrey McManus" date="1352492256"]> any output, including echoing from the controller, will cause a problem

Why? Why is this a problem echoing from CI when it's not a problem echoing the identical data from PHP/mysql?
[/quote]

Because your "pure PHP/mysql" script either doesn't send custom headers or sends them at its very start, while CodeIgniter will send headers after your controller code is executed.

This also answers your other question:

[quote author="Jeffrey McManus" date="1352492256"]
> you probably have some white space characters outputted

It's difficult to see how something like this could be the problem, since it's happening consistently with several separate files.[/quote]

I don't know why you're not seeing any error_messages though, unless you've set your environment to 'production'.


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Jeffrey McManus[/eluser]
My PHP/mysql script does not alter any HTTP headers, that is correct.

So if CI is really attempting to send headers after the controller code is executed (which seems wrong to me), how should I proceed? Remember that I'm seeing this error whether I echo the UTF8 data from a controller function or from within a view.

By "set your environment to production," what do you mean? Are you referring to configuration groups in database.php?


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Narf[/eluser]
[quote author="Jeffrey McManus" date="1352494004"]My PHP/mysql script does not alter any HTTP headers, that is correct.

So if CI is really attempting to send headers after the controller code is executed (which seems wrong to me), how should I proceed? Remember that I'm seeing this error whether I echo the UTF8 data from a controller function or from within a view.
[/quote]

Well, at this point unless you show me your full controller and view(s) code, I can't really say anything more.

[quote author="Jeffrey McManus" date="1352494004"]
By "set your environment to production," what do you mean? Are you referring to configuration groups in database.php?[/quote]

No, there's an ENVIRONMENT constant that is defined in index.php. If you choose 'production' there, you'll see that it also disables error reporting.


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Jeffrey McManus[/eluser]
Ah, I see. No, my environment is set to development.

Here is the full source of my test controller:

http://files.codelesson.com/beta.codelesson.com/test.php.txt

The output of this controller is here:

http://beta.codelesson.com/test


CI can't display UTF8 data that equivalent PHP/mysql code has no trouble with - El Forum - 11-09-2012

[eluser]Narf[/eluser]
You DO have to switch to using views ... It won't get fixed any way if you're still echoing from the controller (unless you change your default apache charset).