NOTE:: Before reporting an issue, make sure you are running the latest version, currently 3.3.1
|Anonymous | Login | Signup for a new account||26-07-17 13:41 BST|
|My View | View Issues | Change Log | Roadmap|
|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0001644||phplist application||HTML Email Support||public||30-08-04 04:27||17-05-11 15:48|
|Target Version||4.0.x||Fixed in Version||2.11.6|
|Summary||0001644: Subject error with UTF-8 encode in Traditional Chinese|
|Description||I'm trying to use phplist in UTF-8 enviornment with Traditional Chinese language. Everything seems OK but the subject. Some characters of the subject were be translated to no meanful ones. Anyone could please tell me where or which of the scripts I can trying to fix this problem?? I'm still looking into the code..|
|Additional Information||Version: 2.8.11|
I've modify the encode of the language file, config for both text and html emails to UTF-8.
|Tags||No tags attached.|
|Attached Files||utf8_fix_for_svn_r1703.diff [^] (3,378 bytes) 21-01-10 10:13 [Show Content]|
|As I wouldn't know how to solve that, do you have any tips of how to make that work?|
I found the problem.
Line 869, 875 in the file 'lists/admin/send_core.php', the script try to use htmlentities function for process characters. In my enviornment, it would be better if use this function in following format:
htmlentities($subject, ENT_QUOTES, $_SESSION['adminlanguage']['charset'])
I don't know if this cause other problems in other enviornments. I could always solve the same problem in other page. :)
|Did this get resolved in 2.10?|
|instead of using the admin language from the session, I've hardcoded UTF-8, because it may as well be that someone has the interface in english, but wants to send chinese. Let's see if that sorts it|
gives me problems with Swedish special characters (åäö). The message subject just disappears when I click "Save Changes". I guess my input is in ISO-8859-1, because
fixes it. So does
which looks even nicer to me. Wouldn't that work for Chinese too?
A somewhat similar issue involving PHP 5.2.5 was discussed in the PHP bug tracker: http://bugs.php.net/bug.php?id=43549 [^]
This discussion seems to indicate that if you set htmlentities to UFF-8, you'll need to make sure that the charset for the html page containing the form is also set to UTF-8.
Currently the 'send a message' page produced by phplist is set to:
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
On my system, entering special characters in the subject field did not make the subject field disappear completely. Only the special characters disappeared, while normal characters still where displayed, e.g.: "This is a Tést" would display as "This is a Tst".
So I ran a quick test which confirms my previous note, i.e., you need to set the content-type of the HTML page holding the input form fields to UTF-8 if you want to htmlentities($subject,ENT_QUOTES,'UTF-8') to work correctly, i.e., if <meta http-equiv="content-type" content="text/html; charset=utf-8" /> is used, the subject field "This is a Tést" would display in full in the received text and html message.
To do so I had to change $strCharSet in english.inc from ISO-8859-1 to utf-8. Other language files than english.inc would probably need recoding all special characters to UTF-8.
My test system is configured as follows.
- Charset for HTML messages: UTF-8
- Charset for Text messages: UTF-8
- $language_module = "english.inc"; # with this change in english.inc: $strCharSet = 'utf-8';
MySQL 4.1.12 - with *database encoding* set to: utf8_unicode_ci
- I expected the charset for the backend's html pages to be defined by the settings in languages.php, e.g.:
"en" => array("English ","iso-8859-1","iso-8859-1, windows-1252 "),
This is not the case, and I'm not sure what exactly the language.php charset settings are used for.
- I wonder whether it is a good idea to hardcode UTF-8 anywhere in the code. It would seem more flexible to have the charset configurable, e.g. through the charset defined on the configuration page.
- Inclusion of UTF-8 encoded .inc language files should perhaps be considered for future phplist releases, along with the existing iso-* encoded files.
- Installation procedures (and documentation) could perhaps include giving the user a choice of charsets to use for database encoding.
I found an interesting article which identifies different aspects that come into play in a PHP/MySQL/UTF-8 application. These are the principal ones:
- the database (individual tables + any text columns) should be set to UTF-8
- the PHP server should send a header telling the browser to expect UTF-8, e.g.:
header('Content-Type: text/html; charset=utf-8' );
- the HTML page's Content-Type should be set to UTF-8, i.e.:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- and, the PHP-MySQL connection should be set to UTF-8, since it will otherwise default to latin1
The article also provides a useful solution (based on SET NAMES) and some code examples.
For more info, please see: http://www.adviesenzo.nl/examples/php_mysql_charset_fix/ [^]
Additional server related info from my test system, collected by using the following query in phpMyAdmin: SHOW VARIABLES LIKE 'character_set_%'
|great, that's very helpful research. thanks for that.|
Glad to help.
Just a few additional comments regarding hardcoding the charset. Currently UTF-8 has been hardcoded in the following files (v2.10.9):
$message = html_entity_decode($message,ENT_QUOTES,'UTF-8');
$line = html_entity_decode($line,ENT_QUOTES,'UTF-8');
$text = html_entity_decode ( $text , ENT_QUOTES , 'UTF-8' );
$this->Body = html_entity_decode($text ,ENT_QUOTES, 'UTF-8' ); #$text;
$this->AltBody .= html_entity_decode($text ,ENT_QUOTES, 'UTF-8' );#$text;
$this->Body .= html_entity_decode($text ,ENT_QUOTES, 'UTF-8' );#$text;
So, the problem will probably not only occur in the subject line, but most likely also in the From: (name) line, and the forward subject line. (Haven't checked the forum for reports on this yet).
It seems to me that instead of hardcoding UTF-8, it might be an improvement if all hardcoded instances of 'UTF-8' were replaced by something like $GLOBALS['strCharSet']
In that case, it is the .inc language file's encoding that will determine the charset for the whole phplist system (frontend, backend, and backend input fields) except for the Charset settings for HTML and Text messages on the configuration page, and except for the encoding for the database and the database connection.
While this would be an improvement, it is not yet an ideal situation, considering that things may still go wrong if the database encoding and/or database connection isn't compatible with the charset, or if the user forgets to change the charset used for message encoding (configuration page) for instance.
I guess the best way to solve this, would be to have a phplist installation script give the user a choice from a number of charsets. The installation script should then make sure the whole system, including the database/db connection, is made ready for use under the chosen charset.
Probably related to:
Possibly related to:
Also found a report on the forum which seems to confirm the issue also occurs when the name in the From: line contains special characters.
See: http://forums.phplist.com/viewtopic.php?p=61323#61323 [^]
A useful suggestion from the forum:
==== Start Quote ====
to support encoding properly you should add these lines:
==== End Quote ====
A similar problem appears in revision 1703 from the svn.
I am trying to write in Spanish but I think it applies to other languages also.
I attach a patch (I think it is not a definitive patch but a workaround) to solve this problem.
In my opinnion most of the problems that I have might come from the fact that, whatever the reason is, pagetop page is not included in any of the admin pages.
But I am not quite sure because I am not an expert on this utf8 issues.
After reading carefully all the expalnations and trying everything described here,
I still have the same problem described above.
When writing the subject in Hebrew, strange signs apear and the text is corrupted.
The same happens with the "From".
I have chosen the laguage text "hebrew-utf8"
I set the Charset for HTML messages = UTF-8, Charset for Text messages = UTF-8
And still no use.
|For more in-depth info on MySQL encoding pitfalls and solutions, see: http://mysql.rjweb.org/doc.php/charcoll [^]|
@haipo: since v2.10.11, you will also need to set the charset of admin interface pages to UTF-8.
See this forum post for more info: http://forums.phplist.com/viewtopic.php?p=80089#p80089 [^]
|Copyright © 2000 - 2017 MantisBT Team|