View Issue Details

IDProjectCategoryView StatusLast Update
0006782phplist applicationSubscriber Importpublic23-05-12 02:53
ReporterRothgar 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
PlatformUnixOSFreeBSDOS Version5.3
Product Version2.10.2 
Target Version4.0.xFixed in Version2.11.7 
Summary0006782: Importing Invalid E-mails
DescriptionHi Guys,
We just imported a tab seperated userlist from our database and for the most part it worked great. It found quite a few invalid e-mails and we removed them from the database and re-exported.

When we imported it showed no invalid e-mails. However upon trying to do a mail-out it was a different story. I am not sure if the mail-out e-mail checked is different from the e-mail validator of the importer but it would be nice if we could fix the import function to better detect invalid e-mails.

Here are a few of the problems the importer missed:
A few of our e-mails were wrongly entered as mailto:e-mailaddress, perhaps you can search for either the whole string mailto: and remove it and then attempt to validate that part. Or simply flag it as invalid for having a ":" which should be an invalid character.

In a few cases there were two e-mail addresses in the e-mail field seperated by a "," or ";" semi-colon which are invalid e-mail characters but you could possibly make a function to split the string by "," or ";" semi-colon and attempt to validate both halves and if in the import you did not select "discard duplicates" you could add both?

Third issue I found was "'" single-quote which is an invalid character in e-mail addresses but it seemed to get past.

Last one It did not detect was double "@" in the form of somestring@somestring@domain I don't know if your validating would have detected "somestring@@domain" however it didn't detect the "somestring@somestring@domain" perhaps just cound the "@"'s

Those were the errors I came across anyway, thought I'd mention them so we can hopefully improve the import process. Thanks.

Regards,
Jason
TagsNo tags attached.

Relationships

related to 0002705 closed PHPList v2.11 release 
related to 0012866 resolvedmichiel Strict format checking of email and other fields prevents numerous imports, possible for mysql injection attacks 
related to 0013672 new how often to retry sending to an address that fails, and give up 
related to 0015632 resolvedmichiel TLDs missing from is_email() validation 
related to 0015207 resolveduser1822 Email validation doesn't seem to work for the local part of an email address 
related to 0015340 resolvedmichiel allows setting empty email address 

Activities

michiel

01-07-06 05:18

manager   ~0014993

yes, that will be useful to have a closer look at

donkiely

01-02-07 04:44

reporter   ~0023103

I've found a number of references that say that ' is a valid character:

http://en.wikipedia.org/wiki/E-mail_address
http://tools.ietf.org/html/rfc2822#section-3.2.4

The Wikipaedia entry explicitly says it, and the RFC seems to back it up, unless I'm misinterpreting it.

msauer

18-06-07 16:03

reporter   ~0028144

There is an error in the pattern for validation.
^([\&\'-_.[:alnum:]]+@)((...
should be
^([\&\'\-_.[:alnum:]]+@)((...

The dash is not escaped so all chars from ' to _ are valid.
Escaping the dash solves double @ issue and maybe some others.

michiel

19-06-07 13:08

manager   ~0028167

ah, nice find. Thanks!

michiel

23-06-07 00:41

manager   ~0028421

Hmm, that's not correct though. I just tried to validate an email with an - in it, and it failed when you escape the - in the regex.

msauer

25-06-07 06:44

reporter   ~0028435

Yes, I had the same problem. So I changed the complete pattern to
^[-!#$%&'*+/0-9=?A-Z^_a-z{|}~](\.?[-!#$%&'*+/0-9=?A-Z^_a-z{|}~])*@[a-zA-Z](-?[a-zA-Z0-9])*(\.[a-zA-Z](-?[a-zA-Z0-9])*)+$
from http://www.email-unlimited.com/stuff/email_address_validator.htm
I made a little change in the pattern: @[a-zA-Z] to @[a-zA-Z0-9] and used preg_match() instead of ereg().

The function is_email() now looks like this:
-----
function is_email($email) {
  if (isset($GLOBALS['config']) && isset($GLOBALS["config"]["dont_require_validemail"]) && $GLOBALS["config"]["dont_require_validemail"])
    return 1;

  $email = trim($email);

  # hmm, it seems people are starting to have emails with & and ' or ` chars in the name
  #'

$pattern =
"/^[-!#$%&'*+\/0-9=?A-Z^_a-z{|}~](\.?[-!#$%&'*+\/0-9=?A-Z^_a-z{|}~])*@[a-zA-Z0-9](-?[a-zA-Z0-9])*(\.[a-zA-Z](-?[a-zA-Z0-9])*)+$/";

  if(preg_match($pattern, $email))
    return(1);
  else
    return(0);
}
-----

That works fine for me.

user1822

05-07-07 14:45

  ~0028933

In PHPlist function is_email in userlib has been updated to support full RFC standard. new global constant EMAIL_ADDRESS_VALIDATION_LEVEL is set to define this.

chrschaffer

07-08-07 08:56

reporter   ~0030124

Hi there,
good to hear that this issue is being resolved in 2.11.4. When is 2.11.x going to be stable officially?
Thanks,
Chris

michiel

10-08-07 18:22

manager   ~0030376

once our office has heating and some desks

user1822

27-08-07 13:54

  ~0031071

There are reasons to accept non emails. Some users use this field as a username and fax their users for instance. So checking should be optional.

Since the import functionality will be redesigned in the next version (2.11) we will not fix this issue now. In defining the requirements for the redesign we will certainly consider your remarks.

michiel

23-05-12 02:53

manager   ~0051592

email address validation continues to be a tricky issue due to those lovely RFC people accepting way too many complicated things. If only they'd kept it simple, that would have saved me, and many other months of our time.