View Issue Details

IDProjectCategoryView StatusLast Update
0020124phpList 3 applicationBounce Managementpublic27-12-19 22:14
ReporterMANTISFCKD Assigned To 
PriorityhighSeverityminorReproducibilityhave not tried
Status newResolutionopen 
Product Version3.4.8 
Target Version3.4.9 
Summary0020124: Database error 1366 and 1064 after processing bounces
DescriptionStill not resolved.. as suggested, the database tables have been checked ( they are OK!!). Indexes rebuilt, OK.

still getting this

Processed messages will be deleted from mailbox
Database error 1366 while doing query Incorrect string value: '\xFE\xFE\xEC\xE0 \xF0...' for column 'header' at row 1
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '16:57 )' at line 13
TagsNo tags attached.

Activities

MANTISFCKD

03-12-19 19:14

reporter   ~0062886

see more details at https://mantis.phplist.org/view.php?id=20123

Database Type mysql
Database Version 5.7.28
Database Collation utf8_general_ci
Database Connection Collation utf8mb4_general_ci
PHP Version 7.3.12

MANTISFCKD

03-12-19 21:10

reporter   ~0062887

Last edited: 04-12-19 18:52

View 2 revisions

more of the same messages as before ...

=====

Processed messages will be deleted from mailbox
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '07:43 )' at line 13
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '08:49 )' at line 13




300 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
48 bounces processed by advanced processing
2328 bounces were not matched by advanced processing rules
Identifying consecutive bounces
unsubscribing 3813 -> 3 bounces
... ... ... ... ... (etc)
unsubscribing 15807 -> 3 bounces
total of 1547 subscribers processed

MANTISFCKD

03-12-19 21:42

reporter   ~0062888

Last edited: 03-12-19 21:45

View 2 revisions

sometimes i get the error - other times I don't (testing with batches of 200 bounces now) ...

I'm wondering if the glitch is somewhere in the "advanced processing section" - since when that doesn't trigger, I don't see the error (COINCIDENCE?)

200 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
0 bounces processed by advanced processing
3204 bounces were not matched by advanced processing rules
Identifying consecutive bounces
total of 1632 subscribers processed

MANTISFCKD

04-12-19 18:52

reporter   ~0062889

Couple small batches went OK, but the last 300 batch yielded these messages

Processed messages will be deleted from mailbox
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '07:49 )' at line 13
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '07:49 )' at line 13

again, this one had some advanced processing happen:

300 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
4 bounces processed by advanced processing
3920 bounces were not matched by advanced processing rules
Identifying consecutive bounces (...)

MANTISFCKD

04-12-19 19:05

reporter   ~0062890

Last edited: 04-12-19 19:06

View 2 revisions

again, another run with no advanced processing yields no error messages:

200 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
0 bounces processed by advanced processing
4142 bounces were not matched by advanced processing rules

PS: why do I have so many bounces?? Well, when we migrated servers last week I did a clean 3.4.8 installation and imported the previously exported user lists.
Unfortunately, that brought in about 3000 previously blacklisted members that didn't get their blacklist status onto the new system. Thus the last few mailers are generating a ton of bounces... :(

MANTISFCKD

04-12-19 19:16

reporter   ~0062891

Hmmm... of course, it's a pesky intermittent problem based on content - only a certain few of the 'advanced processing' seem to trigger the error:
( this one had two and no errors, earlier posts show 48 and two errors, etc etc)
I can see why the earlier ticket was marked 'unreproduceable' - this is clearly some sort of edge case that requires a lot of testing!

375 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
2 bounces processed by advanced processing
4517 bounces were not matched by advanced processing rules

...

duncanc

06-12-19 12:23

updater   ~0062892

The mysql 1064 error is the same as reported here https://mantis.phplist.org/view.php?id=20117

The 1366 error seems to indicate a problem with the "header" column in the bounce table. Possibly it is not defined as UTF8.

duncanc

06-12-19 14:49

updater   ~0062893

But he values in the error message are not valid UTF8. 0xFE is not a valid byte in UTF8.

Incorrect string value: '\xFE\xFE\xEC\xE0

MANTISFCKD

18-12-19 22:55

reporter   ~0062913

Same story today:


50 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
0 bounces processed by advanced processing
7909 bounces were not matched by advanced processing rules
Identifying consecutive bounces
unsubscribing 1542 -> 3 bounces
unsubscribing 3578 -> 3 bounces
unsubscribing 3690 -> 3 bounces
unsubscribing 4270 -> 3 bounces
unsubscribing 6499 -> 3 bounces
unsubscribing 7715 -> 3 bounces
unsubscribing 8833 -> 3 bounces
unsubscribing 8843 -> 3 bounces
unsubscribing 9345 -> 3 bounces
unsubscribing 9939 -> 3 bounces
unsubscribing 13371 -> 3 bounces
unsubscribing 13389 -> 3 bounces
total of 104 subscribers processed

Processed messages will be deleted from mailbox
Database error 1366 while doing query Incorrect string value: '\xFE\xFE\xEC\xE0 \xF0...' for column 'header' at row 1
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '22:01 )' at line 13

========

Incidentally, PHPlist database Verify function reports that all tables are good...
All tables are InnoDB utf8_general_ci

duncanc

19-12-19 08:35

updater   ~0062914

The 1064 error is fixed in the latest release of phplist, 3.4.9.

As I mentioned before, the incorrect string value \xFE\xFE\xEC\xE0 \xF0 doesn't look to be valid UTF8, so the error message might be caused by bad data. Are you able to get the contents of the actual bounce email itself?

MANTISFCKD

19-12-19 16:11

reporter   ~0062915

Have not been able to IDENTIFY any specific bounce email causing this! For years, been getting roughly the same typical hundred or two bounces every mail ( we run about 15,000 sends once or twice a week).
Never ever saw this issue before the 3.4.8 version of PHPlist... Anyway, the whole idea of database software is that it doesn't ALLOW any 'bad data' to get thrown at the database, right? :)

MANTISFCKD

19-12-19 19:23

reporter   ~0062916

Also, please note this 'incorrect string value' is not the only glitch we are seeing - it seems some of the code takes exception to 'something else' as well... (we don't always see the 'incorrect string value')


25 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
2 bounces processed by advanced processing
7935 bounces were not matched by advanced processing rules
Identifying consecutive bounces
unsubscribing 1023 -> 3 bounces
total of 92 subscribers processed

Processed messages will be deleted from mailbox
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '01:49 )' at line 13

MANTISFCKD

25-12-19 20:40

reporter   ~0062919

yesterday's run:

Process bounces

125 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
0 bounces processed by advanced processing
8205 bounces were not matched by advanced processing rules
Identifying consecutive bounces
unsubscribing 1009 -> 3 bounces
unsubscribing 3602 -> 3 bounces
unsubscribing 7851 -> 3 bounces
unsubscribing 10302 -> 3 bounces
unsubscribing 10367 -> 3 bounces
total of 308 subscribers processed

Processed messages will be deleted from mailbox
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '08:31 )' at line 13
Database error 1064 while doing query You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '09:21 )' at line 13

MANTISFCKD

25-12-19 20:43

reporter   ~0062920

Last edited: 25-12-19 20:44

View 2 revisions

PS: 3.4.9 has to be a manual upgrade?? I get this on running "upgrade":

Upgrade phpList
Your database version: 3.4.8

Your database is already the correct version, there is no need to upgrade

duncanc

26-12-19 10:25

updater   ~0062923

The "upgrade phplist" menu is to upgrade the database. You need to upgrade the phplist code first.
See the manual https://www.phplist.org/manual/books/phplist-manual/

MANTISFCKD

27-12-19 22:14

reporter   ~0062924

Wow, that previous one-click code update thing didn't last long... and I went and re-arranged the furniture just to accommodate it!
No matter - will upgrade to 3.4.9. as soon as time permits!

Did not get any 1064 errors today - but still get this "1366" error...

Process bounces

25 done
Closing mailbox, and purging messages
Processing bounces based on active bounce rules
0 bounces processed by advanced processing
8245 bounces were not matched by advanced processing rules
Identifying consecutive bounces
unsubscribing 58 -> 3 bounces
unsubscribing 207 -> 3 bounces
unsubscribing 373 -> 3 bounces
unsubscribing 1432 -> 3 bounces
unsubscribing 3042 -> 3 bounces
unsubscribing 3288 -> 3 bounces
unsubscribing 4393 -> 3 bounces
unsubscribing 4421 -> 3 bounces
unsubscribing 5173 -> 3 bounces
unsubscribing 5630 -> 3 bounces
unsubscribing 5807 -> 3 bounces
unsubscribing 5950 -> 3 bounces
unsubscribing 7148 -> 3 bounces
unsubscribing 7576 -> 3 bounces
unsubscribing 8837 -> 3 bounces
unsubscribing 8860 -> 3 bounces
unsubscribing 8980 -> 3 bounces
unsubscribing 9102 -> 3 bounces
unsubscribing 9232 -> 3 bounces
unsubscribing 9323 -> 3 bounces
total of 309 subscribers processed

Processed messages will be deleted from mailbox
Database error 1366 while doing query Incorrect string value: '\xFE\xFE\xEC\xE0 \xF0...' for column 'header' at row 1