View Issue Details

IDProjectCategoryView StatusLast Update
0003416phplist applicationBounce Managementpublic18-02-08 14:06
Reporterduncan 
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Product Version2.8.12 
Target Version2.10.7Fixed in Version2.10.4 
Summary0003416: Bounce testing not consecutive
DescriptionThe bounce test mechanism does not get consective bounces, it gathers all bounces for a user and unconfirms them even if they have been sent messages that have received with no bounce response in between

A work around is to delete the bounces but that is losing valuable info

It needs a test to say get the last x messages sent to the user - are there bounce replies for all of them - if so unconfirm them, if not leave them be until next time.

Leave the bounce counter as purely info only (I think thats all it is but can't be sure)
Additional InformationThis query in processbounces

  $msg_req = Sql_Query(sprintf('select * from
    %s left join %s on %s.messageid = %s.message
    where (date_add(entered,INTERVAL 5 day) < now())
    and userid = %d
    and (userid = user or user is null) order by entered desc',
    $tables["usermessage"],$tables["user_message_bounce"],
    $tables["usermessage"],$tables["user_message_bounce"],
    $user[0]));

gets their bounce history - then iterates through this until the threshold
TagsNo tags attached.

Relationships

related to 0003721 closed phplist 2.10.x 
has duplicate 0002623 closed manually override bounce status may not work 

Activities

michiel

14-06-05 14:51

manager   ~0005218

this may relate to your Mysql version. can you give the version you're using?

it's worked for a long time, so I can't imagine it broke just now, but maybe the mysql version (4.1 has had quite a few changes in date fields) might be a clue.

also, the consecutiveness is only tested for bounces that are more than 5 days old

and yes, the "bouncecount" field is informational only.

duncan

14-06-05 15:17

reporter   ~0005223

Hi Michiel

Mysql is version 4.0.20

Phplist is V 2.8.12

I have only recently got complaints and noticed this behaviour

I think you are right - the query should pull all my messages withor without bounces but it doesn't. I get some of them but am a bit tired right now to work out which. Will come back to you
I think I also have an that the 5 day time out means the client is continually unsubbed if I send regular messages as they don't get counted as valid until after 5days - and I send messages every three days and process the bounces which ends up unconfirming them - it means maybe I need to only process bounces greater than 5 days - but this could be the date thing.

duncan

05-11-05 22:22

reporter   ~0008434

Hi Michiel

Finally got the sql to do what I want

SELECT userid , messageid, message, bounce, entered
FROM `phplist_usermessage` , (select distinct user AS u from phplist_user_message_bounce) AS busers
LEFT JOIN phplist_user_message_bounce ON (messageid = message AND userid = user)
WHERE userid = busers.u
ORDER BY userid

This gives me a big table with all users who have had a bounce

I could also cut out all messages less than the date of say the threshold plus 10 as if they are not bouncing recently they are not important.

Still maybe its all better in 2.10 - but I get two emails sent so haven't played a lot with it yet.

I don't think the time thing mattered - maybe you have resolved this in 2.10 - haven't checked yet

duncan

05-11-05 22:59

reporter   ~0008435

This is how I have adjusted processbounces.php
Line 340 - 348 - removed where clause and added it to join - works for me now
$msg_req = Sql_Query(sprintf('select * from
                %s left join %s on (%s.messageid = %s.message AND userid=user)
                where (date_add(entered,INTERVAL 5 day) < now())
                and userid = %d
                
                order by entered desc',
                $tables["usermessage"],$tables["user_message_bounce"],
                $tables["usermessage"],$tables["user_message_bounce"],
                $user[0]));

duncan

05-11-05 23:22

reporter   ~0008436

Finally got it working for me as per below

The sql found me the consecutive bounces but the loop didn't care whether they were recent of 6 months ago. The below changes fixed it for me and sped everything up hugely. It looks like 2.10 is still the same so I would suggest looking at this as a change. I am using Mysql 4.1.11 Debian. Not sure how it works on earlier versions though

Cheers Duncan

  #$alive = 1;$removed = 0; DT 051105
  $alive = 1;$removed = 0; $msgokay=0;
        #while ($alive && !$removed && $bounce = Sql_Fetch_Array($msg_req)) { DT 051105
        while ($alive && !$removed && !$msgokay &&$bounce = Sql_Fetch_Array($msg_req)) {
    $alive = checkLock($process_id);
    if ($alive)
      keepLock($process_id);
    else
      ProcessError("Process Killed by other process");
                if (sprintf('%d',$bounce["bounce"]) == $bounce["bounce"]) {
                        $cnt++;
      if ($cnt >= $bounce_unsubscribe_treshold) {
        $removed = 1;
        output(sprintf('unsubscribing %d -> %d bounces',$user[0],$cnt));
        $userurl = PageLink2("user&id=$user[0]",$user[0]);
        logEvent("User $userurl has consecutive bounces ($cnt) over treshold, user marked unconfirmed");
        $emailreq = Sql_Fetch_Row_Query("select email from {$tables["user"]} where id = $user[0]");
        addUserHistory($emailreq[0],"Auto Unsubscribed","User auto unsubscribed for $cnt consecutive bounces");
        Sql_Query(sprintf('update %s set confirmed = 0 where id = %d',$tables["user"],$user[0]));
        $email_req = Sql_Fetch_Row_Query(sprintf('select email from %s where id = %d',$tables["user"],$user[0]));
        $unsubscribed_users .= $email_req[0] . " [$user[0]] ($cnt)\n";
      }
                } elseif ($bounce["bounce"] == "") {
                        #$cnt = 0; DT 051105
                        $cnt = 0;
                        $msgokay = 1; #DT 051105 - escaping loop if message received okay

duncan

08-11-05 08:50

reporter   ~0008484

I have to say it now goes so much faster I will put it in cron now. Before it took forever (many minutes) to process all users, now less than a min

michiel

22-11-05 01:14

manager   ~0008818

I'm afraid your query in 0003416:0008434 is Mysql 4.1 and up only. I wouldn't just yet want to create a dependency on 4.1, I'm using 4.0 myself mostly.

I'll try to look into the rest though, because I'm getting very long process bounces processes as well, so I'd like to make it faster. Thanks for all the effort to investigate this.

user1177

17-04-06 05:41

  ~0012683

Research on a related bug? http://www.phplist.com/forums/viewtopic.php?t=4568

Issue History

Date Modified Username Field Change
14-06-05 14:33 duncan New Issue
14-06-05 14:51 michiel Note Added: 0005218
14-06-05 15:17 duncan Note Added: 0005223
08-07-05 05:03 michiel Relationship added related to 0002623
05-11-05 22:22 duncan Note Added: 0008434
05-11-05 22:59 duncan Note Added: 0008435
05-11-05 23:22 duncan Note Added: 0008436
08-11-05 08:50 duncan Note Added: 0008484
21-11-05 18:33 michiel Relationship added related to 0003721
22-11-05 01:14 michiel Note Added: 0008818
09-02-06 18:56 user1177 Status new => confirmed
09-02-06 18:57 user1177 Relationship replaced has duplicate 0002623
17-04-06 05:41 user1177 Note Added: 0012683
04-10-06 18:03 michiel Status confirmed => new
16-12-06 02:45 michiel Status new => resolved
16-12-06 02:45 michiel Fixed in Version => 2.10.4
16-12-06 02:45 michiel Resolution open => fixed
18-02-08 14:05 user1822 Target Version => 2.10.6