phplist

NOTE:: Before reporting an issue, make sure you are running the latest version, currently 3.3.1


View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0002188phplist applicationBatch Processingpublic17-11-04 02:4918-02-05 02:13
Reportermgifford 
PrioritynormalSeveritymajorReproducibilitysometimes
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version2.9.3 
Target VersionFixed in Version2.9.4 
Summary0002188: Batch processing seems to go out much too slowly..
DescriptionIn the config.php file the following are set:

define("MAILQUEUE_BATCH_SIZE",100);
define("MAILQUEUE_BATCH_PERIOD",60);

this should send a batch of 100 emails every minute. This would be what I expect would happen if a user kept the window open to manually process the queue.

We are trying to automatically process the queue using an hourly cron job on phplist's shell script.

Seems the messages are squeeking out very slowly.. One with 339 Users has been going since yesterday..

PHP is in safe mode..

    * PHPlist version: 2.9.3
    * PHP version: 4.3.8
    * Webserver: Apache/2.0.50 (Fedora)
    * Website: oxnet.org/lists
    * Mysql Info: 3.23.58
    * PHP Modules:
          o yp
          o xml
          o wddx
          o tokenizer
          o sysvshm
          o sysvsem
          o standard
          o sockets
          o shmop
          o session
          o pspell
          o posix
          o pcre
          o overload
          o iconv
          o gmp
          o gettext
          o gd
          o ftp
          o exif
          o dio
          o dbx
          o dba
          o curl
          o ctype
          o calendar
          o bz2
          o bcmath
          o zlib
          o openssl
          o apache2handler
          o imap
          o ldap
          o mbstring
          o mysql


Additional Information
The last 50 events...


Events
    
del
    
page
    
date
    
message
10402 del processqueue 2004-11-16 16:58:05 10 messages sent and 0 messages skipped
10397 del processqueue 2004-11-16 16:57:22 Processing has started, 2 message(s) to process.
10398 del processqueue 2004-11-16 16:57:22 Your webserver is running in safe_mode. Please kee
10399 del processqueue 2004-11-16 16:57:22 Processing message 197
10400 del processqueue 2004-11-16 16:57:22 Looking for users
10401 del processqueue 2004-11-16 16:57:22 Found them: 78 to process
10396 del processqueue 2004-11-16 16:52:46 10 messages sent and 0 messages skipped
10391 del processqueue 2004-11-16 16:52:04 Processing has started, 2 message(s) to process.
10392 del processqueue 2004-11-16 16:52:04 Your webserver is running in safe_mode. Please kee
10393 del processqueue 2004-11-16 16:52:04 Processing message 197
10394 del processqueue 2004-11-16 16:52:04 Looking for users
10395 del processqueue 2004-11-16 16:52:04 Found them: 88 to process
10390 del processqueue 2004-11-16 16:41:03 10 messages sent and 0 messages skipped
10385 del processqueue 2004-11-16 16:40:15 Processing has started, 2 message(s) to process.
10386 del processqueue 2004-11-16 16:40:15 Your webserver is running in safe_mode. Please kee
10387 del processqueue 2004-11-16 16:40:15 Processing message 197
10388 del processqueue 2004-11-16 16:40:15 Looking for users
10389 del processqueue 2004-11-16 16:40:15 Found them: 98 to process
10384 del processqueue 2004-11-16 16:38:23 10 messages sent and 0 messages skipped
10383 del processqueue 2004-11-16 16:37:44 Found them: 108 to process
10379 del processqueue 2004-11-16 16:37:43 Processing has started, 2 message(s) to process.
10380 del processqueue 2004-11-16 16:37:43 Your webserver is running in safe_mode. Please kee
10381 del processqueue 2004-11-16 16:37:43 Processing message 197
10382 del processqueue 2004-11-16 16:37:43 Looking for users
10378 del processqueue 2004-11-16 16:14:49 10 messages sent and 0 messages skipped
10373 del processqueue 2004-11-16 16:14:05 Processing has started, 2 message(s) to process.
10374 del processqueue 2004-11-16 16:14:05 Your webserver is running in safe_mode. Please kee
10375 del processqueue 2004-11-16 16:14:05 Processing message 197
10376 del processqueue 2004-11-16 16:14:05 Looking for users
10377 del processqueue 2004-11-16 16:14:05 Found them: 118 to process
10372 del processqueue 2004-11-16 16:12:53 10 messages sent and 0 messages skipped
10367 del processqueue 2004-11-16 16:12:07 Processing has started, 2 message(s) to process.
10368 del processqueue 2004-11-16 16:12:07 Your webserver is running in safe_mode. Please kee
10369 del processqueue 2004-11-16 16:12:07 Processing message 197
10370 del processqueue 2004-11-16 16:12:07 Looking for users
10371 del processqueue 2004-11-16 16:12:07 Found them: 128 to process
10366 del processqueue 2004-11-16 16:07:48 10 messages sent and 0 messages skipped
10361 del processqueue 2004-11-16 16:07:03 Processing has started, 2 message(s) to process.
10362 del processqueue 2004-11-16 16:07:03 Your webserver is running in safe_mode. Please kee
10363 del processqueue 2004-11-16 16:07:03 Processing message 197
10364 del processqueue 2004-11-16 16:07:03 Looking for users
10365 del processqueue 2004-11-16 16:07:03 Found them: 138 to process
10360 del processqueue 2004-11-16 15:59:29 10 messages sent and 0 messages skipped
10355 del processqueue 2004-11-16 15:58:44 Processing has started, 2 message(s) to process.
10356 del processqueue 2004-11-16 15:58:44 Your webserver is running in safe_mode. Please kee
10357 del processqueue 2004-11-16 15:58:44 Processing message 197
10358 del processqueue 2004-11-16 15:58:44 Looking for users
10359 del processqueue 2004-11-16 15:58:44 Found them: 148 to process
10354 del processqueue 2004-11-16 15:49:55 10 messages sent and 0 messages skipped
10349 del processqueue 2004-11-16 15:49:09 Processing has started, 2 message(s) to process.
    
    Events close


    
logout
main pageconfigurehelpaboutlistssend a messageusersmanage userssubscribe pagesmessagestemplatesprocess queueview bounceseventlog
TagsNo tags attached.
Attached Files

- Relationships
has duplicate 0002379resolvedmichiel Batch processing by cron job gets stuck; messages display 'inprocessing'; cannot be sent 
child of 0002456resolvedmichiel PHPList v2.9.4 release 

-  Notes
(0002650)
mgifford (reporter)
23-11-04 17:19

Just to add a bit more info here..

php's safe mode on.

Process queue is running via cron but also manually using Firefox 1.0 (MacOSX)

In processing a list of over 1600+ emails the message is displayed with:

/lists/admin/?page=messages
Status: inprocess
Action: 273 processed


After manually processing the queue, I get these messages:


/lists/admin/?page=processqueue

Started
Sending in batches of 100 emails
Processing has started, 1 message(s) to process.
Your webserver is running in safe_mode. Please keep this window open. It may reload several times to make sure all messages are sent. Reports will be sent by email to oxnet@oxfam.ca
Processing message 217
Looking for users
Found them: 1336 to process

Started
Sending in batches of 100 emails
Processing has started, 1 message(s) to process.
Your webserver is running in safe_mode. Please keep this window open. It may reload several times to make sure all messages are sent. Reports will be sent by email to oxnet@oxfam.ca
Processing message 217
Looking for users
Found them: 1336 to process
Script stage: 5
Finished, Nothing to do
33 messages sent and 0 messages skipped
Finished, All done


After this is done, a few of them have been added (but far from all), as going back to the messages display provides the following:


/lists/admin/?page=messages

List has 1600+
Status: inprocess
Action: 306 processed


So it ended before all of them were sent.


Mike
(0002651)
mgifford (reporter)
23-11-04 17:23

Is there caching of the config.php file somewhere..

Just turned batch processing off in the config.php file and it still seeems to be acting like it is set at 100.

Also interesting to note that the list to process keeps going down (but by 34 rather than 100 and it isn't reloading).

Started
Sending in batches of 100 emails
This batch will be 85 emails, because in the last 60 seconds 15 emails were sent
Processing has started, 1 message(s) to process.
Your webserver is running in safe_mode. Please keep this window open. It may reload several times to make sure all messages are sent. Reports will be sent by email to oxnet@oxfam.ca
Processing message 217
Looking for users
Found them: 1269 to process
(0002653)
mgifford (reporter)
23-11-04 17:37

Reminder sent to michiel

Any thoughts on this Michiel.. This is just totally bogging down my interaction with this client.. I can't see why it isn't processing more at once..

Mike
(0002654)
mgifford (reporter)
23-11-04 17:52

Events
    del
    page
    date
    message
10899 del processqueue 2004-11-23 12:43:06 33 messages sent and 0 messages skipped
10898 del processqueue 2004-11-23 12:41:59 Found them: 1135 to process
10894 del processqueue 2004-11-23 12:41:57 Processing has started, 1 message(s) to process.
10895 del processqueue 2004-11-23 12:41:57 Your webserver is running in safe_mode. Please kee
10896 del processqueue 2004-11-23 12:41:57 Processing message 217
10897 del processqueue 2004-11-23 12:41:57 Looking for users
10893 del processqueue 2004-11-23 12:29:56 33 messages sent and 0 messages skipped
10888 del processqueue 2004-11-23 12:29:01 Processing has started, 1 message(s) to process.
10889 del processqueue 2004-11-23 12:29:01 Your webserver is running in safe_mode. Please kee
10890 del processqueue 2004-11-23 12:29:01 Processing message 217
10891 del processqueue 2004-11-23 12:29:01 Looking for users
10892 del processqueue 2004-11-23 12:29:01 Found them: 1168 to process
10887 del processqueue 2004-11-23 12:28:23 33 messages sent and 0 messages skipped
10882 del processqueue 2004-11-23 12:27:27 Processing has started, 1 message(s) to process.
10883 del processqueue 2004-11-23 12:27:27 Your webserver is running in safe_mode. Please kee
10884 del processqueue 2004-11-23 12:27:27 Processing message 217
(0002655)
michiel (manager)
24-11-04 09:27

In safe mode the system will enforce batch processing with 100 mails per batch, even if you don't use batch processing in your config file. I can't tell however why it's only sending in batches of 10 as the log looks like.

Best option is to not run in safe mode
(0002657)
mgifford (reporter)
24-11-04 13:25

If there is anything I could do to test this I'm certainly capable of doing it.. Just not sure where to begin doing it..

Not running in safe mode isn't an option with our present hosting environment (for a number of clients).

Where in the code is batch processing managed?

Is it possible to improve the docs somewhere to state what things are changed when running in safe mode?

Thanks!
(0002658)
mgifford (reporter)
24-11-04 13:45

Just adding more to this as it seems to be related.. phplist does not seem to work from the command line in safe mode. Just prints out the html from the login page I think..

Realize that most of the testing will not be done within safe mode.. But it is a problem that I think a lot of folks are having..

Mike
(0002662)
mgifford (reporter)
24-11-04 15:00

uploaded the cvs version of processqueue and the emails were going out at last!

couple things though.. If the batch size is overruled because it is in safe mode, that should show up in the report so that it doesn't indicate that in the report..

Started
Sending in batches of 500 emails
Script stage: 2
Finished, Nothing to do
0 messages sent and 0 messages skipped
Finished this run
Finished, All done


Also, it doesn't seem to produce anything in Mozilla 1.7.. I believe that the report is running, but there isn't any output to the screen..

Mike
(0002873)
michiel (manager)
03-12-04 23:05

as reported resolved (appearantly) in current CVS, will propagate to next release
(0003558)
michiel (manager)
18-02-05 02:13

Hmm, I'm willing to make things work in the browser in safe mode, but to start making things work commandline in safe mode is getting a bit too much all over the place. In that case, it's just as simple to install some 486 with a very old linux in your attic that does the job better than to host it on some killer machine at an ISP (just kidding). If you have commandline access, then why the hell would the ISP only allow safe mode. This is security jumbled up. Safe mode is silly anyway, so why not rehost (I can do you a nice deal :-)

Anyway, you're raising all kinds of other stuff, so if it is important to deal with, it would be great to add another mantis issue.


Copyright © 2000 - 2017 MantisBT Team
Powered by Mantis Bugtracker