View Issue Details

IDProjectCategoryView StatusLast Update
0019730phpList 3 applicationInterface - Administratorpublic05-02-19 18:22
Reporterrandomuser 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionopen 
PlatformlinuxOScentosOS Version7
Product Version3.3.8 
Target Version3.3.9Fixed in Version 
Summary0019730: DASHBOARD slow (10 seconds load) after 3.3.8 upgrade
DescriptionUsed to load very fast.
Other pages load just fine.
We have 24,000 users, six lists and about 150 campaigns in the db ...
DASHBOARD is incredibly slug-slow ( TEN SECONDS!!!) - and it was nearly instantaneous in V3.3.3 ( just upgraded today to 3.3.8)


Steps To Reproduceupgrade from 3.3.3. to 3.3.8
Use the dashboard
Additional InformationPHP Built On Linux 2.6.32-754.9.1.el6.x86_64 #1 SMP Thu Dec 6 08:02:15 UTC 2018 x86_64
Database Type mysql
Database Version 5.5.55
Database Collation utf8_general_ci
Database Connection Collation utf8mb4_general_ci
PHP Version 7.1.26
Tagsdashboard, slow, upgrade

Activities

randomuser

24-01-19 17:08

reporter   ~0061785

some observations posted here: https://mantis.phplist.org/view.php?id=19703

duncanc

26-01-19 19:15

developer   ~0061793

Last edited: 26-01-19 19:17

View 3 revisions

I noticed a similar problem on an installation that has over 200 sent messages. The problem is probably the query for the Last Campaign widget that gets the statistics for the latest message.
The query joins two tables then selects only the first row returned. This is inefficient when there are lots of rows to be joined in the two tables.

@samtuke A better way is to get the id of the latest message, which is done anyway, then use that in the main query.
I have just tried that and the dashboard response is immediate instead of about 10s.

samtuke

27-01-19 11:14

administrator   ~0061794

@duncanc Good point

duncanc

30-01-19 11:52

developer   ~0061802

@samtuke @xheni Is someone going to work on fixing this in time for the next release?

samtuke

30-01-19 13:14

administrator   ~0061803

@duncanc Probably not - @Suela now decides what goes into releases, and in the company we generally agree what to work on for the next release immediately following the last one (e.g. plan a month's work in advance, except for urgent / security issues).

Maybe Xheni would have time for a quick fix, but more likely it will be included in 3.3.10, unless you or another community developer makes a pull request before the first RC is issued on the 7th.

I agree this fix is important, but frequently changing the release priorities for staff developers would mean missing inclusion of other planned / headline features.

duncanc

31-01-19 07:36

developer   ~0061806

I'll look at fixing the specific problem of an inefficient query. There are a couple of other points in the code for the widget that do not look right, but I'll raise new issues for those if necessary.

samtuke

31-01-19 09:50

administrator   ~0061808

@duncanc Great, thank you

duncanc

31-01-19 12:09

developer   ~0061809

See https://github.com/phpList/phplist-ui-bootlist/pull/80

randomuser

31-01-19 19:38

reporter   ~0061810

Will be great to get this fixed... watching the HTOP on my live server, I see that when I open the Dashboard, the database process pulls 100% CPU for several seconds...

duncanc

31-01-19 19:42

developer   ~0061811

You can replace the file from the pull request

lists/admin/ui/phplist-ui-bootlist/pages/home.php

randomuser

31-01-19 19:52

reporter   ~0061812

Last edited: 31-01-19 19:56

View 3 revisions

I just tried that new file, but it actually doesn't much speed things up for me - still taking about 9 seconds to load the Dashboard page...

( watching CPU on server: 7 seconds of 100% mysql - maybe it is three seconds faster?)

duncanc

31-01-19 20:14

developer   ~0061813

Last edited: 31-01-19 20:16

View 2 revisions

Are you sure that you have copied the new home.php file, not the old file? Look at line 81, it should be

if ($_SESSION['logindetails']['superuser']) {
    $result = Sql_Query(sprintf(
        'select id from %s where sent is not null order by entered desc limit 1',
        $GLOBALS['tables']['message']
    ));

I found the change improved response from about 10s to immediately.

You can copy/paste the complete file here
https://raw.githubusercontent.com/phpList/phplist-ui-bootlist/7f6cc2ca30afc1cf283e3b1c621c2c72d88139ae/pages/home.php

randomuser

31-01-19 20:17

reporter   ~0061814

Affirmative! that is in the new file I just tested ( from here https://github.com/phpList/phplist-ui-bootlist/blob/7f6cc2ca30afc1cf283e3b1c621c2c72d88139ae/pages/home.php )

duncanc

31-01-19 20:21

developer   ~0061815

Maybe it is something else. Does the home page display the Community News section?

randomuser

31-01-19 20:23

reporter   ~0061816

Last edited: 31-01-19 20:23

View 2 revisions

Yes, it does; currently showing

 phpList community news
Fri, 11 Jan 2019
phpList 3.3.8 released: New Dashboard
Fri, 14 Dec 2018
phpList 3.3.7 released: new Automatic Updater, 3 features, 5 fixes
Sun, 11 Nov 2018
phpList 3.3.6 released: Display public lists by category

randomuser

31-01-19 20:26

reporter   ~0061817

Last edited: 31-01-19 20:27

View 2 revisions

and.. interestingly, just tried it again, ( just after trying 'rebuild db indexes') this time I got more:

phpList community news
Fri, 11 Jan 2019
phpList 3.3.8 released: New Dashboard
Fri, 14 Dec 2018
phpList 3.3.7 released: new Automatic Updater, 3 features, 5 fixes
Sun, 11 Nov 2018
phpList 3.3.6 released: Display public lists by category
Thu, 11 Oct 2018
phpList 3.3.5 what’s new: 2 features, 9 fixes, 3 UX improvements
Tue, 11 Sep 2018
phpList 3.3.4 released: 3 new features, 5 UX improvements, 20 fixes
Wed, 16 May 2018
phpList 3.3.3 released: GDPR related changes plus UX fixes
Fri, 04 May 2018
phpList 3.3.2 released: New statistics, plugins, and fixes
Tue, 14 Feb 2017
phpList 3.3.1 released: First update for phpList 3.3
Sun, 05 Feb 2017
phpList 3.3.0 – Sleeker. Faster. Safer.
Sun, 05 Feb 2017
phpList 3.3.0 Released: Major update with new user interface

duncanc

31-01-19 20:32

developer   ~0061818

Last edited: 31-01-19 20:36

View 2 revisions

The community news feed is retrieved once per login session then cached, so there would be a delay if it cannot be retrieved. But that doesn't look to be the case.

I have just tried the change again, and the original code takes 15s+ then displays immediately with the changed code.

randomuser

31-01-19 21:47

reporter   ~0061819

Well, new code is there, my /var/www/html/lists/admin/home.php is now 9,211 bytes ( old one is 17,599 bytes) and the load time is 15.61 seconds according to my stopwatch.
If I put the old one back in, then I get 16.09 seconds.

I am testing with ~ 24,000 users, six lists and about 150 campaigns in the db ... what are you testing with?

duncanc

31-01-19 21:49

developer   ~0061820

Last edited: 31-01-19 21:50

View 3 revisions

That's not the right file, the correct path is
lists/admin/ui/phplist-ui-bootlist/pages/home.php

randomuser

31-01-19 21:59

reporter   ~0061821

Last edited: 31-01-19 22:00

View 2 revisions

Ah... OK - that wasn't obvious from anything I saw here. (looked but did not SEE...)
My local folder apparently hasnt been updated for quite some time.

Mystery solved - DASHBOARD LOAD IS NOW INSTANTANEOUS (<0.5 second) .. the new file is a definite improvement, THANKS!

duncanc

31-01-19 22:04

developer   ~0061822

Great, thanks!