History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: CIB-828
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Blocker Blocker
Assignee: Unassigned
Reporter: Sean Brandt
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Pulse

UI slow to unresponsive in large configuration

Created: 15/Dec/06 02:26 PM   Updated: 20/Dec/06 02:02 PM
Component/s: Web UI
Affects Version/s: 1.1.17, 1.2.9
Fix Version/s: 1.2.10

File Attachments: 1. Text File wrapper.log (450 kb)

Environment: linux java 1.5_08


 Description  « Hide
HSQL seems to be VERY slow for our installation, pages can take from minutes..to never showing up on request ( project list, dashboard, etc ). These same symptoms exist in both 1.1.17 and 1.2.9. I've attached a stack trace taken while the pulse main server process is eating 100% of the CPU.


 All   Comments   Change History      Sort Order:
Sean Brandt - 15/Dec/06 02:27 PM
log file containing stack trace

jason - 16/Dec/06 10:48 AM
Hi Sean,

Thanks for the foresight of taking a thread dump for us. Interestingly, I see in both cases the Pulse is trying to render an RSS feed. I wonder if potentially this is the source of the performance issue with Pulse. In particular, some feed readers will hit Pulse very frequently, and in some cases not provide accurate headers about when they last checked, which gives Pulse extra work to do. We have done some work in 1.1.23 to address these issues with caching on the Pulse side. This should also be happening in Pulse 1.2.9, but the caching is a little different on 1.2, so perhaps it is not working effectively. We will investigate the caching, and in the mean time, I wonder if you could try disabling RSS feeds (in the administration section, general settings) and seeing if that improves things? This would at least validate the theory.

Also, is this a new thing you have just started seeing? If so, have there been any other recent changes (new projects or anything) that may be related to this problem? When you say a large configuration, how many projects/users/builds are we talking about? Thanks in advance.

Sean Brandt - 16/Dec/06 04:19 PM
Turning off rss definately speeds things up.

Config:

4 projects, some with MANY tests ( 17k + )
Builds are frequent during the working hours, and currently number in the 500+ per project
Users are relatively few ( < 10 most are anonymous viewing of state )

jason - 18/Dec/06 01:42 PM
Sean,

Thanks for the further details. It appears that it is likely a problem with the RSS caching implementation on 1.2, which we are investigating in more detail now. The size of your configuration is certainly manageable, the only large number being tests/build (good work ;) ).

Daniel Ostermeier - 19/Dec/06 12:25 AM
A fix for the rss performance issues is already in place for the latest 1.1.x releases, so removing this issue from 1.1.x.

Daniel Ostermeier - 19/Dec/06 03:20 PM
Reviewing of the rss and caching mechanims have highlighted a couple of issues.

a) the cache invalidation time being used in 1.2 was too low. It was set to 2 mins, which, given the performance problems that have been reported, might well have been so short that the data was invalidated before it had been finalised.
b) part of the query used to render the build results for the rss feed was found to be excessively slow. This has been fixed.

The result is that performance of the rss feeds should be significantely improved in 1.2.10. Since the systems performance improved with rss disabled, I would expect that these mesures would visibly improve the performance of 1.2.x.

Of course, this has only been tested under artificial conditions, so if the performance is still sluggish we will reopen the investigation.

Daniel Ostermeier - 20/Dec/06 02:02 PM
Hi Sean,

I just wanted to let you know that pulse 1.2.10 is now available for download and contains the fixes mentioned above.


Cheers,
-Daniel