After upgrade to 7.14.2 , slow performance

jperreault's Avatar

jperreault

06 Nov, 2015 06:52 PM

Hi,

We just upgraded our DEV Cascade instance to 7.14.2 and after a day or so, it has really slowed down. Everything is taking a long time, even logging in.

Could there be some startup config that we need to change ?

Here's what I can tell you. Our dev server has 6 GB of memory. In the jar installation I gave Cascade 6192 MB of memory. Maybe too much ?

Here's what top looks like, 91.5 memory used.

top - 09:25:21 up 75 days, 15:45, 1 user, load average: 4.01, 3.48, 3.57
Tasks: 109 total, 1 running, 108 sleeping, 0 stopped, 0 zombie
Cpu(s): 5.9%us, 11.4%sy, 0.0%ni, 35.3%id, 44.9%wa, 1.0%hi, 1.5%si, 0.0%st
Mem: 4043720k total, 4021560k used, 22160k free, 152k buffers
Swap: 6094840k total, 3985528k used, 2109312k free, 11568k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 32259 root 18 0 8573m 3.5g 6252 S 23.9 91.5 431:50.43 java

Here's what the ps tells me, that it started with -Xmx6192M

root 32259 1 33 Nov05 ? 08:11:31 /usr/local/cascade-7.14.2/java/jre/bin/java -Djava.util.logging.config.file=/usr/local/cascade-7.14.2/tomcat/conf/logging.properties -Xmx6192M -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djsse.enableSNIExtension=false -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/local/cascade-7.14.2/tomcat/endorsed -classpath /usr/local/cascade-7.14.2/tomcat/bin/bootstrap.jar -Dcatalina.base=/usr/local/cascade-7.14.2/tomcat -Dcatalina.home=/usr/local/cascade-7.14.2/tomcat -Djava.io.tmpdir=/usr/local/cascade-7.14.2/tomcat/temp org.apache.catalina.startup.Bootstrap start

Do I need to dial that down ? Anything else I can do to diagnose the problem ?

Thanks

  1. Support Staff 1 Posted by Tim on 06 Nov, 2015 07:01 PM

    Tim's Avatar

    Hey there,

    If the dev server has 6GB of RAM total, I would definitely scale down your allocation to Cascade to perhaps 3GB. When you have a moment, try modifying the JAVA_OPTS line in your cascade.sh file and change:

    -Xmx6192M
    
    to this:
    -Xms3G -Xmx3G
    
    Then, restart Cascade.

    The next thing I would recommend checking would be to see when your scheduled link checker may be running. To do this, go to System Menu -> Preferences -> General and look at the Scheduled Link Checker Configuration. If this is currently configured to run daily, I would recommend changing it to run weekly - perhaps very early on Saturday or Sunday mornings.

    If you don't see any improvement after taking these steps, go ahead and zip up your cascade.log file from the past couple of days and I'll be happy to look over them to see if I notice any glaring problems.

    Thanks!

  2. 2 Posted by jperreault on 17 Nov, 2015 11:19 PM

    jperreault's Avatar

    Hi Tim,

    Thanks for you help. I believe that the problem was actually was the Link Checker. We have several websites in our Cascade Server instance. Most are very small, but one has 20,000 pages ( created and updated using Cascade's web services ).

    That site with 20K pages, seems to have been the problem. Once the Link Checker was turned off for that site only, the slow performance disappeared.

    Of course we would have liked to use the link checker, but we can live without it . For your reference, this is the error message we saw in the logs :

    2015-11-06 09:00:51,963 WARN [BrokenLinkReportServiceImpl] : Broken link report generation took longer than 6 hours to complete. Consider disabling link reporting for some Sites. Also consider changing the link report schedule from daily to weekly as this will allow the report to run longer before being interrupted. Broken Link Report may not have been generated for some Sites due the alotted amount of time being depleted before those Sites could be processed.

  3. Support Staff 3 Posted by Tim on 18 Nov, 2015 02:43 PM

    Tim's Avatar

    Thanks for the update! One other thing you can consider (which may or may not help, it's tough to tell) would be to increase the number of CPUs and/or the speed of the processors. Obviously this will be difficult on a physical machine, but should be pretty easy to experiment with if you're running on VMs. That, in combination with the memory increase I suggested above might help a bit.

    Let me know if you need anything else!

  4. Tim closed this discussion on 02 Dec, 2015 02:41 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac