Home → Installation and System Configuration →

Out of memory / extremely slow performance

• Edit

ken

16 Nov, 2010 06:16 PM

We have been seeing severe performance issues when using Cascade (6.7.4 and previous versions), and today received an out of memory error during a publish, which subsequently failed. The error is as follows:

Jobs with Errors (1): [Top]

[Destination: Leadership] Leadership Institute: blog/index java.lang.OutOfMemoryError: Java heap space

           at java.lang.String.(Unknown Source)
           at java.lang.StringBuffer.toString(Unknown Source)
           at java.io.StringWriter.toString(Unknown Source)
           at org.jdom.output.XMLOutputter.outputString(XMLOutputter.java:544)
           at org.jdom.transform.JDOMSource$JDOMInputSource.getCharacterStream(JDOMSource.java:441)
           at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
           at com.hannonhill.cascade.xml.reader.CascadeXMLFilterAdapter.parse(CascadeXMLFilterAdapter.java:136)
           at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:485)
           at com.hannonhill.cascade.model.render.task.PageRewriteTask.transformXML(PageRewriteTask.java:562)
           at com.hannonhill.cascade.model.render.task.PageRewriteTask.rewrite(PageRewriteTask.java:138)
           at com.hannonhill.cascade.model.render.task.PageRewriteTask.execute(PageRewriteTask.java:114)
           at com.hannonhill.cascade.model.service.TaskServiceImpl.executeTask(TaskServiceImpl.java:45)
           at com.hannonhill.cascade.model.service.TaskServiceImpl.executeTaskWithReadOnlyTransaction(TaskServiceImpl.java:28)
           at sun.reflect.GeneratedMethodAccessor326.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
           at java.lang.reflect.Method.invoke(Unknown Source)
           at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
           at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
           at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:77)
           at com.hannonhill.cascade.aspect.TransactionalExceptionEncoderAspect.process(TransactionalExceptionEncoderAspect.java:86)
           at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
           at java.lang.reflect.Method.invoke(Unknown Source)
           at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
           at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
           at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:64)
           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:160)
           at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:77)
           at com.hannonhill.cascade.aspect.TransactionTimer.doSomething(TransactionTimer.java:46)
           at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

cascadew.exe is configured for 2048Mb of RAM, but if I enter that amount in cascade.bat the app will not start. Currently cascade.bat is configured with JAVA_OPTS=-Xmx1024M.

As far as performance issues, at times it takes over 20 seconds to load a single page and publishing may take nearly 4-5 minutes (for 4 pages). The server is a Dual 2Ghz Xeon PowerEdge running x64 Windows 2008 Standard with 4Gb of RAM. When cascade performance is unacceptable the server is busy (CPU, RAM) but nowhere near maxed out.

1. Support Staff 1 Posted by Tim on 16 Nov, 2010 06:40 PM

   

   Hi Ken,

   Are you using the Windows service to start Cascade Server or are you running it from the command prompt?
   

   • Edit

2. 2 Posted by ken on 16 Nov, 2010 07:26 PM

   

   Hi Tim,
   Starting it from the command prompt as we had issues with the service (it was unable to access network resources when the service started as a particular user account with access to those resources). We now login to the server as the same user, and start cascade manually under that account through the batch file.

   • Edit

3. Support Staff 3 Posted by Tim on 16 Nov, 2010 07:41 PM

   

   Thanks for that info. Since you are starting from the command line, the only file you need to worry about is cascade.bat. The cascadew.exe file is only used when Cascade is running as a Windows service.

   So, there are a few things you can try to increase performance for the application. First, I would recommend modifying your cascade.bat file such that it specifies both an initial heap size(Xms) and a maximum heap size(Xmx). Typically I tell clients to set these to the same value. For example, you could try replacing your JAVA_OPTS line with something like this:
   

   set JAVA_OPTS=-Xms1260M -Xmx1260M -XX:MaxPermSize=192m -Djava.awt.headless=true -Dfile.encoding=UTF-8

   If you still notice that you are getting Out of Memory errors, we could try increasing those values even more. If you are using the bundled JRE that comes with Cascade, you will only be able to allocate ~1450M RAM. Attempting to allocate any more than that will prevent the application from starting up. However, if you decide you do want to allocate more for Cascade, you can download a 64-bit JVM (since you are running a 64-bit O/S) and point Cascade to it. Then, you would be able to allocate over 2GB of RAM if you wanted to.

   Let me know if you have any further questions! Otherwise, I'll wait to hear back from you after you make those modifications.

   • Edit

4. 4 Posted by ken on 17 Nov, 2010 12:47 AM

   

   Thanks Tim,
   I've modified the bat file with your suggestions...next step will be to switch to an x64 JVM.

   • Edit

5. Support Staff 5 Posted by Tim on 17 Nov, 2010 03:07 PM

   

   Good deal Ken. I'll leave this ticket open for a few days and wait to hear back from you on how everything is going after making that change.
   

   • Edit

6. 6 Posted by ken on 17 Nov, 2010 03:25 PM

   

   Hi Tim,
   The performance this morning was still unacceptably slow, so we have downloaded and installed the x64 JVM. I'm assuming that for a batch start I simply have to change:

   set JRE_HOME=D:\Cascade Server\jre
   to
   set JRE_HOME=C:\Program Files\Java\jre6

   and then increase the max memory settings?

   • Edit

7. Support Staff 7 Posted by Tim on 17 Nov, 2010 03:33 PM

   

   Yes, I believe that should do the trick. In the meantime, can you attach today's cascade.log file to this issue? I'd like to look over it to see if I notice anything that could be affecting performance.

   Thanks!

   • Edit

8. 8 Posted by ken on 17 Nov, 2010 03:40 PM

   

   sure, here it is...

   • cascade.log 179 KB
   • Edit

9. Support Staff 9 Posted by Tim on 17 Nov, 2010 04:34 PM

   

   Thanks Ken. That particular log file doesn't contain anything out of the ordinary as far as I can tell. It seems to show a startup followed by a shutdown, then another startup followed by a shutdown.

   When you say you are seeing performance issues, what types of behavior are you noticing? Are there certain pages that take a long time to load? Is the system sluggish in general?
   

   If you can provide me with access to your instance, I'd be happy to take a look around to see if I notice anything that could be causing a problem.

   • Edit

10. Tim closed this discussion on 18 Jul, 2011 04:02 PM.