[#CIB-953] crash when trying to view queue with 16 projects queued up

Pulse

crash when trying to view queue with 16 projects queued up

Created: 21/Feb/07 06:16 AM Updated: 03/Mar/07 01:03 AM

Component/s:

Web UI

Affects Version/s:

1.2.17

Fix Version/s:

1.2.19

Description

« Hide

We use the svn:external support, and have about 16 projects all of which share a common ant build through an svn:external. Changes to this build thus trigger 16 concurrent builds. Since we're only running a single pulse instance at the moment (see ~~CIB-948~~) these all get queued up. Trying to access the Web UI at these times typically times out after several minutes (see ~~CIB-943~~). Sometimes though it seems to crash Pulse. This latest time I had clicked the "Queues" tab to try and see what project was currently building. After a few minutes I received this

2/20/07 10:11:32 PM PST (78 ms ago) failure severe count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions

S1000 General error java.lang.NullPointerException in statement [insert into FEATURE (LEVEL, SUMMARY, FEATURE_TYPE, ID) values (?, ?, 'com.zutubi.pulse.core.model.Feature', ?)]

2/20/07 10:11:32 PM PST (100 ms ago) failure warning count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions

SQL Error: 40, SQLState: S1000

2/20/07 10:11:32 PM PST (210 ms ago) failure severe count :: 2 org.hibernate.event.def.AbstractFlushingEventListener :: performExecutions

Could not synchronize database state with session

org.hibernate.exception.GenericJDBCException: could not insert: [com.zutubi.pulse.core.model.Feature]
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:91)
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:79)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2078)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2427)
at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:51)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:243)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:227)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:140)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:296)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1007)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:354)
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:558)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:540)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:510)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:310)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:117)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
at $Proxy28.save(Unknown Source)
at com.zutubi.pulse.BuildController.completeBuild(BuildController.java:624)
at com.zutubi.pulse.BuildController.handleEvent(BuildController.java:209)
at com.zutubi.pulse.events.AsynchronousDelegatingListener$1.run(AsynchronousDelegatingListener.java:42)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.sql.SQLException: S1000 General error java.lang.NullPointerException in statement [insert into FEATURE (LEVEL, SUMMARY, FEATURE_TYPE, ID) values (?, ?, 'com.zutubi.pulse.core.model.Feature', ?)]
at org.hsqldb.jdbc.Util.throwError(Util.java:56)
at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(jdbcPreparedStatement.java:430)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:23)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2062)
... 26 more
)

2/20/07 10:11:32 PM PST (175 ms ago) failure severe count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions

S1000 General error java.lang.NullPointerException in statement [insert into FEATURE (LEVEL, SUMMARY, FEATURE_TYPE, ID) values (?, ?, 'com.zutubi.pulse.core.model.Feature', ?)]

Description

We use the svn:external support, and have about 16 projects all of which share a common ant build through an svn:external. Changes to this build thus trigger 16 concurrent builds. Since we're only running a single pulse instance at the moment (see ~~CIB-948~~) these all get queued up. Trying to access the Web UI at these times typically times out after several minutes (see ~~CIB-943~~). Sometimes though it seems to crash Pulse. This latest time I had clicked the "Queues" tab to try and see what project was currently building. After a few minutes I received this 2/20/07 10:11:32 PM PST (78 ms ago) failure severe count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions S1000 General error java.lang.NullPointerException in statement [insert into FEATURE (LEVEL, SUMMARY, FEATURE_TYPE, ID) values (?, ?, 'com.zutubi.pulse.core.model.Feature', ?)] 2/20/07 10:11:32 PM PST (100 ms ago) failure warning count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions SQL Error: 40, SQLState: S1000 2/20/07 10:11:32 PM PST (210 ms ago) failure severe count :: 2 org.hibernate.event.def.AbstractFlushingEventListener :: performExecutions Could not synchronize database state with session org.hibernate.exception.GenericJDBCException: could not insert: [com.zutubi.pulse.core.model.Feature] at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:91) at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:79) at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2078) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2427) at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:51) at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:243) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:227) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:140) at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:296) at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27) at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1007) at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:354) at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106) at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:558) at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:540) at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:510) at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:310) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:117) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209) at $Proxy28.save(Unknown Source) at com.zutubi.pulse.BuildController.completeBuild(BuildController.java:624) at com.zutubi.pulse.BuildController.handleEvent(BuildController.java:209) at com.zutubi.pulse.events.AsynchronousDelegatingListener$1.run(AsynchronousDelegatingListener.java:42) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) at java.lang.Thread.run(Thread.java:595) Caused by: java.sql.SQLException: S1000 General error java.lang.NullPointerException in statement [insert into FEATURE (LEVEL, SUMMARY, FEATURE_TYPE, ID) values (?, ?, 'com.zutubi.pulse.core.model.Feature', ?)] at org.hsqldb.jdbc.Util.throwError(Util.java:56) at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(jdbcPreparedStatement.java:430) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:23) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2062) ... 26 more ) 2/20/07 10:11:32 PM PST (175 ms ago) failure severe count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions S1000 General error java.lang.NullPointerException in statement [insert into FEATURE (LEVEL, SUMMARY, FEATURE_TYPE, ID) values (?, ?, 'com.zutubi.pulse.core.model.Feature', ?)]

Show »

All

Comments

Change History

Sort Order:

[ Permlink | « Hide ]

Alex Chapman - 21/Feb/07 06:27 AM

Affects Version should be 1.2.17 - we upgraded this evening before experiencing these failures

[ Permlink | « Hide ]

Alex Chapman - 21/Feb/07 06:42 AM

I don't know if this is the same problem (I can log a separate one if that will help in tracking) but I just received another crash. This time there weren't any builds queued up. I had completed just 2 builds since restarting it after the last crash. I had clicked on the "Projects" link. Here are the exceptions

2/20/07 10:40:58 PM PST (38 ms ago) failure warning count :: 1 org.mortbay.jetty.servlet.ServletHandler :: handle

/pulse/viewProjects.action:

org.springframework.orm.hibernate3.HibernateJdbcException: JDBC exception on Hibernate data access; nested exception is org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544]
Caused by: org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544]
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:91)
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:79)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at org.hibernate.loader.Loader.loadCollection(Loader.java:1926)
at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:71)
at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:520)
at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:60)
at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1693)
at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:454)
at org.hibernate.engine.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:827)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:229)
at org.hibernate.loader.Loader.doList(Loader.java:2150)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2029)
at org.hibernate.loader.Loader.list(Loader.java:2024)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:369)
at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:300)
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:153)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1127)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
at org.springframework.orm.hibernate3.HibernateTemplate$29.doInHibernate(HibernateTemplate.java:839)
at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:362)
at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:830)
at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:822)
at com.zutubi.pulse.model.persistence.hibernate.HibernateEntityDao.findAll(HibernateEntityDao.java:48)
at sun.reflect.GeneratedMethodAccessor322.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
at $Proxy11.findAll(Unknown Source)
at com.zutubi.pulse.model.DefaultProjectManager.getAllProjectGroups(DefaultProjectManager.java:536)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
at $Proxy30.getAllProjectGroups(Unknown Source)
at com.zutubi.pulse.web.project.ViewProjectsAction.execute(ViewProjectsAction.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.opensymphony.xwork.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:353)
at com.opensymphony.xwork.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:208)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:182)
at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.DefaultWorkflowInterceptor.intercept(DefaultWorkflowInterceptor.java:125)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.CancelWorkflowInterceptor.intercept(CancelWorkflowInterceptor.java:37)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.LicensedInterceptor.intercept(LicensedInterceptor.java:81)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.webwork.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:136)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.HibernateSessionInterceptor.intercept(HibernateSessionInterceptor.java:36)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:113)
at com.opensymphony.webwork.dispatcher.DispatcherUtils.serviceAction(DispatcherUtils.java:202)
at com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:111)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:688)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264)
at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:181)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:216)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148)
at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:52)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:195)
at com.zutubi.pulse.spring.FilterWrapper.doFilter(FilterWrapper.java:46)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1530)
at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1482)
at org.mortbay.http.HttpServer.service(HttpServer.java:927)
at org.mortbay.http.HttpConnection.service(HttpConnection.java:816)
at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983)
at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833)
at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
Caused by: java.sql.SQLException: out of memory
at org.hsqldb.jdbc.Util.throwError(Util.java:56)
at org.hsqldb.jdbc.jdbcPreparedStatement.executeQuery(jdbcPreparedStatement.java:386)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:139)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1669)
at org.hibernate.loader.Loader.doQuery(Loader.java:662)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
at org.hibernate.loader.Loader.loadCollection(Loader.java:1919)
... 125 more
)

2/20/07 10:40:58 PM PST (53 ms ago) failure severe count :: 1 com.opensymphony.webwork.dispatcher.DispatcherUtils :: serviceAction

Could not execute action

org.springframework.orm.hibernate3.HibernateJdbcException: JDBC exception on Hibernate data access; nested exception is org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544]
Caused by: org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544]
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:91)
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:79)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at org.hibernate.loader.Loader.loadCollection(Loader.java:1926)
at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:71)
at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:520)
at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:60)
at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1693)
at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:454)
at org.hibernate.engine.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:827)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:229)
at org.hibernate.loader.Loader.doList(Loader.java:2150)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2029)
at org.hibernate.loader.Loader.list(Loader.java:2024)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:369)
at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:300)
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:153)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1127)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
at org.springframework.orm.hibernate3.HibernateTemplate$29.doInHibernate(HibernateTemplate.java:839)
at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:362)
at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:830)
at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:822)
at com.zutubi.pulse.model.persistence.hibernate.HibernateEntityDao.findAll(HibernateEntityDao.java:48)
at sun.reflect.GeneratedMethodAccessor322.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
at $Proxy11.findAll(Unknown Source)
at com.zutubi.pulse.model.DefaultProjectManager.getAllProjectGroups(DefaultProjectManager.java:536)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
at $Proxy30.getAllProjectGroups(Unknown Source)
at com.zutubi.pulse.web.project.ViewProjectsAction.execute(ViewProjectsAction.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.opensymphony.xwork.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:353)
at com.opensymphony.xwork.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:208)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:182)
at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.DefaultWorkflowInterceptor.intercept(DefaultWorkflowInterceptor.java:125)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.CancelWorkflowInterceptor.intercept(CancelWorkflowInterceptor.java:37)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.LicensedInterceptor.intercept(LicensedInterceptor.java:81)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.webwork.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:136)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.zutubi.pulse.xwork.interceptor.HibernateSessionInterceptor.intercept(HibernateSessionInterceptor.java:36)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180)
at com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:113)
at com.opensymphony.webwork.dispatcher.DispatcherUtils.serviceAction(DispatcherUtils.java:202)
at com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:111)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:688)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264)
at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:181)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:216)
at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)
at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148)
at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:52)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:195)
at com.zutubi.pulse.spring.FilterWrapper.doFilter(FilterWrapper.java:46)
at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821)
at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1530)
at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1482)
at org.mortbay.http.HttpServer.service(HttpServer.java:927)
at org.mortbay.http.HttpConnection.service(HttpConnection.java:816)
at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983)
at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833)
at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
Caused by: java.sql.SQLException: out of memory
at org.hsqldb.jdbc.Util.throwError(Util.java:56)
at org.hsqldb.jdbc.jdbcPreparedStatement.executeQuery(jdbcPreparedStatement.java:386)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:139)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1669)
at org.hibernate.loader.Loader.doQuery(Loader.java:662)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
at org.hibernate.loader.Loader.loadCollection(Loader.java:1919)
... 125 more
)

2/20/07 10:40:58 PM PST (57 ms ago) failure severe count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions

out of memory

2/20/07 10:40:58 PM PST (61 ms ago) failure warning count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions

SQL Error: 72, SQLState: S1000

[ Show » ]

Alex Chapman - 21/Feb/07 06:42 AM I don't know if this is the same problem (I can log a separate one if that will help in tracking) but I just received another crash. This time there weren't any builds queued up. I had completed just 2 builds since restarting it after the last crash. I had clicked on the "Projects" link. Here are the exceptions 2/20/07 10:40:58 PM PST (38 ms ago) failure warning count :: 1 org.mortbay.jetty.servlet.ServletHandler :: handle /pulse/viewProjects.action: org.springframework.orm.hibernate3.HibernateJdbcException: JDBC exception on Hibernate data access; nested exception is org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544] Caused by: org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544] at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:91) at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:79) at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43) at org.hibernate.loader.Loader.loadCollection(Loader.java:1926) at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:71) at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:520) at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:60) at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1693) at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:454) at org.hibernate.engine.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:827) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:229) at org.hibernate.loader.Loader.doList(Loader.java:2150) at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2029) at org.hibernate.loader.Loader.list(Loader.java:2024) at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:369) at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:300) at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:153) at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1127) at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79) at org.springframework.orm.hibernate3.HibernateTemplate$29.doInHibernate(HibernateTemplate.java:839) at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:362) at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:830) at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:822) at com.zutubi.pulse.model.persistence.hibernate.HibernateEntityDao.findAll(HibernateEntityDao.java:48) at sun.reflect.GeneratedMethodAccessor322.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209) at $Proxy11.findAll(Unknown Source) at com.zutubi.pulse.model.DefaultProjectManager.getAllProjectGroups(DefaultProjectManager.java:536) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209) at $Proxy30.getAllProjectGroups(Unknown Source) at com.zutubi.pulse.web.project.ViewProjectsAction.execute(ViewProjectsAction.java:48) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at com.opensymphony.xwork.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:353) at com.opensymphony.xwork.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:208) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:182) at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.DefaultWorkflowInterceptor.intercept(DefaultWorkflowInterceptor.java:125) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.CancelWorkflowInterceptor.intercept(CancelWorkflowInterceptor.java:37) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.LicensedInterceptor.intercept(LicensedInterceptor.java:81) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.webwork.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:136) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.HibernateSessionInterceptor.intercept(HibernateSessionInterceptor.java:36) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:113) at com.opensymphony.webwork.dispatcher.DispatcherUtils.serviceAction(DispatcherUtils.java:202) at com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:111) at javax.servlet.http.HttpServlet.service(HttpServlet.java:688) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:181) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:216) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118) at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:52) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:195) at com.zutubi.pulse.spring.FilterWrapper.doFilter(FilterWrapper.java:46) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568) at org.mortbay.http.HttpContext.handle(HttpContext.java:1530) at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633) at org.mortbay.http.HttpContext.handle(HttpContext.java:1482) at org.mortbay.http.HttpServer.service(HttpServer.java:927) at org.mortbay.http.HttpConnection.service(HttpConnection.java:816) at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983) at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833) at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244) at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) Caused by: java.sql.SQLException: out of memory at org.hsqldb.jdbc.Util.throwError(Util.java:56) at org.hsqldb.jdbc.jdbcPreparedStatement.executeQuery(jdbcPreparedStatement.java:386) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92) at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:139) at org.hibernate.loader.Loader.getResultSet(Loader.java:1669) at org.hibernate.loader.Loader.doQuery(Loader.java:662) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224) at org.hibernate.loader.Loader.loadCollection(Loader.java:1919) ... 125 more ) 2/20/07 10:40:58 PM PST (53 ms ago) failure severe count :: 1 com.opensymphony.webwork.dispatcher.DispatcherUtils :: serviceAction Could not execute action org.springframework.orm.hibernate3.HibernateJdbcException: JDBC exception on Hibernate data access; nested exception is org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544] Caused by: org.hibernate.exception.GenericJDBCException: could not initialize a collection: [com.zutubi.pulse.model.ProjectGroup.projects#26476544] at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:91) at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:79) at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43) at org.hibernate.loader.Loader.loadCollection(Loader.java:1926) at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:71) at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:520) at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:60) at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1693) at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:454) at org.hibernate.engine.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:827) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:229) at org.hibernate.loader.Loader.doList(Loader.java:2150) at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2029) at org.hibernate.loader.Loader.list(Loader.java:2024) at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:369) at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:300) at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:153) at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1127) at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79) at org.springframework.orm.hibernate3.HibernateTemplate$29.doInHibernate(HibernateTemplate.java:839) at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:362) at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:830) at org.springframework.orm.hibernate3.HibernateTemplate.find(HibernateTemplate.java:822) at com.zutubi.pulse.model.persistence.hibernate.HibernateEntityDao.findAll(HibernateEntityDao.java:48) at sun.reflect.GeneratedMethodAccessor322.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209) at $Proxy11.findAll(Unknown Source) at com.zutubi.pulse.model.DefaultProjectManager.getAllProjectGroups(DefaultProjectManager.java:536) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209) at $Proxy30.getAllProjectGroups(Unknown Source) at com.zutubi.pulse.web.project.ViewProjectsAction.execute(ViewProjectsAction.java:48) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at com.opensymphony.xwork.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:353) at com.opensymphony.xwork.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:208) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:182) at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.ErrorHandlingInterceptor.intercept(ErrorHandlingInterceptor.java:31) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.DefaultWorkflowInterceptor.intercept(DefaultWorkflowInterceptor.java:125) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.CancelWorkflowInterceptor.intercept(CancelWorkflowInterceptor.java:37) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.LicensedInterceptor.intercept(LicensedInterceptor.java:81) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.webwork.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:136) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:32) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.zutubi.pulse.xwork.interceptor.HibernateSessionInterceptor.intercept(HibernateSessionInterceptor.java:36) at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:180) at com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:113) at com.opensymphony.webwork.dispatcher.DispatcherUtils.serviceAction(DispatcherUtils.java:202) at com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:111) at javax.servlet.http.HttpServlet.service(HttpServlet.java:688) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:830) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:181) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:216) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118) at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:52) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:195) at com.zutubi.pulse.spring.FilterWrapper.doFilter(FilterWrapper.java:46) at org.mortbay.jetty.servlet.WebApplicationHandler$CachedChain.doFilter(WebApplicationHandler.java:821) at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:471) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:568) at org.mortbay.http.HttpContext.handle(HttpContext.java:1530) at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:633) at org.mortbay.http.HttpContext.handle(HttpContext.java:1482) at org.mortbay.http.HttpServer.service(HttpServer.java:927) at org.mortbay.http.HttpConnection.service(HttpConnection.java:816) at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:983) at org.mortbay.http.HttpConnection.handle(HttpConnection.java:833) at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244) at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) Caused by: java.sql.SQLException: out of memory at org.hsqldb.jdbc.Util.throwError(Util.java:56) at org.hsqldb.jdbc.jdbcPreparedStatement.executeQuery(jdbcPreparedStatement.java:386) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92) at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:139) at org.hibernate.loader.Loader.getResultSet(Loader.java:1669) at org.hibernate.loader.Loader.doQuery(Loader.java:662) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224) at org.hibernate.loader.Loader.loadCollection(Loader.java:1919) ... 125 more ) 2/20/07 10:40:58 PM PST (57 ms ago) failure severe count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions out of memory 2/20/07 10:40:58 PM PST (61 ms ago) failure warning count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions SQL Error: 72, SQLState: S1000

[ Permlink | « Hide ]

Daniel Ostermeier - 21/Feb/07 07:13 AM

Hi Alex,

From the exception, it looks like part of one of the builds failed trying to store a build feature in the database. From past experience, this will typically only happen when the disk on which pulse is running has filled up. Can you please check if this is the case here?

If so, this usually (and unfortunately) means that the HSQL database has been corrupted.

In this case, what we usually do is
a) recommend that you free up some disk space. (see http://confluence.zutubi.com/x/gBM for full details on the layout of the PULSE_DATA directory)
b) restart pulse and see how it goes, although i suspect that it will still fail.
c) send us the database so see if we can manually fix it.

Fortunately, however, because you recently upgraded to 1.2.17, pulse will have generated a backup in the PULSE_DATA/backups directory. These backups are snapshots of the PULSE_DATA directory at the time of the upgrade, and contain all the data needed to get you up and running. The noticeable exclusions from the backups are the projects and recipes directories. That is, no build / project artifacts are included in the backup to ensure that the size of the backup remains manageable.

What I would recommend is firstly freeing up some disk space (assuming that the disk is full). Then, extract the latest available backup and point a pulse instance to that extracted backup. What this will do is give you a fully running pulse installation as before, but without the build artifacts.

Alternatively, if you would like me to take a look at your database to see what can be done, please let me know.

Once you are up and running again, I would recommend that you move to an external database (mysql or postgres) at your earliest convenience to ensure that this does not happen again. Details for migrating available at http://confluence.zutubi.com/x/ahM

Please let me know how you go and what you decide to do.

Cheers,
-Daniel

[ Show » ]

Daniel Ostermeier - 21/Feb/07 07:13 AM Hi Alex, From the exception, it looks like part of one of the builds failed trying to store a build feature in the database. From past experience, this will typically only happen when the disk on which pulse is running has filled up. Can you please check if this is the case here? If so, this usually (and unfortunately) means that the HSQL database has been corrupted. In this case, what we usually do is a) recommend that you free up some disk space. (see http://confluence.zutubi.com/x/gBM for full details on the layout of the PULSE_DATA directory) b) restart pulse and see how it goes, although i suspect that it will still fail. c) send us the database so see if we can manually fix it. Fortunately, however, because you recently upgraded to 1.2.17, pulse will have generated a backup in the PULSE_DATA/backups directory. These backups are snapshots of the PULSE_DATA directory at the time of the upgrade, and contain all the data needed to get you up and running. The noticeable exclusions from the backups are the projects and recipes directories. That is, no build / project artifacts are included in the backup to ensure that the size of the backup remains manageable. What I would recommend is firstly freeing up some disk space (assuming that the disk is full). Then, extract the latest available backup and point a pulse instance to that extracted backup. What this will do is give you a fully running pulse installation as before, but without the build artifacts. Alternatively, if you would like me to take a look at your database to see what can be done, please let me know. Once you are up and running again, I would recommend that you move to an external database (mysql or postgres) at your earliest convenience to ensure that this does not happen again. Details for migrating available at http://confluence.zutubi.com/x/ahM Please let me know how you go and what you decide to do. Cheers, -Daniel

[ Permlink | « Hide ]

Alex Chapman - 21/Feb/07 01:07 PM

I don't think we ran out of disk space. Both /opt/pulse and /root/pulse-data are on /dev/md1 which has 26G available

nextgen1:~/pulse-data/config # df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md1 35G 8.4G 26G 25% /
udev 2.0G 152K 2.0G 1% /dev
/dev/md0 137G 60G 78G 44% /repos
/dev/sdd3 27G 7.6G 19G 29% /tmp
continuum:/repos/ivyrep-sandbox
1014G 101G 913G 10% /repos/ivyrep-sandbox

We can try restoring from backup, but I'm not sure that will help, since we were seeing intermittent crashes before the backup too (although not as frequent). Perhaps the answer is to send you our data and see if you can help fix it. What is the best way to send you this data?

[ Permlink | « Hide ]

Daniel Ostermeier - 21/Feb/07 01:59 PM

Hi Alex,

Plenty of disk space available? This is good. Having another look at the second exception you pasted above, I noticed the OOME reference:

2/20/07 10:40:58 PM PST (57 ms ago) failure severe count :: 1 org.hibernate.util.JDBCExceptionReporter :: logExceptions

out of memory

This OOME is likely due to the HSQLDB and pulse running within the same JVM. As the pulse dataset gets larger, with more builds/features/tests, the memory requirements will increase until, as you have seen, something gives. The positive is that it could well be that adding more memory to the pulse process will help avoid the OOME, and hence stop the crashing. You can increase the amount of memory by setting the JAVA_OPTS environment property, either in the startup scripts or in the shell environment, depending on how you are starting pulse.

Whatever the case, I would strongly encourage you to migrate pulse from the embedded HSQLDB instance to either postgres or mysql. Doing so will both improve the performance and memory requirements of the pulse process and help guard against database corruption.

I am happy to take a look at your data for you. The easiest thing to would be to upload the backup pulse took when you upgraded. I will send you details of where to upload the file in a separate email.

Cheers,
-Daniel

[ Permlink | « Hide ]

Daniel Ostermeier - 22/Feb/07 09:42 AM

Hi Alex,

I have had a look through the backup 'auto-backup-1.2.11.zip', and everything appears as it should. Without being able to test everything, I have clicked through a few of the projects and results without problems. So it appears that there are no data corruption issues with the snapshot of the database.

Cheers,
-Daniel

[ Permlink | « Hide ]

Alex Chapman - 02/Mar/07 04:50 AM

Since we recreated our environment in MySQL we haven't had a recurrence of this problem. I think you can safely resolve it as unreproducable. Thanks.

[ Permlink | « Hide ]

Daniel Ostermeier - 03/Mar/07 01:03 AM

Hi Alex

That is excellent news. Thanks for getting back to me. Please let me know if you have any further problems.

Cheers,
-Daniel