[Neo4j] ​Neo4j, third party JTA Transaction Manager integration and potential bug.

Andreas Guenther andreas.guenther at web.de
Sat Aug 14 18:48:39 CEST 2010


Francesco,

We tried the same thing where we wanted to have Neo participate in MySQL 
and Compass transactions. After several attempts of trying to combine 
them all using JTA/XA and going down a similar path you did we finally 
ended up using chained transactions as outlined @ 
http://www.javaworld.com/javaworld/jw-01-2009/jw-01-spring-transactions.html. 
In our specific case Compass automatically registers with MySQL so that 
our chain consists of database transaction manager and Neo4j transaction 
manager only. Certainly, there's a slight chance to run into commit 
inconsistencies based on how this works but that chance is very slim 
knowing the failure point. This has been working for us for over a year 
now and we even experienced some MySQL and Neo datasource failures 
leading to proper rollbacks. Maybe this is something you want to further 
explore. Another option we tried is to build the same transaction 
participation around Neo that Compass uses but we eventually disliked 
the dependencies resulting from this effort.

I am very curious how you will end up using Neo4j with other transaction 
managers. So far, I haven't seen any concrete example of Database-Neo4j 
JTA/XA transactions in practice other than many claiming it should work ;)

Hope this somewhat helps,
-Andreas

On 8/14/2010 4:49 AM, Degrassi Francesco wrote:
> Hello everybody,
>
> We're evaluating Neo4j for integration in two products we are developing.
>
> One of the aspects we are interested in is making an embedded neo4j instance
> participate in JTA/XA transactions with a postgresql datasource and possibly
> JMS.
>
>   
>
> I'm sorry if this is a bit long but the matter is complex.
>
>   
>
>  From what i understood after a bit of research, currently neo4j does not allow
> to use an external TransactionManager out of the box. 
>
> I then dug into the source code and simply patched the following classes to
> allow me to pass an external JTA TransactionManager instance down to the
> TxModule, replacing Neo4j own TxManager.
>
>   
>
> For our tests we used the following:
>
>   * Neo4J 1.1
>
>   * Atomikos TransactionEssentials JTA Manager 3.6.5
>
>   * postgresql XA-enabled JDBC driver wrapped in an AtomikosDataSourceBean
>
>   
>
> We created a simple test app which creates a very minimal Spring
> ApplicationContext which sets up the neo4j database, the JTA manager, the JDBC
> datasource and a sample transactional service which interacts with both the
> relational database and the graph database; we used Spring
> "tx:annotation-driven" infrastructure to handle begin/commit/rollback of the
> global transactions.
>
>   
>
> Upon starting it, we got the following exception:
>
>   
>
> org.neo4j.kernel.impl.transaction.LockNotFoundException: No transaction lock
> element found for Placebo tx for thread
> Thread[net.emaze.springexperiment.App.main(),5,net.emaze.springexperiment.App]
>
> at org.neo4j.kernel.impl.transaction.RWLock.releaseWriteLock(RWLock.java:345)
>
> at
> org.neo4j.kernel.impl.transaction.LockManager.releaseWriteLock(LockManager.java:202)
>
> at org.neo4j.kernel.impl.core.LockReleaser.releaseLocks(LockReleaser.java:337)
>
> at
> org.neo4j.kernel.impl.core.LockReleaser$ReadOnlyTxReleaser.afterCompletion(LockReleaser.java:713)
>
> at com.atomikos.icatch.jta.Sync2Sync.afterCompletion(Sync2Sync.java:91)
>
> at com.atomikos.icatch.imp.SynchToFSM.doAfterCompletion(SynchToFSM.java:38)
>
> at com.atomikos.icatch.imp.SynchToFSM.entered(SynchToFSM.java:59)
>
> at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:197)
>
> at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:288)
>
> at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:498)
>
> at
> com.atomikos.icatch.imp.CoordinatorImp.setStateHandler(CoordinatorImp.java:328)
>
> at
> com.atomikos.icatch.imp.CoordinatorStateHandler.commit(CoordinatorStateHandler.java:730)
>
> at
> com.atomikos.icatch.imp.IndoubtStateHandler.commit(IndoubtStateHandler.java:225)
>
> at com.atomikos.icatch.imp.CoordinatorImp.commit(CoordinatorImp.java:828)
>
> at com.atomikos.icatch.imp.CoordinatorImp.terminate(CoordinatorImp.java:1127)
>
> at
> com.atomikos.icatch.imp.CompositeTerminatorImp.commit(CompositeTerminatorImp.java:151)
>
> at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:298)
>
> at
> com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:612)
>
> at
> com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:168)
>
> at
> org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
>
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
>
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
>
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
>
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
>
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
>
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>
> at $Proxy9.success(Unknown Source)
>
> at net.emaze.springexperiment.App.main(App.java:21)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:291)
>
> at java.lang.Thread.run(Thread.java:619)
>
>   
>
> Further analysis resulted in the following hypothesis.
>
> 1. the external JTA TransactionManager correctly completes the transaction
>
> 2. the TransactionManager then calls the Synchronization.afterCompletion()
> method on the LockReleaser$ReadOnlyTxReleaser for the (now finished)
> transaction.
>
> 3. This causes LockReleaser.releaseLocks() to be called, which through various
> calls invokes the RWLock.releaseWriteLock() method.
>
> 4. At this point RWLock.releaseWriteLock() calls
> RagManager.getCurrentTransaction(), which delegates to the TransactionManager,
> which correctly returns null, since the transaction is already completed.
>
> 5. RWLock.releaseWriteLock then creates a PlaceboTransaction and looks up the
> txLockElementMap for the TxLockElement associated to the PlaceboTransaction.
> Since the TxLockElement it is looking for was previously associated to the
> actual global transaction which was just completed, this fails and the
> LockNotFoundException is thrown.
>
>   
>
> The problem, IMHO, is caused by the fact that, in the execution flow of an
> afterCompletion() callback, the TransactionManager is requested for the current
> global transaction, which does not exist at this point.
>
> To fix it, i hacked the following classes to allow the
> LockReleaser$ReadOnlyTxReleaser to pass the transaction reference stored at
> creation time down to the RWLock.releaseWriteLock() method, which now can look
> up the correct TxLockElement in the txLockElementMap using the correct key.
>
>   
>
> Changed classes:
>
> org.neo4j.kernel.core.LockReleaser
>
> org.neo4j.kernel.impl.transaction.LockManager
>
> org.neo4j.kernel.impl.transaction.RWLock
>
>   
>
> At this point, my doubts are the following:
>
>   * I thought that injecting an external TransactionManager for neo4j to use
> would be easier; is my approach valid in this regard? Is there a specific reason
> why the TransactionManager implementation is not easily pluggable ?
>
>   * Regarding the potential bug i found after injecting an external
> TransactionManager, is my analysis correct ? Can it, or some other solution, be
> integrated in a future release ?
>
>   
>
> Handling this scenario is pretty important for us in choosing if we should
> proceed in using neo4j in our app.
>
> I can send you the two patches i made fron neo4j-kernel trunk and the sample
> app, if needed; i did not know if it was ok to attach it to the mailing list
> post.
>
>   
>
> Thanks for your time.
>
>   
>
> Francesco Degrassi
> _______________________________________________
> Neo4j mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>    



More information about the User mailing list