[Neo4j] Why doInternalRecovery is necessary?

Mattias Persson mattias at neotechnology.com
Wed Oct 26 17:06:16 CEST 2011


Good question,

so neo4j implements all the ACID principles of transactions, but that
doesn't mean that there's a guarantee that when you've committed a
transaction the data is in the store files on disk. In fact that wouldn't be
performant at all. The data is written to the store files with buffering or
memory mapping depending on environment so when it actually trickles down to
actual disk is up to when neo4j decides to "unmap" that area of the store
file and when the OS decides to flush its buffers. Instead the only place
your transaction is guaranteed to exist on disk after a commit is in the
logical log, so for a clean shutdown all the buffers/memory mapped areas of
the store files are closed and forced to disk whereas a non-clean shutdown
(a crash or whatever) wouldn't do that and there would be no guarantee that
all the store file changes would be written. So in case of a non-clean
shutdown neo4j must look in the most recently active logical log and replay
all those transactions to the store files... and that's what the recovery
process does after a non-clean shutdown.

Best,
Mattias

2011/10/26 Balazs E. Pataki <pataki at dsd.sztaki.hu>

> Hi,
>
> after crashing my application the next time I start it up I get the usual
>
> Oct 26, 2011 3:28:42 PM
> org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog
> doInternalRecovery
> INFO: Non clean shutdown detected on log [/db/nioneo_logical.log.1].
> Recovery started ...
> INFO: Non clean shutdown detected on log [/db/index/lucene.log.1].
> Recovery started ...
>
> kind of messages.
>
> I have a 8GB database and now this recovery has been running for quite
> some time, and I started to thinking why is this recovery is necessary
> at all? AS far as I'm aware the transactions are ACID, so they either
> commit, or rollback, in both case the state of the database must be OK.
> In this case I could see no reason why to do any recovery (from
> transactional point of view). If the DB crash happens while transactions
> are open, then there coudl be some problems. But in my case I could live
> with loosing such transactions and just not have recovery at all.
>
> To sum up my question: is this doInternalRecovery process essential? And
> if not, is there a way to avoid it at startup?
>
> Thanks,
> ---
> balazs
>
> _______________________________________________
> Neo4j mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Mattias Persson, [mattias at neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com


More information about the User mailing list