[Neo] Persisting store metadata

Tobias Ivarsson tobias.ivarsson at neotechnology.com
Tue Jan 5 12:18:17 CET 2010


That's an interesting idea. However something in my gut tells me that it's
not good/better:
* It's still open for people to mock about with, however now it's in the
node space using up a relationship type that is now unusable by the
application (at least as a relationship from the reference node).
* This is intended (at least by how I envision it) to be read by the kernel
to know which components to load in order to have a consistent store, it
seems to me like having the data in the node space would require too much
bootstrapping before it can be accessed. Remember that this information is
meant to be part of the bootstrapping process.

It does have the advantage that the consistency of the information would be
guaranteed.

/Tobias

On Tue, Jan 5, 2010 at 9:34 AM, Mattias Persson
<mattias at neotechnology.com>wrote:

> Great initiative.
>
> Instead of storing such information in a plain-text file it could be
> stored in the neo4j store itself (as a sub-reference node or
> something).
>
> I don't know if that'd be good/better though.
>
> 2010/1/4 Tobias Ivarsson <tobias.ivarsson at neotechnology.com>:
> > Hi,
> >
> > Neo4j today is made up of a core service that persists nodes,
> relationships
> > and properties. In addition to that there are a number of additional
> > services, where index-util is probably the most used and most important
> at
> > the moment. index-util is also a good example of the problem I would like
> to
> > discuss in this email.
> >
> > With the current architecture there is no way of introspecting which
> > additional services have been initialized with a particular Neo store,
> for
> > example there is no way of telling which (if any) IndexService has been
> > used. This causes problems with the transaction recovery process, where
> the
> > recovery mechanism today has to know about all possible extension
> services
> > without having a compile time dependency on them. Needless to say this
> > causes a mess. The code for this is ugly and borderline-buggy. It also
> > causes a problem when introspecting a store, or even restarting your
> > software, since it is up to the programmer to remember to restart all of
> the
> > same services as last time.
> >
> > I would like to propose an addition to the store, in a separate metadata
> > file in the store directory, where we store a simple list of all started
> > additional services. Does anyone have any suggestions to what this should
> > look like to be reasonable future proof (i.e. be able to handle some
> service
> > that is not implemented yet as well as the current index-utils).
> >
> > What comes to mind is something similar to the Java ServiceLoader API
> [1],
> > but simplified with the fact that we can require all classes referenced
> in
> > the file to implement one specific interface, and we know where the file
> > will be, in effect taking out all the complications involving
> ClassLoaders.
> > What I'm worrying about is that storing Java classnames in a file will
> make
> > this metadata unusable to any future implementations of the NeoStore
> format
> > outside of the Java platform. Any suggestions to how information about
> > loaded services can be stored is welcome, my idea of a clear text file
> might
> > not even be good (since people have a tendency to think they can patch
> those
> > manually).
> >
> > Since we have a community of smart and entrepreneurial individuals I
> thought
> > I'd throw the question out here and see if I got any good responses.
> >
> > Cheers,
> > Tobias
> >
> > [1] http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html
> > Simple explanation of how it works:
> >  * You have an interface with fully qualified classname:
> > com.somedomain.someproduct.ServiceInterface
> >  * In each jar that provides implementations of the ServiceInterface you
> add
> > a file called
> >   META-INF/services/com.somedomain.someproduct.ServiceInterface
> >  * In this file you put the fully qualified classnames of each
> implementing
> > class separated by newline
> >  * In your code you do: ServiceLoader<ServiceInterface> impls =
> > ServiceLoader.load(ServiceInterface.class);
> > --
> > Tobias Ivarsson <tobias.ivarsson at neotechnology.com>
> > Hacker, Neo Technology
> > www.neotechnology.com
> > Cellphone: +46 706 534857
> > _______________________________________________
> > Neo mailing list
> > User at lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
>
>
>
> --
> Mattias Persson, [mattias at neotechnology.com]
> Neo Technology, www.neotechnology.com
> _______________________________________________
> Neo mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Tobias Ivarsson <tobias.ivarsson at neotechnology.com>
Hacker, Neo Technology
www.neotechnology.com
Cellphone: +46 706 534857


More information about the User mailing list