[Neo] Typing of nodes (WAS: Java outof 64 GB ram)

rick.bullotta at burningskysoftware.com rick.bullotta at burningskysoftware.com
Fri Feb 26 15:02:33 CET 2010


   Hi, Mattias.



   We had a similar requirement (to track the "type" of a node), and while
   we were able to achieve this with relationships in very constrained
   scenarios, we could not do so in a more general sense.  Let me explain
   and maybe you can suggest some ideas.



   We have "collections" of typed entities: dogs, trees, cars.  Each "dog"
   node relates to a "dogs" node, which in turn relates to the root node.
   This way, we can easily traverse all of the "dogs" and "trees", etc...



   Imagine then a collection of "locations".  Dogs, trees, and cars can be
   related to a location.  Thus, we can find all of the entities
   associated with a specific location.  However, when we traverse these
   relationships from location to other entities, we lose the "type-ness",
   since all we have is a generic node identity at the end of the
   relationship.  We thought about iterating through a series of
   "hasSingleRelationship" checks on all of the known "type" relationships
   on the node, but that seemed excessive.   Thus, we now set a property
   on each node that corresponds to the string representation
   (enum.name()) for its "type" relationship.  We avoided using the
   ordinal of the enum as the property since that could be fragile and
   also to make the property more "readable" during debugging.



   One advantage of this approach is that it is very easy to "invert" this
   String property using enum.valueOf() to turn it back into the
   underlying relationship type enum (for faster comparisons, different
   types of traversals, etc.).  Another is that it provides a much easier
   way to deal with import/export of graph subsets since we can infer
   directly from the node what it is and how to serialize/deserialize it
   to XML, JSON, etc.  I haven't looked at the source code for the various
   POJO<->Neo libraries, but I'd suspect that they have to tag the node
   with a "type" property as well.



   Thus far, it seems to work well.  I welcome your thoughts as to
   alternative approaches, however.



   Rick

   -------- Original Message --------
   Subject: Re: [Neo] Java outof 64 GB ram
   From: Mattias Persson <mattias at neotechnology.com>
   Date: Fri, February 26, 2010 5:11 am
   To: Neo user discussions <user at lists.neo4j.org>
   2010/2/26 Miguel Ãngel Ãguila <maguila at ac.upc.edu>:
   > I will explain me better.
   > Every node has:
   > - One String that have 5 chars.
   > - One String that have 3 chars.
   > - One String that have 4 chars.
   > - One String that have 100 chars.
   Is one of them any kind of "node type"? Because often you can infer a
   type from its relationships or surrounding environment, making that
   property redundant in a way. (nodes can often represent one or more
   logical entities, hence a single type wouldn't suffice and that's why
   it isn't built in to the API).
   >
   > Mike
   >
   > El vie, 26-02-2010 a las 10:05 +0100, Miguel Ãngel Ãguila Lorente
   > escribió:
   >> I don't know if I'm answering your question but every node has 4
   pairs
   >> of String, therefore 4 pairs of char(15).
   >>
   >> Mike
   >>
   >>
   >>
   >> El jue, 25-02-2010 a las 07:18 -0700,
   >> rick.bullotta at burningskysoftware.com escribió:
   >> > Yes, exactly the question.  If it's a big string, it is quite
   possible
   >> >    to have the file grow this large.
   >> >
   >> >
   >> >
   >> >    Miguel, approximately how long is the property value?
   >> >
   >> >
   >> >
   >> >    -------- Original Message --------
   >> >    Subject: Re: [Neo] Java outof 64 GB ram
   >> >    From: Johan Svensson <johan at neotechnology.com>
   >> >    Date: Thu, February 25, 2010 7:13 am
   >> >    To: Neo user discussions <user at lists.neo4j.org>
   >> >    If you store 322M strings (one string property/relationship),
   were
   >> >    each string is 200 bytes, the string store will be 60GB+ in
   size.
   >> >    How large are the strings you are storing?
   >> >    -Johan
   >> >    On Thu, Feb 25, 2010 at 2:55 PM,
   <rick.bullotta at burningskysoftware.com>
   >> >    wrote:
   >> >    >   The string propertystore file seems really, really large to
   me,
   >> >    based
   >> >    >   on the # of nodes/relationships/properties.  The reason I
   ask is
   >> >    that I
   >> >    >   saw similar behavior in some of our early testing.
   Extremely large
   >> >    >   string propertystores file.  It would be helpful to
   understand why
   >> >    the
   >> >    >   file gets that large, and whether it is related to some of
   the
   >> >    memory
   >> >    >   stress issues.
   >> >    >
   >> >    >
   >> >    >
   >> >    >
   >> >    >
   >> >    >   -------- Original Message --------
   >> >    >   Subject: Re: [Neo] Java outof 64 GB ram
   >> >    >   From: Miguel ngel_guila Lorente <maguila at ac.upc.edu>
   >> >    >   Date: Thu, February 25, 2010 1:57 am
   >> >    >   To: Neo user discussions <user at lists.neo4j.org>
   >> >    >   There are 322 million relationships, no 57 million
   relationships.
   >> >    The
   >> >    >   only propierty I save in the relationship is a String. In
   this 322
   >> >    >   million relationships there are 3 diferents types of
   relations.
   >> >    >   Do you think that 13 GB is a strange number? Why?
   >> >    _______________________________________________
   >> >    Neo mailing list
   >> >    User at lists.neo4j.org
   >> >    [1][1]https://lists.neo4j.org/mailman/listinfo/user
   >> >
   >> > References
   >> >
   >> >    1. [2]https://lists.neo4j.org/mailman/listinfo/user
   >> > _______________________________________________
   >> > Neo mailing list
   >> > User at lists.neo4j.org
   >> > [3]https://lists.neo4j.org/mailman/listinfo/user
   >>
   >>
   >> _______________________________________________
   >> Neo mailing list
   >> User at lists.neo4j.org
   >> [4]https://lists.neo4j.org/mailman/listinfo/user
   >
   >
   > _______________________________________________
   > Neo mailing list
   > User at lists.neo4j.org
   > [5]https://lists.neo4j.org/mailman/listinfo/user
   >
   --
   Mattias Persson, [mattias at neotechnology.com]
   Neo Technology, [6]www.neotechnology.com
   _______________________________________________
   Neo mailing list
   User at lists.neo4j.org
   [7]https://lists.neo4j.org/mailman/listinfo/user

References

   1. https://lists.neo4j.org/mailman/listinfo/user
   2. https://lists.neo4j.org/mailman/listinfo/user
   3. https://lists.neo4j.org/mailman/listinfo/user
   4. https://lists.neo4j.org/mailman/listinfo/user
   5. https://lists.neo4j.org/mailman/listinfo/user
   6. http://www.neotechnology.com/
   7. https://lists.neo4j.org/mailman/listinfo/user


More information about the User mailing list