[Neo] Typing of nodes (WAS: Java outof 64 GB ram)
rick.bullotta at burningskysoftware.com
rick.bullotta at burningskysoftware.com
Fri Feb 26 15:02:33 CET 2010
Hi, Mattias.
We had a similar requirement (to track the "type" of a node), and while
we were able to achieve this with relationships in very constrained
scenarios, we could not do so in a more general sense. Let me explain
and maybe you can suggest some ideas.
We have "collections" of typed entities: dogs, trees, cars. Each "dog"
node relates to a "dogs" node, which in turn relates to the root node.
This way, we can easily traverse all of the "dogs" and "trees", etc...
Imagine then a collection of "locations". Dogs, trees, and cars can be
related to a location. Thus, we can find all of the entities
associated with a specific location. However, when we traverse these
relationships from location to other entities, we lose the "type-ness",
since all we have is a generic node identity at the end of the
relationship. We thought about iterating through a series of
"hasSingleRelationship" checks on all of the known "type" relationships
on the node, but that seemed excessive. Thus, we now set a property
on each node that corresponds to the string representation
(enum.name()) for its "type" relationship. We avoided using the
ordinal of the enum as the property since that could be fragile and
also to make the property more "readable" during debugging.
One advantage of this approach is that it is very easy to "invert" this
String property using enum.valueOf() to turn it back into the
underlying relationship type enum (for faster comparisons, different
types of traversals, etc.). Another is that it provides a much easier
way to deal with import/export of graph subsets since we can infer
directly from the node what it is and how to serialize/deserialize it
to XML, JSON, etc. I haven't looked at the source code for the various
POJO<->Neo libraries, but I'd suspect that they have to tag the node
with a "type" property as well.
Thus far, it seems to work well. I welcome your thoughts as to
alternative approaches, however.
Rick
-------- Original Message --------
Subject: Re: [Neo] Java outof 64 GB ram
From: Mattias Persson <mattias at neotechnology.com>
Date: Fri, February 26, 2010 5:11 am
To: Neo user discussions <user at lists.neo4j.org>
2010/2/26 Miguel Ãngel Ãguila <maguila at ac.upc.edu>:
> I will explain me better.
> Every node has:
> - One String that have 5 chars.
> - One String that have 3 chars.
> - One String that have 4 chars.
> - One String that have 100 chars.
Is one of them any kind of "node type"? Because often you can infer a
type from its relationships or surrounding environment, making that
property redundant in a way. (nodes can often represent one or more
logical entities, hence a single type wouldn't suffice and that's why
it isn't built in to the API).
>
> Mike
>
> El vie, 26-02-2010 a las 10:05 +0100, Miguel Ãngel Ãguila Lorente
> escribió:
>> I don't know if I'm answering your question but every node has 4
pairs
>> of String, therefore 4 pairs of char(15).
>>
>> Mike
>>
>>
>>
>> El jue, 25-02-2010 a las 07:18 -0700,
>> rick.bullotta at burningskysoftware.com escribió:
>> > Yes, exactly the question. If it's a big string, it is quite
possible
>> > to have the file grow this large.
>> >
>> >
>> >
>> > Miguel, approximately how long is the property value?
>> >
>> >
>> >
>> > -------- Original Message --------
>> > Subject: Re: [Neo] Java outof 64 GB ram
>> > From: Johan Svensson <johan at neotechnology.com>
>> > Date: Thu, February 25, 2010 7:13 am
>> > To: Neo user discussions <user at lists.neo4j.org>
>> > If you store 322M strings (one string property/relationship),
were
>> > each string is 200 bytes, the string store will be 60GB+ in
size.
>> > How large are the strings you are storing?
>> > -Johan
>> > On Thu, Feb 25, 2010 at 2:55 PM,
<rick.bullotta at burningskysoftware.com>
>> > wrote:
>> > > The string propertystore file seems really, really large to
me,
>> > based
>> > > on the # of nodes/relationships/properties. The reason I
ask is
>> > that I
>> > > saw similar behavior in some of our early testing.
Extremely large
>> > > string propertystores file. It would be helpful to
understand why
>> > the
>> > > file gets that large, and whether it is related to some of
the
>> > memory
>> > > stress issues.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > -------- Original Message --------
>> > > Subject: Re: [Neo] Java outof 64 GB ram
>> > > From: Miguel ngel_guila Lorente <maguila at ac.upc.edu>
>> > > Date: Thu, February 25, 2010 1:57 am
>> > > To: Neo user discussions <user at lists.neo4j.org>
>> > > There are 322 million relationships, no 57 million
relationships.
>> > The
>> > > only propierty I save in the relationship is a String. In
this 322
>> > > million relationships there are 3 diferents types of
relations.
>> > > Do you think that 13 GB is a strange number? Why?
>> > _______________________________________________
>> > Neo mailing list
>> > User at lists.neo4j.org
>> > [1][1]https://lists.neo4j.org/mailman/listinfo/user
>> >
>> > References
>> >
>> > 1. [2]https://lists.neo4j.org/mailman/listinfo/user
>> > _______________________________________________
>> > Neo mailing list
>> > User at lists.neo4j.org
>> > [3]https://lists.neo4j.org/mailman/listinfo/user
>>
>>
>> _______________________________________________
>> Neo mailing list
>> User at lists.neo4j.org
>> [4]https://lists.neo4j.org/mailman/listinfo/user
>
>
> _______________________________________________
> Neo mailing list
> User at lists.neo4j.org
> [5]https://lists.neo4j.org/mailman/listinfo/user
>
--
Mattias Persson, [mattias at neotechnology.com]
Neo Technology, [6]www.neotechnology.com
_______________________________________________
Neo mailing list
User at lists.neo4j.org
[7]https://lists.neo4j.org/mailman/listinfo/user
References
1. https://lists.neo4j.org/mailman/listinfo/user
2. https://lists.neo4j.org/mailman/listinfo/user
3. https://lists.neo4j.org/mailman/listinfo/user
4. https://lists.neo4j.org/mailman/listinfo/user
5. https://lists.neo4j.org/mailman/listinfo/user
6. http://www.neotechnology.com/
7. https://lists.neo4j.org/mailman/listinfo/user
More information about the User
mailing list