[Neo4j] Neo4jRestNet Update

Tatham Oddie tatham at oddie.com.au
Tue Oct 25 20:41:51 CEST 2011


Peter,

In this case we really do want all that data, so that paged enumerator actually handles it quite nicely.

I'm not sure why you're saying that 'JSON needs to be built before sending over the request'. JSON is almost always written forward-only, and HTTP has supported streaming for a long time now. In fact, the Twitter Streaming APIs are just that - streaming JSON.


-- Tatham

-----Original Message-----
From: user-bounces at lists.neo4j.org [mailto:user-bounces at lists.neo4j.org] On Behalf Of Peter Neubauer
Sent: Tuesday, 25 October 2011 2:21 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] Neo4jRestNet Update

Tatham,
yes, serialization is an issue in that the JSON needs to be built before sending over the request. What you could do is to trim the query down by not pulling out the nodes, but just the properties you need (including node IDs) which will dramatically change the amount of data transferred, see e.g.

http://docs.neo4j.org/chunked/snapshot/cypher-plugin.html#rest-api-send-queries-with-parameters

or

http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html#rest-api-group-count

Would that work?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org              - NOSQL for the Enterprise.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.



On Mon, Oct 24, 2011 at 7:35 AM, Tatham Oddie <tatham at oddie.com.au> wrote:
> More info:
>
> That's all transparent ... consumers of our client just enumerate the result like normal and see no difference. Under the covers, this gets split up into pages of 100 nodes loaded on-demand.
>
> Added it because we wanted to pull out 38k nodes from a query and the 
> REST plugin was exploding the Java heap space trying to append the 
> string. :) (Um, streaming anyone? ;))
>
>
> -- Tatham
>
>
> -----Original Message-----
> From: user-bounces at lists.neo4j.org 
> [mailto:user-bounces at lists.neo4j.org] On Behalf Of Romiko Derbynew
> Sent: Monday, 24 October 2011 10:46 PM
> To: Neo4j user discussions
> Subject: Re: [Neo4j] Neo4jRestNet Update
>
> Tatham has also added support for paging queries, so queries returning more than 100 results are retrieved via an enumerator per 100 results, this optimises the heap usage as well, by leveraging the groovy Take and Drop functions.
>
> Cheers.
>
> -----Original Message-----
> From: user-bounces at lists.neo4j.org 
> [mailto:user-bounces at lists.neo4j.org] On Behalf Of Tatham Oddie
> Sent: Friday, 21 October 2011 4:37 PM
> To: Neo4j user discussions
> Subject: Re: [Neo4j] Neo4jRestNet Update
>
> Hi Kan,
>
> FYI - we added parametized Gremlin queries in our implementation and have seen a nice memory heap improvement on the Java side as a result.
>
> That is ... instead of:
>
> g.v(123).outE[[label:'FOO']]
>
> we send:
>
> {
>  query: 'g.v(p0).outE[[label:p1]]',
>  params: {
>    p0: 123,
>    p1: 'FOO'
>  }
> }
>
> This allows the query to be cached by neo4j and then just called with different parameters later.
>
>
> -- Tatham
>
>
> -----Original Message-----
> From: user-bounces at lists.neo4j.org 
> [mailto:user-bounces at lists.neo4j.org] On Behalf Of KanTube
> Sent: Thursday, 20 October 2011 9:36 AM
> To: user at lists.neo4j.org
> Subject: [Neo4j] Neo4jRestNet Update
>
> fwiw...
>
> I updated my .Net wrapper
> https://github.com/SepiaGroup/Neo4jRestNet/
>
> you can now use LINQ on the flilter and sort commands for example
>
> Node MyNode = Node.GetNode(123);
> GremlinScript script = new GremlinScript(MyNode)
>        .Out("Like")
>        .Filter(it => it.GetProperty("MyProp").ToLowerCase() == 
> "TestValue" ||
> it.GetProperty("AnotherProp").Contains("SomeValue"))
>
> IEnumerable<Node> ReturnNodes = Gremlin.Post<Node>(script);
>
>
> You can also return a DataTable
>
> GremlinScript script = new GremlinScript();
> script.NewTable("t")
>        .NodeIndexLookup(new Dictionary<string, object>() { { "FirstName" , "Jack"
> }, { "LastName", "Shaw" } })
>        .Filter(it => it.GetProperty("UID") == "jshaw")
>        .As("UserNode")
>        .OutE("Like")
>        .As("LikeRel")
>        .InV()
>        .As("FriendNode")
>        .Table("t", "UserNode", "LikeRel", "FriendNode")
>        
> .Append("{{it}}{{it.getProperty('{0}')}}{{it.getProperty('{1}')}} >> 
> -1; t","Date", "FirstName");
>
> DataTable tbl = Gremlin.GetTable(script);
>
>
> The above table example will submit the following Gremlin script to Neo4j:
>
> t = new Table();
> g.V[['FirstName':'Jack','LastName':'Shaw']]
>        .filter{it.getProperty('UID') == 'jshaw'}
>        .as('UserNode')
>        .outE('Likes')
>        .as('LikeRel')
>        .inV()
>        .as('FriendNode')
>        .table(t,
> ['UserNode','LikeRel','FriendNode']){it}{it.getProperty('Date')}{it.ge
> tProperty('FirstName')}
>>> -1; t");
>
> And the returned table will have column names of "UserNode", 
> "LikeRel", "FriendNode" with typed values of Node, DateTime, string 
> (assuming the properties are stored with the correct types)
>
>
>
>
> --
> View this message in context: 
> http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jRestNet-U
> pdate-tp3436032p3436032.html Sent from the Neo4j Community Discussions 
> mailing list archive at Nabble.com.
> _______________________________________________
> Neo4j mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
> _______________________________________________
> Neo4j mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
> _______________________________________________
> Neo4j mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
> _______________________________________________
> Neo4j mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
_______________________________________________
Neo4j mailing list
User at lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


More information about the User mailing list