[Neo4j] Creating a graph database with BatchInserter and getting the node degree of every node

Linan Wang tali.wang at gmail.com
Tue Sep 20 17:53:24 CEST 2011


hi stephan,
my theory is that most of the time would spent on retrieving imcoming
relationships. could you try again but this time only retrieve
outgoing relationship?

for (Node node : db.getAllNodes()) {
                       if (node.getId() > 0) {
                               long test = System.currentTimeMillis();
                               Iterable<Relationship> rels =
node.getRelationships(knows, Direction.OUTGOING);
                               System.out.print("Retrieval:"
                                               +
(System.currentTimeMillis() - test));

                               test = System.currentTimeMillis();
                               int count =
com.google.common.collect.Iterables.size(rels);
                               System.out.print("ms; Counting:"
                                               +
(System.currentTimeMillis() - test));
                               System.out.println("ms; number of
edges:" + count);
                       }
               }

On Tue, Sep 20, 2011 at 4:37 PM, st3ven <st3ven at web.de> wrote:
> Hello again,
>
> the bottle neck is at the iteration.
> I did some tests with it to check whether the iteration or relationship
> retrievel is to slow.
>
> My test results look like this:
>
> Retrieval:1ms; Counting:158ms; number of edges:116407
> Retrieval:0ms; Counting:2ms; number of edges:1804
> Retrieval:0ms; Counting:0ms; number of edges:22
> Retrieval:0ms; Counting:0ms; number of edges:31
> Retrieval:0ms; Counting:0ms; number of edges:39
> Retrieval:0ms; Counting:2ms; number of edges:1213
> Retrieval:0ms; Counting:0ms; number of edges:57
> Retrieval:0ms; Counting:36ms; number of edges:59420
> Retrieval:0ms; Counting:335ms; number of edges:175156
> Retrieval:1ms; Counting:168ms; number of edges:146697
> Retrieval:0ms; Counting:354ms; number of edges:285051
> Retrieval:0ms; Counting:0ms; number of edges:50
> Retrieval:0ms; Counting:11ms; number of edges:20960
> Retrieval:0ms; Counting:0ms; number of edges:43
> Retrieval:0ms; Counting:0ms; number of edges:51
> Retrieval:0ms; Counting:1ms; number of edges:647
> Retrieval:0ms; Counting:5ms; number of edges:10216
> Retrieval:0ms; Counting:2ms; number of edges:3444
> Retrieval:0ms; Counting:0ms; number of edges:1128
> Retrieval:1ms; Counting:312ms; number of edges:319127
> Retrieval:1ms; Counting:0ms; number of edges:5
> Retrieval:0ms; Counting:760ms; number of edges:104741
> Retrieval:0ms; Counting:11ms; number of edges:9210
> Retrieval:0ms; Counting:0ms; number of edges:31
> Retrieval:1ms; Counting:3ms; number of edges:3116
> Retrieval:0ms; Counting:37ms; number of edges:70835
> Retrieval:0ms; Counting:383ms; number of edges:296445
> Retrieval:1ms; Counting:0ms; number of edges:120
> Retrieval:0ms; Counting:2ms; number of edges:1526
> Retrieval:0ms; Counting:0ms; number of edges:71
> Retrieval:0ms; Counting:42ms; number of edges:35960
> Retrieval:0ms; Counting:90ms; number of edges:9644
> Retrieval:0ms; Counting:186ms; number of edges:129981
> Retrieval:0ms; Counting:1ms; number of edges:1213
> Retrieval:1ms; Counting:143ms; number of edges:124495
> Retrieval:0ms; Counting:0ms; number of edges:58
> Retrieval:0ms; Counting:75ms; number of edges:56195
> Retrieval:0ms; Counting:99ms; number of edges:92574
> Retrieval:0ms; Counting:0ms; number of edges:13
> Retrieval:0ms; Counting:50ms; number of edges:26350
> Retrieval:0ms; Counting:2ms; number of edges:1856
> Retrieval:1ms; Counting:376ms; number of edges:114166
> Retrieval:0ms; Counting:9528ms; number of edges:11956
> Retrieval:0ms; Counting:50047ms; number of edges:12645
> Retrieval:1ms; Counting:43687ms; number of edges:15025
>
> The first results came up very fast, because they seem to have been cached
> cause I did that quite often.
> As you can see the last 4 results weren't cached and it took a huge amount
> of time to do the iteration over the relationships.
>
> I checked that with the following code:
>
> for (Node node : db.getAllNodes()) {
>                        if (node.getId() > 0) {
>                                long test = System.currentTimeMillis();
>                                Iterable<Relationship> rels = node.getRelationships(knows);
>                                System.out.print("Retrieval:"
>                                                + (System.currentTimeMillis() - test));
>
>                                test = System.currentTimeMillis();
>                                int count = com.google.common.collect.Iterables.size(rels);
>                                System.out.print("ms; Counting:"
>                                                + (System.currentTimeMillis() - test));
>                                System.out.println("ms; number of edges:" + count);
>                        }
>                }
> Is there maybe a possibilty to cache more relationships or do you have any
> idea hot to speedup the iteration progress.
>
> Thanks for your help again!
>
> Cheers,
> Stephan
>
> --
> View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Creating-a-graph-database-with-BatchInserter-and-getting-the-node-degree-of-every-node-tp3351599p3352415.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> _______________________________________________
> Neo4j mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Best regards

Linan Wang


More information about the User mailing list