[Neo] Batch Insert in Python

Peter Neubauer neubauer.peter at gmail.com
Fri Apr 16 16:12:21 CEST 2010


Jon,
Tobias knows more about the current support in Python for
batchinserts, but there is batchinsert support in Java, see
http://wiki.neo4j.org/content/Batch_Insert

That should crank up your insert speed a LOT. Let us know if that is ok for you?

Cheers,

/peter neubauer

COO and Sales, Neo Technology

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://nosqleu.com                    - The biggest NOSQL event. Ever.
http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



On Thu, Apr 15, 2010 at 8:13 PM, Jon Noronha <cheeselord at gmail.com> wrote:
> Hello,
>
> I'm wondering if neo4j.py has any way of running a batch insert.
>
> I'm interested in building a DB of about 10 million nodes and 60
> million edges. My initial approach was to add each node one at a time
> in a transaction. Each transaction would create the node, then add
> each edge coming off it, adding the other node if necessary. Nodes and
> edges each have a small number of properties.
>
> This ended up taking forever and would have taken 25 days to run on
> the whole DB, so now I'm trying to find a better way. One thing I
> tried was doing bulk insertions, adding all the nodes first and then
> adding edges in batches of 100,000. This seems faster but I still
> wonder if I'm wasting a lot of time with transactions, etc.
>
> How do you all recommend going about this?
>
> Thanks,
> Jon
>
> PS: I'm brand new to Neo4j, so please take nothing for granted in my
> understanding :)
> _______________________________________________
> Neo mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>


More information about the User mailing list