[Neo] Possible causes for: org.apache.lucene.index.MergePolicy$MergeException

Mattias Persson mattias at neotechnology.com
Fri Jan 29 15:24:32 CET 2010


2010/1/29 Symeon (Akis) Papadopoulos <papadop at iti.gr>:
>
>> I just fixed the bug and your benchmarks run fine w/o any modification
>> on my machine, so they should run fine on your end too!
>>
> Thank you Mattias!
> So, in the end it was a neo4j-index bug? Thus I assume we don't need to
> modify our code (of course we'll take into consideration your previous
> recommendations for using the BatchInserter, but for the sake of it, we
> would also like to run the benchmark in transactional mode as well).
Yep, there shouldn't be any need to modify your code at all. Yep, I
understand if you'd like to benchmark the transactional mode as well!
>
>> The bug is just committed so the latest build will be available within
>> an hour. I see that you aren't using maven, am I right? In that case
>> you can download it from
>> http://m2.neo4j.org/org/neo4j/neo4j-index/1.0-SNAPSHOT/ (look for .jar
>> files around this date/time + 1 hour or something). Or you could check
>> out the code and build it yourself:
>>
>>   svn co https://svn.neo4j.org/components/index/trunk
>>
> Out of the two built jars of 29th Jan 2010, I guess we should use the
> most recent one (neo4j-index-1.0-20100129.113202-3.jar), right?
Hmm, I don't think it have been deployed yet... so wait a little
longer for this change to be deployed. We have a little "lag" in
deployed jar files, which is kind of annoying in situations like this
:)
>
>
>> You could update to the latest neo4j-kernel as well, but maybe that's
>> not necessary though.
>>
> We'll first try out to update just the neo-index.
> If we face more problems we'll proceed with the kernel update as well.
>
> Thanks again for the prompt response!
It should be me thanking you, you gave me a complete test case
triggering a bug... very nice
>
>> 2010/1/29 Mattias Persson <mattias at neotechnology.com>:
>>
>>> 2010/1/29 Symeon (Akis) Papadopoulos <papadop at iti.gr>:
>>>
>>>> Mattias Persson wrote:
>>>>
>>>>> Another problem I see is that you're having too granular transactions
>>>>> which will slow down the insertion process quite a bit. Try grouping a
>>>>> couple of thousands operations in one transaction and you'll see a
>>>>> performance boost!
>>>>>
>>>>> FYI: I can trigger the problem you were having with lucene "too many
>>>>> open files" issue. And I'm almost 100% sure that it will be resolved
>>>>> if you increase the span of your transactions.
>>>>>
>>>>>
>>>> That's what we are already doing through the class SimpleBatchTxManager
>>>> (in package org.neo4j.util). We group read and write transactions in
>>>> batches of some thousands. Isn't this correct?
>>>>
>>> You're right, sorry for missing that.
>>>
>>> However I just found the bug which causes this... hang on, I'll fix it
>>> in an hour or two!
>>>
>>>> _______________________________________________
>>>> Neo mailing list
>>>> User at lists.neo4j.org
>>>> https://lists.neo4j.org/mailman/listinfo/user
>>>>
>>>>
>>>
>>> --
>>> Mattias Persson, [mattias at neotechnology.com]
>>> Neo Technology, www.neotechnology.com
>>>
>>>
>>
>>
>>
>>
>
> _______________________________________________
> Neo mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Mattias Persson, [mattias at neotechnology.com]
Neo Technology, www.neotechnology.com


More information about the User mailing list