[Neo] neo4j on .NET

Antonello Provenzano antonello at deveel.com
Wed May 6 02:05:20 CEST 2009


Johan,

The plan of using "common" architectures is a great idea and will fix
the issues on porting and maintaining neo4j to .NET/Mono (please
remember that, as contributor to Mono, I'm attached more to this
framework than to Microsoft's one and I always seek for the full
compatibility with it).

I will try to elaborate a plan of porting as soon as it will be
possible, coming back to you if i will find anything that will be an
obstacle to it. In the meanwhile, I thought that the name "neo4n"
would be appropriate for the project, being the "j" in the name
referred to "java".

Cheers.
Antonello


On Tue, May 5, 2009 at 8:22 PM, Johan Svensson <johan at neotechnology.com> wrote:
> Next release of Neo4j (b9) will have an option not to use memory
> mapped buffers but instead either use direct buffers or normal Java
> buffers. A .NET port could replace the
> org.neo4j.impl.nioneo.store.MappedPersistenceWindow with a different
> implementation that does not use the MappedByteBuffer (b9 will soon
> have such an implementation).
>
> In the transaction package any usage of MappedByteBuffer can be
> replaced with a direct or normal buffer also (they all just append
> data to some file).
>
> Hopefully I will get my test code of the new persistence windows into
> trunk soon. Once that is done a .NET port could make use of that and
> replace all other usages of memory mapped I/O with normal buffers.
>
> Regards,
> Johan
>
> On Sun, May 3, 2009 at 10:03 PM, Antonello Provenzano
> <antonello at deveel.com> wrote:
>> I dug a bit the NIO package and the use done in neo4j: unfortunately
>> there's not counter part for it on .NET nor the possibility to
>> recreate [easily] classes to support it. In fact, you make a wide and
>> important use of MappedByteBuffer on files: although implement the
>> logics of a ByteBuffer class is not difficult (in .NET schema is the
>> association of two classes: MemoryStream and
>> BinaryWriter/BinaryReader), the behavior of MappedByteBuffer appears
>> to be more complex, since access to portion of data in file which
>> access must be coordinated with the memory, and this is not a simple
>> issue.
>>
>> As of Java implementation, this is done natively and it's a complex
>> operation to replicate the same functionality.
>>
>> Once this is solved, I see no further [major] obstacles to the port:
>> avoiding the integration with the System.Transactions mechanism, the
>> base of neo4j port would work in its core functionalities.
>>
>> Cheers.
>> Antonello
>>
>>
>> On Sun, May 3, 2009 at 4:51 PM, Johan Svensson <johan at neotechnology.com> wrote:
>>> See my comments inline:
>>>
>>> On Sun, May 3, 2009 at 4:05 PM, Antonello Provenzano
>>> <antonello at deveel.com> wrote:
>>>> Johan,
>>>>
>>>> I see. My knowledge of NIO is pretty limited indeed: since you're
>>>> telling me it's re-designable with traditional java.io
>>>> implementations, I see no particular issues on that side (also
>>>> because, as you described some of the core functionalities, many
>>>> features are already present in System.IO). In fact, what i did know
>>>> was that NIO was used to stream remotely, opening communication
>>>> channels between the local JVM and the remote endpoint (which could
>>>> have been another running application in the same JVM).
>>>>
>>>
>>> Yes, but we only use the native I/O part for files. Simply put, we
>>> want to avoid the extra copy of buffers that takes place from outside
>>> jvm memory to jvm buffers. NIO allows us to allocation direct buffers
>>> (that are allocated and managed outside the normal java heap). I also
>>> like the design of the nio package with channels and it is good that
>>> you can use the same interfaces to shuffle data over the network but
>>> Neo does not use those features (yet).
>>>
>>>> Coming back to Transactions: it's true that JTA is just a set of
>>>> specifications abstracted by the interfaces provided (which are easily
>>>> convertible into .NET with a custom JTA packaging), but it's also true
>>>> that .NET provides a "framework" for handling application transactions
>>>> in the same fashion as JTA.
>>>> In fact, when a developer instantiate a TransactionScope class, opens
>>>> a transactional context for th execution of the calls withing that
>>>> instance: to support this the JTA support present in neo4j should be
>>>> adapted to the specifications provided by System.Transactions, which
>>>> don't support XA nor 2PC (heavily used by neo4j).
>>>>
>>>
>>> It should be possible to port Neo4j's transaction handling to support
>>> .NET's way of handling transactions. Neo will only make use of 2PC if
>>> there are multiple resources participating in the transaction. If you
>>> only use Neo and nothing else all commits will be optimized one phase
>>> commits.
>>>
>>> How would .NET handle a transaction that would like to modify
>>> something in two different databases in the same transactional
>>> context? That is what we use the two phase commit protocol for but if
>>> you only use Neo4j there is only one resources enlisted in the
>>> transaction so then it is not needed.
>>>
>>>> It's true that it would be possible to create a custom transaction
>>>> namespace within the ported neo4j, but this would change the behavior
>>>> of the application: I don't know for sure, but JTA interacts with
>>>> underlying systems for the management of transactions, as
>>>> System.Transaction does; having a custom transaction mechanism would
>>>> bypass all this.
>>>>
>>>
>>> Yes, also it might be possible to just write some glue code between
>>> the .NET TransactionScope and the NeoService.beginTx() &
>>> org.neo4j.api.core.Transaction classes. If all you need is single
>>> resource, begin,commit and rollback it would be much easier to
>>> integrate there and just do a straightforward port of the transaction
>>> package.
>>>
>>> Regards,
>>> Johan
>>>
>>>> Cheers.
>>>> Antonello
> _______________________________________________
> Neo mailing list
> User at lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>


More information about the User mailing list