Fixes #1327 replacing py2neo with neo4j

[dewardvide]

Problem Statement

We need to migrate from py2neo to official neo4j or neomodels driver because py2neo has been deprecated since 2021 as pointed out in issue #1327.

Issue analysis & solution

• The issue affects pymisp/tools/neo4j.py which migrates events into a Neo4j graph, establishing relationships between the events, attributes and attribute values.
• Instead of a 1:1 port I made the following architectural changes:
  • I decided to use neo4j because of the simplicity of the script. Using it allows better performance and support because it is the official Neo4j lib
  • By choosing neo4j the Cypher queries were defined directly below is a summarized mapping of the old functions and the new ones:

Aspect	py2neo (ORM)	neo4j (Cypher)
Connection	authenticate() + Graph()	GraphDatabase.driver() with Bolt protocol
Node creation	Python Node() objects	Cypher CREATE (e:Event {$event.uuid})
Relationships	Python Relationship() objects	Cypher CREATE (a)-[:type]->(b)
Transactions	tx.begin() / tx.commit()	session.execute_write(callback)
Deduplication	tx.merge(subgraph)	Cypher MERGE keyword

• Further implementation details are as follows:

  1. Fixed Protocol & Port: Changed from HTTP (7474) to Bolt (7687) — modern protocol for Neo4j
  2. Added Label Sanitization: MISP attribute types contain special characters (ip-src, domain|ip). Neo4j label names only accept alphanumeric + underscores. Solution: re.sub(r'[^A-Za-z0-9_]', '_', a.type)
  3. Ensured Resource Safety: Used with context managers to guarantee session/connection cleanup
  4. Parameterized Queries: All parameters use $parameter syntax, preventing Cypher injection
  5. Added Debugging: Warning messages when attribute types are sanitized
  6. Added dependency declaration: Added neo4j = [ "neo4j (>=5.0)" ] to pyproject.toml under optional-dependencies

Testing

• I created test/test_neo4j.py to test the new version of neo4j.py.
  • The test included the creation of a mock attribute using MISPEvent.add_attribute()
  • Connection to a local neo4j instance
  • Importing the events data using import_event() which is the main function
  • Verifying the event data and their relationships exist
  • Verifying the label sanitation
  • Cleaning up

*test outcome*

• All tests were successful!

Disclaimer : AI assisted me in analyzing the issue suggesting different approaches and streamlining testing.

[Rafiot]

Alright, I did a very quick review of the code and I have a few notes.

Just making sure: you're using neo4j regularly and know what you're doing, right? I have not tested your code yet, this review is simply based on a quick read through.

[Rafiot]

You also need to update the poetry.lock file.

[dewardvide]

Done, also changed the lowest version to 5.26 to support dynamic labels as in https://neo4j.com/blog/developer/cypher-dynamism/

[Rafiot]

Quick question on that: why not using the version 6.1 of the package?

[dewardvide]

5.6 is the oldest version that supports the changes I made but I've changed it to 6.1 since the neo4j driver docs recommend using the latest driver as it supports older server versions.

[dewardvide]

Thank you @Rafiot for the valuable feedback. I will look into the comments and respond/make the necessary changes within the course of the week

[Rafiot]

That looks great, can you just add the typing markings so mypy is happy? It's not a very big deal, but nice to have.

[dewardvide]

I've added the types, should be good now

[Rafiot]

Can you make sure to update the lock file? I'd like to get the action to pass.

[dewardvide]

I just realized that I messed something up with the poetry.lock file, I ran poetry update instead of poetry add, let me fix this, I will let you know once it's done

[dewardvide]

Just double checked everything. All should be good now.

[Rafiot]

Excellent, thank you very much!

[dewardvide]

Thank you too! I appreciate the patience and guidance!