tolerant parsing of OAF-store-graph mdstores #180

Merged
claudio.atzori merged 2 commits from graph_interpretation_mdstores into beta 2 years ago
Owner

This PR introduces a change in the serialisation of the records stored in the OAF-store-graph mdstores. They were stored in Kryo (binary) format, therefore any slight change in the schema version would made them unreadable, forcing us to recreate them quite frequently. Instead, we're going to store them in as a json serialization, so that reading them again in the graph construction phase with a parser (com.fasterxml.jackson.databind.ObjectMapper) we have better changes to support backward compatible changes in the evolution of the schema.

val mapper = new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
This PR introduces a change in the serialisation of the records stored in the `OAF-store-graph` mdstores. They were stored in Kryo (binary) format, therefore any slight change in the schema version would made them unreadable, forcing us to recreate them quite frequently. Instead, we're going to store them in as a json serialization, so that reading them again in the graph construction phase with a parser (`com.fasterxml.jackson.databind.ObjectMapper`) we have better changes to support backward compatible changes in the evolution of the schema. ``` val mapper = new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) ```
claudio.atzori added the
enhancement
label 2 years ago
claudio.atzori added 2 commits 2 years ago
claudio.atzori merged commit cb30770a0b into beta 2 years ago
The pull request has been merged as cb30770a0b.
You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.
git checkout -b graph_interpretation_mdstores beta
git pull origin graph_interpretation_mdstores

Step 2:

Merge the changes and update on Gitea.
git checkout beta
git merge --no-ff graph_interpretation_mdstores
git push origin beta
Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#180
Loading…
There is no content yet.