[Measures] added new measure (UsageCounts) #214

Merged
claudio.atzori merged 8 commits from eosc_dimitris into beta 2 years ago
Collaborator

This PR introduces a new oozie workflow for the creation of a new action set that includes the usage counts (views and downloads) as a new measure on the result level.

It will have as provenaceaction.classid the value measure:usage_counts and as id for the measure the keywords downloads and views referring respectively to the attribute with the same names in the usage stats table, while the key for the unit will be count.

The values are calculated as the sum of the counts, aggregated for each record id.

The remaining part of the measure element is set exactly as for the bip!Finder integration.

This PR introduces a new oozie workflow for the creation of a new action set that includes the usage counts (views and downloads) as a new measure on the result level. It will have as **provenaceaction.classid** the value **measure:usage_counts** and as id for the measure the keywords **downloads** and **views** referring respectively to the attribute with the same names in the usage stats table, while the **key** for the unit will be **count**. The values are calculated as the sum of the counts, aggregated for each record id. The remaining part of the measure element is set exactly as for the **bip!Finder** integration.
miriam.baglioni added the
enhancement
label 2 years ago
claudio.atzori was assigned by miriam.baglioni 2 years ago
miriam.baglioni added 3 commits 2 years ago
miriam.baglioni added 1 commit 2 years ago
claudio.atzori changed title from [Measures] added new measure (usagecounts) to [Measures] added new measure (UsageCounts) 2 years ago
claudio.atzori reviewed 2 years ago
@ -392,2 +392,4 @@
return null;
}
public static KeyValue newKeyValueInstance (String key, String value, DataInfo dataInfo){
Owner

Please compile before pushing. It seems the code formatting was not applied here.

Please compile before pushing. It seems the code formatting was not applied here.
claudio.atzori reviewed 2 years ago
@ -0,0 +33,4 @@
/**
* created the Atomic Action for each tipe of results
Owner

typo: tipe == type

typo: tipe == type
miriam.baglioni marked this conversation as resolved
claudio.atzori reviewed 2 years ago
@ -0,0 +65,4 @@
SparkConf conf = new SparkConf();
conf.set("hive.metastore.uris", parser.get("hive_metastore_uris"));
final String dbname = parser.get("statsdb");
Owner

Minor: I would name the parameter as usagestatsdb to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.

Minor: I would name the parameter as `usagestatsdb` to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.
miriam.baglioni marked this conversation as resolved
claudio.atzori reviewed 2 years ago
@ -0,0 +19,4 @@
},
{
"paramName": "sdb",
"paramLongName": "statsdb",
Owner

Minor: I would name the parameter as usagestatsdb to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.

Minor: I would name the parameter as `usagestatsdb` to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.
claudio.atzori reviewed 2 years ago
@ -0,0 +5,4 @@
<description>the path where to store the actionset</description>
</property>
<property>
<name>statsdb</name>
Owner

Minor: I would name the parameter as usagestatsdb to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.

Minor: I would name the parameter as `usagestatsdb` to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.
claudio.atzori reviewed 2 years ago
@ -0,0 +6,4 @@
</property>
<property>
<name>statsdb</name>
<description>the path where to store the actionset</description>
Owner

the description is wrong. Looks like a copy&paste

the description is wrong. Looks like a copy&paste
claudio.atzori requested changes 2 years ago
claudio.atzori left a comment
Owner

Overall looks ok, just minor changes. Please check the inline comments and remember to compile before issuing the commit&push so that the common formatting is applied.

Overall looks ok, just minor changes. Please check the inline comments and remember to compile before issuing the commit&push so that the common formatting is applied.
claudio.atzori added 1 commit 2 years ago
miriam.baglioni added 2 commits 2 years ago
miriam.baglioni added 1 commit 2 years ago
claudio.atzori merged commit a289c9eae2 into beta 2 years ago
claudio.atzori deleted branch eosc_dimitris 2 years ago

Reviewers

claudio.atzori requested changes 2 years ago
The pull request has been merged as a289c9eae2.
You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.
git checkout -b eosc_dimitris beta
git pull origin eosc_dimitris

Step 2:

Merge the changes and update on Gitea.
git checkout beta
git merge --no-ff eosc_dimitris
git push origin beta
Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#214
Loading…
There is no content yet.