[Measures] added new measure (UsageCounts) #214

Merged
claudio.atzori merged 8 commits from eosc_dimitris into beta 2022-04-21 12:19:19 +02:00

This PR introduces a new oozie workflow for the creation of a new action set that includes the usage counts (views and downloads) as a new measure on the result level.

It will have as provenaceaction.classid the value measure:usage_counts and as id for the measure the keywords downloads and views referring respectively to the attribute with the same names in the usage stats table, while the key for the unit will be count.

The values are calculated as the sum of the counts, aggregated for each record id.

The remaining part of the measure element is set exactly as for the bip!Finder integration.

This PR introduces a new oozie workflow for the creation of a new action set that includes the usage counts (views and downloads) as a new measure on the result level. It will have as **provenaceaction.classid** the value **measure:usage_counts** and as id for the measure the keywords **downloads** and **views** referring respectively to the attribute with the same names in the usage stats table, while the **key** for the unit will be **count**. The values are calculated as the sum of the counts, aggregated for each record id. The remaining part of the measure element is set exactly as for the **bip!Finder** integration.
miriam.baglioni added the
enhancement
label 2022-04-20 16:21:06 +02:00
claudio.atzori was assigned by miriam.baglioni 2022-04-20 16:21:06 +02:00
miriam.baglioni added 3 commits 2022-04-20 16:21:07 +02:00
miriam.baglioni added 1 commit 2022-04-21 11:27:39 +02:00
claudio.atzori changed title from [Measures] added new measure (usagecounts) to [Measures] added new measure (UsageCounts) 2022-04-21 11:49:02 +02:00
claudio.atzori reviewed 2022-04-21 11:50:39 +02:00
@ -392,2 +392,4 @@
return null;
}
public static KeyValue newKeyValueInstance (String key, String value, DataInfo dataInfo){

Please compile before pushing. It seems the code formatting was not applied here.

Please compile before pushing. It seems the code formatting was not applied here.
claudio.atzori reviewed 2022-04-21 11:52:51 +02:00
@ -0,0 +33,4 @@
/**
* created the Atomic Action for each tipe of results

typo: tipe == type

typo: tipe == type
miriam.baglioni marked this conversation as resolved
claudio.atzori reviewed 2022-04-21 11:56:25 +02:00
@ -0,0 +65,4 @@
SparkConf conf = new SparkConf();
conf.set("hive.metastore.uris", parser.get("hive_metastore_uris"));
final String dbname = parser.get("statsdb");

Minor: I would name the parameter as usagestatsdb to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.

Minor: I would name the parameter as `usagestatsdb` to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.
miriam.baglioni marked this conversation as resolved
claudio.atzori reviewed 2022-04-21 11:57:08 +02:00
@ -0,0 +19,4 @@
},
{
"paramName": "sdb",
"paramLongName": "statsdb",

Minor: I would name the parameter as usagestatsdb to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.

Minor: I would name the parameter as `usagestatsdb` to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.
claudio.atzori reviewed 2022-04-21 11:57:25 +02:00
@ -0,0 +5,4 @@
<description>the path where to store the actionset</description>
</property>
<property>
<name>statsdb</name>

Minor: I would name the parameter as usagestatsdb to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.

Minor: I would name the parameter as `usagestatsdb` to avoid confusion with the other stats database. In this context there is no ambiguity, but from the workflow caller there might be.
claudio.atzori reviewed 2022-04-21 11:57:56 +02:00
@ -0,0 +6,4 @@
</property>
<property>
<name>statsdb</name>
<description>the path where to store the actionset</description>

the description is wrong. Looks like a copy&paste

the description is wrong. Looks like a copy&paste
claudio.atzori requested changes 2022-04-21 11:59:51 +02:00
claudio.atzori left a comment
Owner

Overall looks ok, just minor changes. Please check the inline comments and remember to compile before issuing the commit&push so that the common formatting is applied.

Overall looks ok, just minor changes. Please check the inline comments and remember to compile before issuing the commit&push so that the common formatting is applied.
claudio.atzori added 1 commit 2022-04-21 12:00:08 +02:00
miriam.baglioni added 2 commits 2022-04-21 12:11:01 +02:00
miriam.baglioni added 1 commit 2022-04-21 12:14:13 +02:00
claudio.atzori merged commit a289c9eae2 into beta 2022-04-21 12:19:19 +02:00
claudio.atzori deleted branch eosc_dimitris 2022-04-21 12:19:22 +02:00
Sign in to join this conversation.
No description provided.