Update Broker events [PROD OCEAN]
Data Provision
30
Set the path containing the GRAPH to scan
graphInputPath
Set the datasource Ids Whitelist
datasourceIdWhitelist
openaire____::9ecafa3655143cbc4bc75853035cd432,opendoar____::dc6e224a8d74ce03bf301152d6e33e97,openaire____::09da65eaaa6deac2f785df1e0ae95a06,openaire____::3db634fc5446f389d0b826ea400a5da6,openaire____::5a38cb462ac487bf26bdb86009fe3e74,openaire____::3c29379cc184f66861e858bc7aa9615b,openaire____::4657147e48a1f32637bfe3743bce76c6,openaire____::c3267ea1c3f378c456209b6df241624e,opendoar____::358aee4cc897452c00244351e4d91f69,re3data_____::7b0ad08687b2c960d5aeef06f811d5e6,opendoar____::798ed7d4ee7138d49b8828958048130a,opendoar____::6f4922f45568161a8cdf4ad2299f6d23,opendoar____::4aa0e93b918848be0b7728b4b1568d8a,openaire____::02b55e4f52388520bfe11f959f836e68
Set the datasource type Whitelist
datasourceTypeWhitelist
pubsrepository::unknown,pubsrepository::institutional,pubsrepository::thematic,datarepository::unknown,orprepository,softwarerepository
Set the datasource Id Blacklist
datasourceIdBlacklist
-
Set the TOPIC whitelist (* = all topics)
topicWhitelist
ENRICH/MISSING/SUBJECT/DDC,ENRICH/MISSING/SUBJECT/JEL,ENRICH/MISSING/SUBJECT/MESHEUROPMC,ENRICH/MISSING/PUBLICATION_DATE,ENRICH/MISSING/PID,ENRICH/MISSING/PROJECT,ENRICH/MISSING/SUBJECT/ACM,ENRICH/MISSING/SUBJECT/ARXIV,ENRICH/MISSING/OPENACCESS_VERSION,ENRICH/MISSING/AUTHOR/ORCID,ENRICH/MISSING/ABSTRACT,ENRICH/MORE/SUBJECT/ACM,ENRICH/MORE/SUBJECT/ARXIV,ENRICH/MORE/SUBJECT/DDC,ENRICH/MORE/SUBJECT/JEL,ENRICH/MORE/OPENACCESS_VERSION,ENRICH/MORE/SUBJECT/MESHEUROPMC,ENRICH/MORE/PID
Set the output path to store the Event records
outputDir
/var/lib/dnet/broker_PROD/events
wait configurations
update the BROKER events
executeOozieJob
IIS
{
'graphInputPath' : 'graphInputPath',
'datasourceIdWhitelist' : 'datasourceIdWhitelist',
'datasourceTypeWhitelist' : 'datasourceTypeWhitelist',
'datasourceIdBlacklist' : 'datasourceIdBlacklist',
'topicWhitelist' : 'topicWhitelist',
'outputDir' : 'outputDir'
}
{
'oozie.wf.application.path' : '/lib/dnet/PROD/oa/broker/generate_events/oozie_app',
'esEventIndexName' : '',
'esNotificationsIndexName' : '',
'esIndexHost' : '',
'maxIndexedEventsForDsAndTopic' : '100',
'esBatchWriteRetryCount' : '8',
'esBatchWriteRetryWait' : '60s',
'esBatchSizeEntries' : '200',
'esNodesWanOnly' : 'true',
'brokerApiBaseUrl' : '',
'brokerDbUrl' : '',
'brokerDbUser' : '',
'brokerDbPassword' : '',
'sparkDriverMemory' : '3G',
'sparkExecutorMemory' : '7G',
'sparkExecutorCores' : '6',
'workingDir' : '/tmp/prod_provision/working_dir/broker_events'
}
build-report
wf_20210709_073839_206
2021-07-09T11:01:01+00:00
FAILURE