lsmyrnaios
/
UrlsWorker
1
0
Fork 1
Code Issues Pull Requests Releases 12 Wiki Activity
164 Commits 1 Branch 12 Tags 1.3 MiB
Commit Graph

164 Commits

Author SHA1 Message Date
Lampros Smyrnaios 62ce7ee4a5 - Process the Error of PDF-aggregation. Distinguish between "couldRetry" and "noRetry" cases. 
- Add a "test"-switch in order to easily switch between test and normal mode.
- Fix an NPE, when requesting for the "AssignmentRequest".
- Upgrade the "installPublicationsRetriever.sh" to "installAndRun.sh", which takes care of everything.
- Define the newest SpringBoot-version in "build.gradle".
- Code cleanup.
 2021-08-05 15:09:28 +03:00
Lampros Smyrnaios 6cc2673fca - Add the ability to upload the files on an S3-ObjectStore. 
- Change the server's port and the port of the controller-api.
- Update dependencies.
 2021-07-29 09:01:53 +03:00
Lampros Smyrnaios 6307cda23a - Refactor the assignments-handling. In order to match with the database schema, now the AssignmentRequest returns a list of Assignments instead of a single assignment having a list of Tasks. 
- Cleanup the members of the "Payload" model.
 2021-07-05 15:00:29 +03:00
Lampros Smyrnaios f6e53ca289 Integrate the "PublicationsRetriever" program as a plugin, which downloads the full-texts of the publications. Afterwards, the retrieved data info is transferred to the Controller. 
The "PublicationsRetriever" can be installed locally as a library, using the "installPublicationsRetriever.sh" script.
 2021-06-22 05:58:07 +03:00
Lampros Smyrnaios 83d1bd2def Update the "WorkerReport" response and the "UrlReport" and "Payload" models. 2021-06-19 07:12:24 +03:00
Lampros Smyrnaios 3550ed71d9 Execute the "AssignmentHandler.handleAssignment()", only from the scheduler, as it starts automatically when the program starts. 2021-06-11 13:44:33 +03:00
Lampros Smyrnaios 5f3409e072 - Update Spring and add the "gradle-wrapper.properties" file which defines the gradle version. 
- Improve an info-logging message and cleanup the code.
 2021-06-10 14:29:20 +03:00
Lampros Smyrnaios 53ccea869a Add an "assignmentId" field in the "Assignment"-class. 2021-06-09 05:45:07 +03:00
Lampros Smyrnaios 82e12655e7 - Add an "AssignmentHandler", which retrieves the assignment from the controller and categorises the tasks using their datasource. In the future, it will execute the tasks of the assignment, using different plugins. It runs upon the Application start and also every 30 mins (if no other job is in execution). 
- Add the "isWorkerAvailableForWork-endpoint.
 2021-05-20 03:28:48 +03:00
Lampros Smyrnaios a4c97dffbf - Add the "Datasource" class in the "Task" class and include it in the Assignment that the worker retrieves. 
- Update dependencies.
 2021-05-20 02:58:08 +03:00
Lampros Smyrnaios 137744a8ce - Add classes: "Assignment", "AssignmentRequest", "UrlReport", "WorkerReport" and "WorkerResponse". 
- Add interface "WorkerConstants".
 2021-04-24 21:10:35 +03:00
Lampros Smyrnaios b2dfd524e1 Update "Task" and "Error" classes. 2021-04-24 21:08:02 +03:00
Lampros Smyrnaios 6a87d3e478 Add "Task" and "Error" classes. 2021-04-15 03:37:54 +03:00
Lampros Smyrnaios 08eabe6f08 Initial commit of UrlsWorker. 2021-03-16 18:38:53 +02:00