Commit Graph

61 Commits

Author SHA1 Message Date
Lampros Smyrnaios 4dc34429f8 - Increase the waiting-time before checking the docker containers' status, in order to catch configuration-crashes.
- Code polishing.
2023-04-10 22:28:53 +03:00
Lampros Smyrnaios c39fef2654 Upgrade payload-table to payload-view which consists of three separate payload tables: "payload_legacy", "payload_aggregated" and "payload_bulk_import". 2023-04-10 15:55:50 +03:00
Lampros Smyrnaios 37363100fd Prioritize most recent publications. 2023-04-10 15:00:23 +03:00
Lampros Smyrnaios 4280f89296 - Set the default value of the "isTestEnvironment" property to "true", in order to avoid undesired outcomes in the production db.
- Code polishing.
2023-03-21 17:04:28 +02:00
Lampros Smyrnaios 003c0bf179 - Add support for excluding specific datasources from being crawled. These datasources may be aggregated through bulk-imports, by other pieces of software. Such a datasource is "arXiv.org".
- Fix an issue, where the "datasource-type" was retrieved instead of the "datasource-name".
- Polish the "findAssignmentsQuery".
2023-03-21 07:19:35 +02:00
Lampros Smyrnaios ff13af7abb Use a StatsService interface. 2023-03-13 12:39:39 +02:00
Lampros Smyrnaios 38643c76a3 - Code polishing.
- Update Gradle.
2023-03-07 16:55:41 +02:00
Lampros Smyrnaios c8485d472e Code polishing. 2023-02-24 13:53:09 +02:00
Lampros Smyrnaios 8893662a81 Refactor the UrlsController: a) offload the business-logic to the dedicated "UrlsService" and b) move the "checkParquetFilesSuccess()"-method to "ParquetFileUtils". 2023-02-21 15:36:35 +02:00
Lampros Smyrnaios a1c16ffc19 - Exclude empty and null urls in the assignments.
- Update the "getFullTextsImproved"-call to "getFullTexts", now that the "improved" version is stable.
- Update Gradle.
- Code polishing.
2023-02-16 14:24:47 +02:00
Lampros Smyrnaios 2253f05bf5 Refactor the "StatsController"-code, by offloading it to a dedicated "StatsService". 2023-02-09 19:25:48 +02:00