Lampros Smyrnaios
|
cd1fb0af88
|
- Process the WorkerReports in background Jobs and post the reportResults to the Workers.
- Save the workerReports to json files, until they are processed successfully.
- Show some custom metrics in prometheus.
|
2023-05-24 13:52:28 +03:00 |
Lampros Smyrnaios
|
9412391903
|
- In test-environment mode, check for already existing file-hashes only in the "payload_aggregated" table, instead of the whole "payload" view. This way the investigation for false-positive docUrls is easier, as we avoid checking against the millions of "legacy" payloads.
- Improve performance in production, by not creating the string objects for "trace"-logs.
|
2023-05-15 12:44:16 +03:00 |
Lampros Smyrnaios
|
8381df70c6
|
- Improve performance of uploading parquet-files to HDFS.
- Add some logs.
- Code polishing.
|
2023-05-11 19:40:48 +03:00 |
Lampros Smyrnaios
|
b6e8cd1889
|
New feature: BulkImport full-text files from compatible datasources.
|
2023-05-11 03:07:55 +03:00 |