This website requires JavaScript.
Explore
Help
Sign In
lsmyrnaios
/
UrlsWorker
Watch
1
Star
0
Fork
You've already forked UrlsWorker
1
Code
Issues
Pull Requests
Releases
14
Wiki
Activity
29a54f0b30
UrlsWorker
/
src
/
main
/
java
/
eu
/
openaire
/
urls_worker
History
Lampros Smyrnaios
29a54f0b30
Remove the "shutDownOrCancelCode" from security checks, since we have an IP whitelisting mechanism in place.
2023-05-03 15:15:46 +03:00
..
components
Use restTemplates with different read timeouts depending on the operation. For the assignments-request we need a shorter read timeout, than the one we need for the worker-report. This guarantees that the connection does not hungs for so long, when the Controller crashes before sending the assignments.
2023-04-29 17:24:16 +03:00
controllers
Remove the "shutDownOrCancelCode" from security checks, since we have an IP whitelisting mechanism in place.
2023-05-03 15:15:46 +03:00
models
Fix a "@JsonProperty" annotation inside "Payload.java".
2022-04-01 23:43:43 +03:00
payloads
- Use the "Timestamp" data-type instead of the "Date", in order to include more information.
2021-11-27 02:37:33 +02:00
security
- Add deletion, of the cookies in the newly-supported CookieManager, after each batch.
2022-06-27 17:58:02 +03:00
services
- Improve performance of full-texts transferring to the Controller, by preloading some bytes for faster response to the Controller's read requests.
2023-03-08 13:12:17 +02:00
util
Reduce memory usage.
2023-04-28 17:59:36 +03:00
UrlsWorkerApplication.java
- Handle the case, where an instance of a urlReport record (having the same id and sourceUrl), may have failed to give a docUrl, due to en error, even if another instance gives the docUrl and the docFile. The absence of that handling could lead to a record-instance, being assigned a "fileLocation" which was actually an error-message (comment), and as a result the real "fileLocation" would have never been reached to be assigned, so the payload would be lost.
2023-02-21 15:22:49 +02:00