MaDgIK
/
explore-services
13
0
Fork 0
Code Issues Pull Requests Projects Releases 22 Wiki Activity
1,104 Commits 17 Branches 22 Tags 478 MiB
Commit Graph

64 Commits

Author SHA1 Message Date
Konstantina Galouni 6af07478b4 [develop | DONE | CHANGED]: cache.js: Increased "cacheMaxSize" from 300 to 1000 requests. 2024-02-12 13:29:23 +02:00
argirok 3b058ee74d [ develop | DONE | CHANGED] Cache: update isAllowed Method, correct https check, decode urls before check 2024-02-12 11:13:38 +02:00
Konstantina Galouni fbdc3710dc [develop | DONE | FIXED]: cache.js: Updated checks in method "isAllowedToBeCached()" to check for encoded urls. 2024-02-08 19:43:31 +02:00
Konstantina Galouni 9952b55f2c [develop | DONE | CHANGED]: cache.js: Added "lab.idiap.ch/enermaps" in "isAllowedToBeCached()" method. 2024-02-08 17:55:22 +02:00
argirok b0db89f826 [ develop | DONE | CHANGED] Cache: change to LRU cache, remove short/long cache related code and properties, add preload requests in properties to load on init cache (on start, on clear and on daily reset) 2024-02-08 15:58:43 +02:00
argirok 7761eeeedc [ develop | DONE | CHANGED] Utils services: update funders response, remove open access filter from results query in /explore/search 2024-02-07 11:05:55 +02:00
argirok 46ad6aef36 [ develop | DONE | ADDED] Utils services: add /explore/funders method that groups requests for funder page 2024-02-06 11:31:47 +02:00
argirok 629d3efb08 [ develop | DONE | ADDED] Cache services: add initialization method to initiate grouped requests from utils service. Init after clear and when cache starts 2024-02-05 12:08:34 +02:00
argirok a9d06e61fa [ develop | DONE | ADDED] Utils services: group common requests in a single request 2024-02-05 12:07:34 +02:00
Konstantina Galouni 109be1c6f6 [cache | develop | DONE | ADDED]: cache.js: Added in "longCachingRequests", the endpoint "/utils-service/explore/home" to be used as a grouped response for the numbers in explore home page. 2024-02-02 12:44:54 +02:00
Konstantina Galouni 3678f7c5ea [cache | develop | DONE | CHANGED]: cache.js: Increased time of short cache from 2 minutes to 1 hour. 2024-02-02 12:12:57 +02:00
Konstantinos Triantafyllou 6d4f7d5645 Revert port of Upload to 8000 2023-08-31 13:53:24 +03:00
Konstantinos Triantafyllou 95f749e9f0 Update versions in sitemaps. Need to be checked. 2023-08-04 15:45:18 +03:00
Konstantinos Triantafyllou 3a463f9bab Update dependencies and node version in cache and utils-service 2023-08-04 13:54:46 +03:00
Konstantinos Triantafyllou 36f4bd1c60 Utils Service: Change user info property. 2023-03-23 10:41:37 +02:00
argirok 82afd05444 update dependencies for cache and utils services 2022-10-03 13:00:37 +03:00
argirok e4586e58f2 Services: add properties files and scripts to create separate configuration per environment 2022-10-03 12:22:13 +03:00
Konstantina Galouni 0b53cea58a [Library | new-theme]: Allow also ?pid url parameter in datasource landing | Fixes in newSearchPage for service filters. 
1. dataProvider.component.ts: [Bug fix] Allow also ?pid url parameter, query accordingly and set canonicalUrl to use it in seoService and scema2jsonld.
2. dataProvider.component.html: Set canonicalUrl into URL of <schema2jsonld> | Use <landing-header> instead of <showTitle>.
3. dataProvider.service.ts: Set url for querying a datasource by pid (if ?pid in landing url param) and parse also the whole record, the objIdentifier and the relcanId.
4. landing-header.component.ts: Added @Input() isSticky: boolean = false; to set less margins when sticky.
5. resultLanding.component.html: Use <landing-header> instead of <showTitle>.
6. resultLanding.component.ts: Use renamed Identifier.getResultPIDFromIdentifiers --> Identifier.getPIDFromIdentifiers.
7. metrics.service.ts: Removed console.log.
8. searchDataproviders.service.ts: Added parsing for relcanId.
9. dataProviderInfo.ts: Added relcanId, objIdentifier, record.
10. result-preview.component.ts: Use renamed Identifier.getResultPIDFromIdentifiers --> Identifier.getPIDFromIdentifiers.
11. string-utils.class.ts: Renamed Identifier.getResultPIDFromIdentifiers --> Identifier.getPIDFromIdentifiers.
12. [SITEMAPS] extractUrlsFromSearch.ts: Use renamed Identifier.getResultPIDFromIdentifiers --> Identifier.getPIDFromIdentifiers.
13. newSearchPage.component.ts: [Bug fix]
   a. entityType for datasources is "dataprovider".
   b. Added service filter options only when entityType == "service".
 2022-05-30 10:39:10 +03:00
argirok 01febea53b utils - service: update prod properties for user-info 2022-01-28 15:07:39 +02:00
Konstantinos Triantafyllou 4b4aa45645 Create a userInfo method. Add label as path param 2021-11-26 11:50:01 +02:00
Konstantinos Triantafyllou 488aaf4a2a Merge branch 'master' of code-repo.d4science.org:MaDgIK/explore-services 2021-11-25 18:54:29 +02:00
Konstantinos Triantafyllou f8ede1d232 Update authorization base on new roles. Create new methods for upload and delete 2021-11-25 18:53:59 +02:00
Konstantina Galouni 2209804919 [Explore]: services/sitemaps: In sitemaps creation, added case for querying with subjects (instead of refine). 
1. extractUrlsFromSearch.ts:
   a. Added input read for result type and for subject (can be given as empty).
   b. Added creation of sitemaps for results related to a specific subject (many methods are parametrized accordingly).
2. run.sh: Added a second parameter for subject.
3. .gitignore: Updated gitignore file to ignore sitemaps created locally in commit process.
 2021-11-25 13:51:17 +02:00
Konstantina Galouni 91a0b86675 [Library]: searchDatacite.service.ts & searchCrossref.service.ts: [Bug fix] Extra checks added in author parsing | In Crossref parse author.name if author.family and/or author.given is missing. 2021-11-11 12:13:25 +02:00
Konstantina Galouni a4410578b3 [Trunk | Explore Services]: extractUrlsFromSearch.ts: [Bug fix] Add check for undefined value in log. 2021-10-08 10:33:03 +03:00
Konstantina Galouni 6350a9f550 [Explore Service | Services]: 1. extractUrlsFromSearch.ts: a. Do not query for results but for parametrized "resultType". 
b. When final urls > 50000, create new files every 50000.
2. run.sh: Added $1 for argument "resultType".
 2021-08-10 12:57:25 +03:00
Konstantina Galouni f5cb4c2acd [Explore Service | Services]: 
1. extractUrlsFromSearch.ts: changed results parsing (used landing parsing instead of searh) to call method "checkIfAllowed()" (no index filtering) | Added more numbers for statistics.
2. package.json: Upgrade node version.
 2021-08-06 14:19:58 +03:00
k.triantafyllou 20badf397f [Explore-Service]: Update to angular 11. Delete branches 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61402 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-07-14 15:41:12 +00:00
konstantina.galouni 6dbdecafb4 [Trunk | Explore Services]: extractUrlsFromSearch.ts: Fixes in script | Services on production, not on beta. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61367 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-07-12 14:09:55 +00:00
konstantina.galouni 5b399090d5 [Trunk | Explore Services]: extractUrlsFromSearch.ts: Added .catch(error => ...) in rejected promises. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61229 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-16 08:03:07 +00:00
konstantina.galouni 13b9972ee4 [Trunk | Explore Services]: Renamed "sitempas" folder to "sitemaps". 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61228 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-15 14:44:27 +00:00
konstantina.galouni 1990ce1eca [Trunk | Explore Services] 
1. extractUrlsFromSearch.ts: 
	a. [Bug fix] In request for getting communities, resolve and reject appropriately.
	b. Removed require('express'), require('url'), require('cors').
	c. Updated file names (date.getMonth() returns 0-11) | Add .txt in error file name.
2. package.json: Removed dependencies cors and express.
3. tsconfig.json: Added in compilerOptions: "lib": ["es2017", "dom"].
4. svn:ignore: Ignore node_module, package-lock.json, *.xml, *.txt.


git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61227 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-15 14:41:59 +00:00
konstantina.galouni 4d5d3ea0f0 [Trunk | Explore]: Added sitemaps folder to create sitemap.xml file with all urls from research outcomes search page with one filter value selected foreach filter value. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61220 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-15 13:19:37 +00:00
konstantina.galouni 0bf05409cc [Trunk | Cache]: cache.js: In longCachingRequests add "/api/communities/" (zenodo communities request) and "/openaire/contexts/" (communities with status info). 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59649 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-10-20 14:53:26 +00:00
k.triantafyllou d71054cca7 [Utils Service | Trunk]: Add upload method for stakeholders 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59489 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-28 21:41:23 +00:00
k.triantafyllou 2af5bb33e4 [Utils Service | Trunk]: Add upload method for stakeholders 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59488 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-28 21:32:09 +00:00
konstantina.galouni a7b3be3faa [Trunk | Explore Services]: cache.js: In longCachingRequests include paths "/full" (and "/communityFull") and "/openaire/info" (for last index date). 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59383 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-15 11:54:35 +00:00
k.triantafyllou aac1a2d5e9 [Cache]: Add usage counts endpoints to long cache 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59363 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-14 15:07:05 +00:00
k.triantafyllou 977c465079 [Cache | Trunk]: Change size to memsize while it is setting the cache current entries on metrics. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59194 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-07-29 14:09:45 +00:00
argiro.kokogiannaki cac44d718e Cache service: 
-change console.debug to log
-mv resetAtMidnight inside app.listen


git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59002 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-06-26 14:44:28 +00:00
argiro.kokogiannaki a9652670b3 [Cahce service] 
Add:
const expireShort = 2 * 60 * 1000; //2mins
const expireLong = 24 * 60 * 60 * 1000; //24 hours
const cacheMaxSize =500;
const longCachingRequests = ["/communityFull/", "/pagehelpcontent","/provision/mvc/vocabularies/","/pages?page_route="];

-use expireLongTime for longCachingRequests
-add /clear request
-add timer to clean the cache on midnight
-when maxcachesize is reached clean the cache



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59001 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-06-26 13:33:35 +00:00
argiro.kokogiannaki cfed36d65b Cache: 
Change URL method to work with older nodejs versions



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58699 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-05-14 16:48:40 +00:00
k.triantafyllou 86c65dbd81 [Cache | Trunk]: Fix some issues on metrics. Remove encode url 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58495 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-04-15 13:53:37 +00:00
k.triantafyllou eabd920aa2 [Cache | Trunk]: Add new metrics, encode url 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58478 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-04-10 14:29:46 +00:00
k.triantafyllou 1fff0bd89e [Cache | Trunk]: Resolved some warnings 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58226 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-03-11 12:53:14 +00:00
k.triantafyllou b2b15b5d6c [Cache | Trunk]: Add prometheus metrics 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58225 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-03-11 12:51:10 +00:00
argiro.kokogiannaki 05a7e9560e [Trunk| Services: utils] 
add file with production properties
UPload:
	add parameter type (default is csv)
	allow upload of json
Check for post: Split url to ? in case it contains query parameters




git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@57871 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-12-13 09:35:51 +00:00
argiro.kokogiannaki fe92edf0da [Cache service | Trunk] 
1.add url in logs
2. add response in cache only for requests that return code 200



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@57529 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-11-06 11:09:06 +00:00
k.triantafyllou 2d888181a0 [UtilsService]: Add on allowed file types application/vnd.ms-excel for windows. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56956 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-09-03 13:12:09 +00:00
argiro.kokogiannaki 4078b4db45 UploadService: 
add timer before deleting the file (sometimes it was deleting the file, before read it)



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56752 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-07-24 12:49:32 +00:00