MaDgIK
/
explore-services
13
0
Fork 0
Code Issues Pull Requests Projects Releases 17 Wiki Activity
615 Commits 11 Branches 17 Tags 478 MiB
Commit Graph

42 Commits

Author SHA1 Message Date
Konstantina Galouni 2209804919 [Explore]: services/sitemaps: In sitemaps creation, added case for querying with subjects (instead of refine). 
1. extractUrlsFromSearch.ts:
   a. Added input read for result type and for subject (can be given as empty).
   b. Added creation of sitemaps for results related to a specific subject (many methods are parametrized accordingly).
2. run.sh: Added a second parameter for subject.
3. .gitignore: Updated gitignore file to ignore sitemaps created locally in commit process.
 2021-11-25 13:51:17 +02:00
Konstantina Galouni 91a0b86675 [Library]: searchDatacite.service.ts & searchCrossref.service.ts: [Bug fix] Extra checks added in author parsing | In Crossref parse author.name if author.family and/or author.given is missing. 2021-11-11 12:13:25 +02:00
Konstantina Galouni a4410578b3 [Trunk | Explore Services]: extractUrlsFromSearch.ts: [Bug fix] Add check for undefined value in log. 2021-10-08 10:33:03 +03:00
Konstantina Galouni 6350a9f550 [Explore Service | Services]: 1. extractUrlsFromSearch.ts: a. Do not query for results but for parametrized "resultType". 
b. When final urls > 50000, create new files every 50000.
2. run.sh: Added $1 for argument "resultType".
 2021-08-10 12:57:25 +03:00
Konstantina Galouni f5cb4c2acd [Explore Service | Services]: 
1. extractUrlsFromSearch.ts: changed results parsing (used landing parsing instead of searh) to call method "checkIfAllowed()" (no index filtering) | Added more numbers for statistics.
2. package.json: Upgrade node version.
 2021-08-06 14:19:58 +03:00
k.triantafyllou 20badf397f [Explore-Service]: Update to angular 11. Delete branches 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61402 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-07-14 15:41:12 +00:00
konstantina.galouni 6dbdecafb4 [Trunk | Explore Services]: extractUrlsFromSearch.ts: Fixes in script | Services on production, not on beta. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61367 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-07-12 14:09:55 +00:00
konstantina.galouni 5b399090d5 [Trunk | Explore Services]: extractUrlsFromSearch.ts: Added .catch(error => ...) in rejected promises. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61229 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-16 08:03:07 +00:00
konstantina.galouni 13b9972ee4 [Trunk | Explore Services]: Renamed "sitempas" folder to "sitemaps". 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61228 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-15 14:44:27 +00:00
konstantina.galouni 1990ce1eca [Trunk | Explore Services] 
1. extractUrlsFromSearch.ts: 
	a. [Bug fix] In request for getting communities, resolve and reject appropriately.
	b. Removed require('express'), require('url'), require('cors').
	c. Updated file names (date.getMonth() returns 0-11) | Add .txt in error file name.
2. package.json: Removed dependencies cors and express.
3. tsconfig.json: Added in compilerOptions: "lib": ["es2017", "dom"].
4. svn:ignore: Ignore node_module, package-lock.json, *.xml, *.txt.


git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61227 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-15 14:41:59 +00:00
konstantina.galouni 4d5d3ea0f0 [Trunk | Explore]: Added sitemaps folder to create sitemap.xml file with all urls from research outcomes search page with one filter value selected foreach filter value. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@61220 d315682c-612b-4755-9ff5-7f18f6832af3
 2021-06-15 13:19:37 +00:00
konstantina.galouni 0bf05409cc [Trunk | Cache]: cache.js: In longCachingRequests add "/api/communities/" (zenodo communities request) and "/openaire/contexts/" (communities with status info). 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59649 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-10-20 14:53:26 +00:00
k.triantafyllou d71054cca7 [Utils Service | Trunk]: Add upload method for stakeholders 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59489 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-28 21:41:23 +00:00
k.triantafyllou 2af5bb33e4 [Utils Service | Trunk]: Add upload method for stakeholders 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59488 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-28 21:32:09 +00:00
konstantina.galouni a7b3be3faa [Trunk | Explore Services]: cache.js: In longCachingRequests include paths "/full" (and "/communityFull") and "/openaire/info" (for last index date). 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59383 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-15 11:54:35 +00:00
k.triantafyllou aac1a2d5e9 [Cache]: Add usage counts endpoints to long cache 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59363 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-09-14 15:07:05 +00:00
k.triantafyllou 977c465079 [Cache | Trunk]: Change size to memsize while it is setting the cache current entries on metrics. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59194 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-07-29 14:09:45 +00:00
argiro.kokogiannaki cac44d718e Cache service: 
-change console.debug to log
-mv resetAtMidnight inside app.listen


git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59002 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-06-26 14:44:28 +00:00
argiro.kokogiannaki a9652670b3 [Cahce service] 
Add:
const expireShort = 2 * 60 * 1000; //2mins
const expireLong = 24 * 60 * 60 * 1000; //24 hours
const cacheMaxSize =500;
const longCachingRequests = ["/communityFull/", "/pagehelpcontent","/provision/mvc/vocabularies/","/pages?page_route="];

-use expireLongTime for longCachingRequests
-add /clear request
-add timer to clean the cache on midnight
-when maxcachesize is reached clean the cache



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@59001 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-06-26 13:33:35 +00:00
argiro.kokogiannaki cfed36d65b Cache: 
Change URL method to work with older nodejs versions



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58699 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-05-14 16:48:40 +00:00
k.triantafyllou 86c65dbd81 [Cache | Trunk]: Fix some issues on metrics. Remove encode url 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58495 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-04-15 13:53:37 +00:00
k.triantafyllou eabd920aa2 [Cache | Trunk]: Add new metrics, encode url 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58478 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-04-10 14:29:46 +00:00
k.triantafyllou 1fff0bd89e [Cache | Trunk]: Resolved some warnings 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58226 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-03-11 12:53:14 +00:00
k.triantafyllou b2b15b5d6c [Cache | Trunk]: Add prometheus metrics 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@58225 d315682c-612b-4755-9ff5-7f18f6832af3
 2020-03-11 12:51:10 +00:00
argiro.kokogiannaki 05a7e9560e [Trunk| Services: utils] 
add file with production properties
UPload:
	add parameter type (default is csv)
	allow upload of json
Check for post: Split url to ? in case it contains query parameters




git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@57871 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-12-13 09:35:51 +00:00
argiro.kokogiannaki fe92edf0da [Cache service | Trunk] 
1.add url in logs
2. add response in cache only for requests that return code 200



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@57529 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-11-06 11:09:06 +00:00
k.triantafyllou 2d888181a0 [UtilsService]: Add on allowed file types application/vnd.ms-excel for windows. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56956 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-09-03 13:12:09 +00:00
argiro.kokogiannaki 4078b4db45 UploadService: 
add timer before deleting the file (sometimes it was deleting the file, before read it)



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56752 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-07-24 12:49:32 +00:00
argiro.kokogiannaki 962446ba06 Cache:update dependencies 
Upload service moved to utils-services



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56738 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-07-24 08:17:05 +00:00
k.triantafyllou f393aa0bd6 [Trunk|Services]: Add at beta properties file size limit. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56261 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-28 10:13:53 +00:00
k.triantafyllou c79f57b8a4 [Trunk|Services]: Upload Service: add file size limit 200KB. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56255 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-28 09:33:28 +00:00
k.triantafyllou 7f6c98120e [Trunk|Services]: Upload Service: add file size limit 200KB. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56253 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-28 09:28:25 +00:00
k.triantafyllou 7b07addaf5 [UploadService]: Check if a url is already tiny. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56240 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-27 13:44:54 +00:00
argiro.kokogiannaki 44dc17c6c1 UploadService: 
add properties file
	make it work for beta
	add new property local path


git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56159 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-21 11:11:50 +00:00
k.triantafyllou b70716fb5f [UploadService]: Allow specific Origin for every request. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56113 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-19 08:34:20 +00:00
k.triantafyllou d8c7ab76f7 [Trunk|Services]: UploadService: Tiny response is a plain text. 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56032 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-10 07:18:45 +00:00
k.triantafyllou 00900b83cf [Trunk|Service]: Add tinyUrl get method at /tiny?url=? 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56010 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-06 10:06:50 +00:00
argiro.kokogiannaki ccf1419ab6 [Services-Upload|Trunk] 
Merge  -r 55574:56005 into Trunk
	new upload/ download/delete methods
	create and use properties file



git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@56006 d315682c-612b-4755-9ff5-7f18f6832af3
 2019-06-06 09:31:03 +00:00
argiro.kokogiannaki bad084b3e9 Initial commit of production portal folder 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@52744 d315682c-612b-4755-9ff5-7f18f6832af3
 2018-07-13 11:56:03 +00:00
argiro.kokogiannaki 109db6590f Add host for redis, get proper answer, add cors 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@49714 d315682c-612b-4755-9ff5-7f18f6832af3
 2017-10-27 11:59:57 +00:00
argiro.kokogiannaki c668c7368b Commit a first draft of caching service for redis and memcache 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@49707 d315682c-612b-4755-9ff5-7f18f6832af3
 2017-10-26 15:33:59 +00:00
argiro.kokogiannaki 411f437f45 moving files under folder 'portal', create folder services and add upload service 
git-svn-id: https://svn.driver.research-infrastructures.eu/driver/dnet40/modules/uoa-services-portal/trunk@43614 d315682c-612b-4755-9ff5-7f18f6832af3
 2016-09-09 07:59:26 +00:00