andreas.czerniak
/
BrBETA_dnet-hadoop
forked from D-Net/dnet-hadoop
1
1
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

sync with master branch

This commit is contained in:
Claudio Atzori 2020-05-04 16:44:13 +02:00
parent cc67dbff81 de5fbe325c
commit 24d8d097b6
384 changed files with 26137 additions and 17398 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

661
LICENSE Normal file
View File

@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<http://www.gnu.org/licenses/>.

2
dhp-build/dhp-build-assembly-resources/pom.xml
View File

@ -12,6 +12,8 @@
<artifactId>dhp-build-assembly-resources</artifactId> <artifactId>dhp-build-assembly-resources</artifactId>
<packaging>jar</packaging> <packaging>jar</packaging>
<description>This module contains a set of scripts supporting the build lifecycle for the dnet-hadoop project</description>
<build> <build>
<plugins> <plugins>
<plugin> <plugin>

49
dhp-build/dhp-build-properties-maven-plugin/pom.xml
View File

@ -12,22 +12,29 @@
<artifactId>dhp-build-properties-maven-plugin</artifactId> <artifactId>dhp-build-properties-maven-plugin</artifactId>
<packaging>maven-plugin</packaging> <packaging>maven-plugin</packaging>
<description>This module is a maven plugin implementing custom properties substitutions in the build lifecycle</description>
<dependencies> <dependencies>
<dependency> <dependency>
<groupId>org.apache.maven</groupId> <groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-api</artifactId> <artifactId>maven-plugin-api</artifactId>
<version>2.0</version> <version>3.6.3</version>
</dependency> </dependency>
<dependency> <dependency>
<groupId>org.apache.maven</groupId> <groupId>org.apache.maven</groupId>
<artifactId>maven-project</artifactId> <artifactId>maven-project</artifactId>
<version>2.0</version> <version>2.2.1</version>
</dependency> </dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-artifact</artifactId>
<version>2.2.1</version>
</dependency>
<dependency> <dependency>
<groupId>org.kuali.maven.plugins</groupId> <groupId>org.kuali.maven.plugins</groupId>
<artifactId>properties-maven-plugin</artifactId> <artifactId>properties-maven-plugin</artifactId>
<version>1.3.2</version> <version>${properties.maven.plugin.version}</version>
</dependency> </dependency>
<dependency> <dependency>
<groupId>com.google.code.findbugs</groupId> <groupId>com.google.code.findbugs</groupId>
@ -73,44 +80,10 @@
<artifactId>maven-javadoc-plugin</artifactId> <artifactId>maven-javadoc-plugin</artifactId>
<configuration> <configuration>
<detectLinks>true</detectLinks> <detectLinks>true</detectLinks>
<doclint>none</doclint>
</configuration> </configuration>
</plugin> </plugin>
</plugins> </plugins>
<pluginManagement>
<plugins>
<!--This plugin's configuration is used to store Eclipse m2e settings only. It has no influence on the Maven build itself.-->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>
org.apache.maven.plugins
</groupId>
<artifactId>
maven-plugin-plugin
</artifactId>
<versionRange>
[3.2,)
</versionRange>
<goals>
<goal>descriptor</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore />
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build> </build>
</project> </project>

101
dhp-build/dhp-build-properties-maven-plugin/src/main/java/eu/dnetlib/maven/plugin/properties/GenerateOoziePropertiesMojo.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.maven.plugin.properties; package eu.dnetlib.maven.plugin.properties;
import java.io.File; import java.io.File;
@ -12,60 +13,64 @@ import org.apache.maven.plugin.MojoFailureException;
/** /**
* Generates oozie properties which were not provided from commandline. * Generates oozie properties which were not provided from commandline.
* @author mhorst
* *
* @author mhorst
* @goal generate-properties * @goal generate-properties
*/ */
public class GenerateOoziePropertiesMojo extends AbstractMojo { public class GenerateOoziePropertiesMojo extends AbstractMojo {
public static final String PROPERTY_NAME_WF_SOURCE_DIR = "workflow.source.dir"; public static final String PROPERTY_NAME_WF_SOURCE_DIR = "workflow.source.dir";
public static final String PROPERTY_NAME_SANDBOX_NAME = "sandboxName"; public static final String PROPERTY_NAME_SANDBOX_NAME = "sandboxName";
private final String[] limiters = {"dhp", "dnetlib", "eu"};
@Override
public void execute() throws MojoExecutionException, MojoFailureException {
if (System.getProperties().containsKey(PROPERTY_NAME_WF_SOURCE_DIR) &&
!System.getProperties().containsKey(PROPERTY_NAME_SANDBOX_NAME)) {
String generatedSandboxName = generateSandboxName(System.getProperties().getProperty(
PROPERTY_NAME_WF_SOURCE_DIR));
if (generatedSandboxName!=null) {
System.getProperties().setProperty(PROPERTY_NAME_SANDBOX_NAME,
generatedSandboxName);
} else {
System.out.println("unable to generate sandbox name from path: " +
System.getProperties().getProperty(PROPERTY_NAME_WF_SOURCE_DIR));
}
}
}
/**
* Generates sandbox name from workflow source directory.
* @param wfSourceDir
* @return generated sandbox name
*/
private String generateSandboxName(String wfSourceDir) {
// utilize all dir names until finding one of the limiters
List<String> sandboxNameParts = new ArrayList<String>();
String[] tokens = StringUtils.split(wfSourceDir, File.separatorChar);
ArrayUtils.reverse(tokens);
if (tokens.length>0) {
for (String token : tokens) {
for (String limiter : limiters) {
if (limiter.equals(token)) {
return sandboxNameParts.size()>0?
StringUtils.join(sandboxNameParts.toArray()):null;
}
}
if (sandboxNameParts.size()>0) {
sandboxNameParts.add(0, File.separator);
}
sandboxNameParts.add(0, token);
}
return StringUtils.join(sandboxNameParts.toArray());
} else {
return null;
}
}
private final String[] limiters = {
"dhp", "dnetlib", "eu"
};
@Override
public void execute() throws MojoExecutionException, MojoFailureException {
if (System.getProperties().containsKey(PROPERTY_NAME_WF_SOURCE_DIR)
&& !System.getProperties().containsKey(PROPERTY_NAME_SANDBOX_NAME)) {
String generatedSandboxName = generateSandboxName(
System.getProperties().getProperty(PROPERTY_NAME_WF_SOURCE_DIR));
if (generatedSandboxName != null) {
System.getProperties().setProperty(PROPERTY_NAME_SANDBOX_NAME, generatedSandboxName);
} else {
System.out
.println(
"unable to generate sandbox name from path: "
+ System.getProperties().getProperty(PROPERTY_NAME_WF_SOURCE_DIR));
}
}
}
/**
* Generates sandbox name from workflow source directory.
*
* @param wfSourceDir
* @return generated sandbox name
*/
private String generateSandboxName(String wfSourceDir) {
// utilize all dir names until finding one of the limiters
List<String> sandboxNameParts = new ArrayList<String>();
String[] tokens = StringUtils.split(wfSourceDir, File.separatorChar);
ArrayUtils.reverse(tokens);
if (tokens.length > 0) {
for (String token : tokens) {
for (String limiter : limiters) {
if (limiter.equals(token)) {
return sandboxNameParts.size() > 0
? StringUtils.join(sandboxNameParts.toArray())
: null;
}
}
if (sandboxNameParts.size() > 0) {
sandboxNameParts.add(0, File.separator);
}
sandboxNameParts.add(0, token);
}
return StringUtils.join(sandboxNameParts.toArray());
} else {
return null;
}
}
} }

701
dhp-build/dhp-build-properties-maven-plugin/src/main/java/eu/dnetlib/maven/plugin/properties/WritePredefinedProjectProperties.java
View File

@ -1,17 +1,15 @@
/** /**
* * Licensed under the Educational Community License, Version 2.0 (the "License"); you may not use
* Licensed under the Educational Community License, Version 2.0 (the "License"); * this file except in compliance with the License. You may obtain a copy of the License at
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* *
* http://www.opensource.org/licenses/ecl2.php * <p>http://www.opensource.org/licenses/ecl2.php
* *
* Unless required by applicable law or agreed to in writing, software * <p>Unless required by applicable law or agreed to in writing, software distributed under the
* distributed under the License is distributed on an "AS IS" BASIS, * License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * express or implied. See the License for the specific language governing permissions and
* See the License for the specific language governing permissions and
* limitations under the License. * limitations under the License.
*/ */
package eu.dnetlib.maven.plugin.properties; package eu.dnetlib.maven.plugin.properties;
import java.io.File; import java.io.File;
@ -41,346 +39,361 @@ import org.springframework.core.io.ResourceLoader;
import edu.umd.cs.findbugs.annotations.SuppressFBWarnings; import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
/** /**
* Writes project properties for the keys listed in specified properties files. * Writes project properties for the keys listed in specified properties files. Based on:
* Based on: * http://site.kuali.org/maven/plugins/properties-maven-plugin/2.0.1/write-project-properties-mojo.html
* http://site.kuali.org/maven/plugins/properties-maven-plugin/1.3.2/write-project-properties-mojo.html *
* @author mhorst * @author mhorst
* @goal write-project-properties * @goal write-project-properties
*/ */
public class WritePredefinedProjectProperties extends AbstractMojo { public class WritePredefinedProjectProperties extends AbstractMojo {
private static final String CR = "\r"; private static final String CR = "\r";
private static final String LF = "\n"; private static final String LF = "\n";
private static final String TAB = "\t"; private static final String TAB = "\t";
protected static final String PROPERTY_PREFIX_ENV = "env."; protected static final String PROPERTY_PREFIX_ENV = "env.";
private static final String ENCODING_UTF8 = "utf8"; private static final String ENCODING_UTF8 = "utf8";
/** /** @parameter property="properties.includePropertyKeysFromFiles" */
* @parameter property="properties.includePropertyKeysFromFiles"
*/
private String[] includePropertyKeysFromFiles; private String[] includePropertyKeysFromFiles;
/**
* @parameter default-value="${project}"
* @required
* @readonly
*/
protected MavenProject project;
/**
* The file that properties will be written to
*
* @parameter property="properties.outputFile"
* default-value="${project.build.directory}/properties/project.properties";
* @required
*/
protected File outputFile;
/** /**
* If true, the plugin will silently ignore any non-existent properties files, and the build will continue * @parameter default-value="${project}"
* * @required
* @parameter property="properties.quiet" default-value="true" * @readonly
*/ */
private boolean quiet; protected MavenProject project;
/**
* Comma separated list of characters to escape when writing property values. cr=carriage return, lf=linefeed,
* tab=tab. Any other values are taken literally.
*
* @parameter default-value="cr,lf,tab" property="properties.escapeChars"
*/
private String escapeChars;
/** /**
* If true, the plugin will include system properties when writing the properties file. System properties override * The file that properties will be written to
* both environment variables and project properties. *
* * @parameter property="properties.outputFile"
* @parameter default-value="false" property="properties.includeSystemProperties" * default-value="${project.build.directory}/properties/project.properties";
*/ * @required
private boolean includeSystemProperties; */
protected File outputFile;
/** /**
* If true, the plugin will include environment variables when writing the properties file. Environment variables * If true, the plugin will silently ignore any non-existent properties files, and the build will continue
* are prefixed with "env". Environment variables override project properties. *
* * @parameter property="properties.quiet" default-value="true"
* @parameter default-value="false" property="properties.includeEnvironmentVariables" */
*/ private boolean quiet;
private boolean includeEnvironmentVariables;
/** /**
* Comma separated set of properties to exclude when writing the properties file * Comma separated list of characters to escape when writing property values. cr=carriage return, lf=linefeed,
* * tab=tab. Any other values are taken literally.
* @parameter property="properties.exclude" *
*/ * @parameter default-value="cr,lf,tab" property="properties.escapeChars"
private String exclude; */
private String escapeChars;
/** /**
* Comma separated set of properties to write to the properties file. If provided, only the properties matching * If true, the plugin will include system properties when writing the properties file. System properties override
* those supplied here will be written to the properties file. * both environment variables and project properties.
* *
* @parameter property="properties.include" * @parameter default-value="false" property="properties.includeSystemProperties"
*/ */
private String include; private boolean includeSystemProperties;
/* (non-Javadoc) /**
* @see org.apache.maven.plugin.AbstractMojo#execute() * If true, the plugin will include environment variables when writing the properties file. Environment variables
*/ * are prefixed with "env". Environment variables override project properties.
@Override *
@SuppressFBWarnings({"NP_UNWRITTEN_FIELD","UWF_UNWRITTEN_FIELD"}) * @parameter default-value="false" property="properties.includeEnvironmentVariables"
public void execute() throws MojoExecutionException, MojoFailureException { */
Properties properties = new Properties(); private boolean includeEnvironmentVariables;
// Add project properties
properties.putAll(project.getProperties());
if (includeEnvironmentVariables) {
// Add environment variables, overriding any existing properties with the same key
properties.putAll(getEnvironmentVariables());
}
if (includeSystemProperties) {
// Add system properties, overriding any existing properties with the same key
properties.putAll(System.getProperties());
}
// Remove properties as appropriate /**
trim(properties, exclude, include); * Comma separated set of properties to exclude when writing the properties file
*
* @parameter property="properties.exclude"
*/
private String exclude;
String comment = "# " + new Date() + "\n"; /**
List<String> escapeTokens = getEscapeChars(escapeChars); * Comma separated set of properties to write to the properties file. If provided, only the properties matching
* those supplied here will be written to the properties file.
*
* @parameter property="properties.include"
*/
private String include;
getLog().info("Creating " + outputFile); /*
writeProperties(outputFile, comment, properties, escapeTokens); * (non-Javadoc)
} * @see org.apache.maven.plugin.AbstractMojo#execute()
*/
@Override
@SuppressFBWarnings({
"NP_UNWRITTEN_FIELD", "UWF_UNWRITTEN_FIELD"
})
public void execute() throws MojoExecutionException, MojoFailureException {
Properties properties = new Properties();
// Add project properties
properties.putAll(project.getProperties());
if (includeEnvironmentVariables) {
// Add environment variables, overriding any existing properties with the same key
properties.putAll(getEnvironmentVariables());
}
if (includeSystemProperties) {
// Add system properties, overriding any existing properties with the same key
properties.putAll(System.getProperties());
}
/** // Remove properties as appropriate
* Provides environment variables. trim(properties, exclude, include);
* @return environment variables
*/
protected static Properties getEnvironmentVariables() {
Properties props = new Properties();
for (Entry<String, String> entry : System.getenv().entrySet()) {
props.setProperty(PROPERTY_PREFIX_ENV + entry.getKey(), entry.getValue());
}
return props;
}
/** String comment = "# " + new Date() + "\n";
* Removes properties which should not be written. List<String> escapeTokens = getEscapeChars(escapeChars);
* @param properties
* @param omitCSV
* @param includeCSV
* @throws MojoExecutionException
*/
protected void trim(Properties properties, String omitCSV, String includeCSV) throws MojoExecutionException {
List<String> omitKeys = getListFromCSV(omitCSV);
for (String key : omitKeys) {
properties.remove(key);
}
List<String> includeKeys = getListFromCSV(includeCSV);
// mh: including keys from predefined properties
if (includePropertyKeysFromFiles!=null && includePropertyKeysFromFiles.length>0) {
for (String currentIncludeLoc : includePropertyKeysFromFiles) {
if (validate(currentIncludeLoc)) {
Properties p = getProperties(currentIncludeLoc);
for (String key : p.stringPropertyNames()) {
includeKeys.add(key);
}
}
}
}
if (includeKeys!=null && !includeKeys.isEmpty()) {
// removing only when include keys provided
Set<String> keys = properties.stringPropertyNames();
for (String key : keys) {
if (!includeKeys.contains(key)) {
properties.remove(key);
}
}
}
}
/** getLog().info("Creating " + outputFile);
* Checks whether file exists. writeProperties(outputFile, comment, properties, escapeTokens);
* @param location }
* @return true when exists, false otherwise.
*/
protected boolean exists(String location) {
if (StringUtils.isBlank(location)) {
return false;
}
File file = new File(location);
if (file.exists()) {
return true;
}
ResourceLoader loader = new DefaultResourceLoader();
Resource resource = loader.getResource(location);
return resource.exists();
}
/** /**
* Validates resource location. * Provides environment variables.
* @param location *
* @return true when valid, false otherwise * @return environment variables
* @throws MojoExecutionException */
*/ protected static Properties getEnvironmentVariables() {
protected boolean validate(String location) throws MojoExecutionException { Properties props = new Properties();
boolean exists = exists(location); for (Entry<String, String> entry : System.getenv().entrySet()) {
if (exists) { props.setProperty(PROPERTY_PREFIX_ENV + entry.getKey(), entry.getValue());
return true; }
} return props;
if (quiet) { }
getLog().info("Ignoring non-existent properties file '" + location + "'");
return false;
} else {
throw new MojoExecutionException("Non-existent properties file '" + location + "'");
}
}
/** /**
* Provides input stream. * Removes properties which should not be written.
* @param location *
* @return input stream * @param properties
* @throws IOException * @param omitCSV
*/ * @param includeCSV
protected InputStream getInputStream(String location) throws IOException { * @throws MojoExecutionException
File file = new File(location); */
if (file.exists()) { protected void trim(Properties properties, String omitCSV, String includeCSV)
return new FileInputStream(location); throws MojoExecutionException {
} List<String> omitKeys = getListFromCSV(omitCSV);
ResourceLoader loader = new DefaultResourceLoader(); for (String key : omitKeys) {
Resource resource = loader.getResource(location); properties.remove(key);
return resource.getInputStream(); }
}
/** List<String> includeKeys = getListFromCSV(includeCSV);
* Creates properties for given location. // mh: including keys from predefined properties
* @param location if (includePropertyKeysFromFiles != null && includePropertyKeysFromFiles.length > 0) {
* @return properties for given location for (String currentIncludeLoc : includePropertyKeysFromFiles) {
* @throws MojoExecutionException if (validate(currentIncludeLoc)) {
*/ Properties p = getProperties(currentIncludeLoc);
protected Properties getProperties(String location) throws MojoExecutionException { for (String key : p.stringPropertyNames()) {
InputStream in = null; includeKeys.add(key);
try { }
Properties properties = new Properties(); }
in = getInputStream(location); }
if (location.toLowerCase().endsWith(".xml")) { }
properties.loadFromXML(in); if (includeKeys != null && !includeKeys.isEmpty()) {
} else { // removing only when include keys provided
properties.load(in); Set<String> keys = properties.stringPropertyNames();
} for (String key : keys) {
return properties; if (!includeKeys.contains(key)) {
} catch (IOException e) { properties.remove(key);
throw new MojoExecutionException("Error reading properties file " + location, e); }
} finally { }
IOUtils.closeQuietly(in); }
} }
}
/** /**
* Provides escape characters. * Checks whether file exists.
* @param escapeChars *
* @return escape characters * @param location
*/ * @return true when exists, false otherwise.
protected List<String> getEscapeChars(String escapeChars) { */
List<String> tokens = getListFromCSV(escapeChars); protected boolean exists(String location) {
List<String> realTokens = new ArrayList<String>(); if (StringUtils.isBlank(location)) {
for (String token : tokens) { return false;
String realToken = getRealToken(token); }
realTokens.add(realToken); File file = new File(location);
} if (file.exists()) {
return realTokens; return true;
} }
ResourceLoader loader = new DefaultResourceLoader();
Resource resource = loader.getResource(location);
return resource.exists();
}
/** /**
* Provides real token. * Validates resource location.
* @param token *
* @return real token * @param location
*/ * @return true when valid, false otherwise
protected String getRealToken(String token) { * @throws MojoExecutionException
if (token.equalsIgnoreCase("CR")) { */
return CR; protected boolean validate(String location) throws MojoExecutionException {
} else if (token.equalsIgnoreCase("LF")) { boolean exists = exists(location);
return LF; if (exists) {
} else if (token.equalsIgnoreCase("TAB")) { return true;
return TAB; }
} else { if (quiet) {
return token; getLog().info("Ignoring non-existent properties file '" + location + "'");
} return false;
} } else {
throw new MojoExecutionException("Non-existent properties file '" + location + "'");
}
}
/** /**
* Returns content. * Provides input stream.
* @param comment *
* @param properties * @param location
* @param escapeTokens * @return input stream
* @return content * @throws IOException
*/ */
protected String getContent(String comment, Properties properties, List<String> escapeTokens) { protected InputStream getInputStream(String location) throws IOException {
List<String> names = new ArrayList<String>(properties.stringPropertyNames()); File file = new File(location);
Collections.sort(names); if (file.exists()) {
StringBuilder sb = new StringBuilder(); return new FileInputStream(location);
if (!StringUtils.isBlank(comment)) { }
sb.append(comment); ResourceLoader loader = new DefaultResourceLoader();
} Resource resource = loader.getResource(location);
for (String name : names) { return resource.getInputStream();
String value = properties.getProperty(name); }
String escapedValue = escape(value, escapeTokens);
sb.append(name + "=" + escapedValue + "\n");
}
return sb.toString();
}
/** /**
* Writes properties to given file. * Creates properties for given location.
* @param file *
* @param comment * @param location
* @param properties * @return properties for given location
* @param escapeTokens * @throws MojoExecutionException
* @throws MojoExecutionException */
*/ protected Properties getProperties(String location) throws MojoExecutionException {
protected void writeProperties(File file, String comment, Properties properties, List<String> escapeTokens) InputStream in = null;
throws MojoExecutionException { try {
try { Properties properties = new Properties();
String content = getContent(comment, properties, escapeTokens); in = getInputStream(location);
FileUtils.writeStringToFile(file, content, ENCODING_UTF8); if (location.toLowerCase().endsWith(".xml")) {
} catch (IOException e) { properties.loadFromXML(in);
throw new MojoExecutionException("Error creating properties file", e); } else {
} properties.load(in);
} }
return properties;
/** } catch (IOException e) {
* Escapes characters. throw new MojoExecutionException("Error reading properties file " + location, e);
* @param s } finally {
* @param escapeChars IOUtils.closeQuietly(in);
* @return }
*/ }
protected String escape(String s, List<String> escapeChars) {
String result = s;
for (String escapeChar : escapeChars) {
result = result.replace(escapeChar, getReplacementToken(escapeChar));
}
return result;
}
/** /**
* Provides replacement token. * Provides escape characters.
* @param escapeChar *
* @return replacement token * @param escapeChars
*/ * @return escape characters
protected String getReplacementToken(String escapeChar) { */
if (escapeChar.equals(CR)) { protected List<String> getEscapeChars(String escapeChars) {
return "\\r"; List<String> tokens = getListFromCSV(escapeChars);
} else if (escapeChar.equals(LF)) { List<String> realTokens = new ArrayList<String>();
return "\\n"; for (String token : tokens) {
} else if (escapeChar.equals(TAB)) { String realToken = getRealToken(token);
return "\\t"; realTokens.add(realToken);
} else { }
return "\\" + escapeChar; return realTokens;
} }
}
/** /**
* Provides real token.
*
* @param token
* @return real token
*/
protected String getRealToken(String token) {
if (token.equalsIgnoreCase("CR")) {
return CR;
} else if (token.equalsIgnoreCase("LF")) {
return LF;
} else if (token.equalsIgnoreCase("TAB")) {
return TAB;
} else {
return token;
}
}
/**
* Returns content.
*
* @param comment
* @param properties
* @param escapeTokens
* @return content
*/
protected String getContent(String comment, Properties properties, List<String> escapeTokens) {
List<String> names = new ArrayList<String>(properties.stringPropertyNames());
Collections.sort(names);
StringBuilder sb = new StringBuilder();
if (!StringUtils.isBlank(comment)) {
sb.append(comment);
}
for (String name : names) {
String value = properties.getProperty(name);
String escapedValue = escape(value, escapeTokens);
sb.append(name + "=" + escapedValue + "\n");
}
return sb.toString();
}
/**
* Writes properties to given file.
*
* @param file
* @param comment
* @param properties
* @param escapeTokens
* @throws MojoExecutionException
*/
protected void writeProperties(
File file, String comment, Properties properties, List<String> escapeTokens)
throws MojoExecutionException {
try {
String content = getContent(comment, properties, escapeTokens);
FileUtils.writeStringToFile(file, content, ENCODING_UTF8);
} catch (IOException e) {
throw new MojoExecutionException("Error creating properties file", e);
}
}
/**
* Escapes characters.
*
* @param s
* @param escapeChars
* @return
*/
protected String escape(String s, List<String> escapeChars) {
String result = s;
for (String escapeChar : escapeChars) {
result = result.replace(escapeChar, getReplacementToken(escapeChar));
}
return result;
}
/**
* Provides replacement token.
*
* @param escapeChar
* @return replacement token
*/
protected String getReplacementToken(String escapeChar) {
if (escapeChar.equals(CR)) {
return "\\r";
} else if (escapeChar.equals(LF)) {
return "\\n";
} else if (escapeChar.equals(TAB)) {
return "\\t";
} else {
return "\\" + escapeChar;
}
}
/**
* Returns list from csv. * Returns list from csv.
*
* @param csv * @param csv
* @return list of values generated from CSV * @return list of values generated from CSV
*/ */
@ -396,41 +409,39 @@ public class WritePredefinedProjectProperties extends AbstractMojo {
return list; return list;
} }
public void setIncludeSystemProperties(boolean includeSystemProperties) { public void setIncludeSystemProperties(boolean includeSystemProperties) {
this.includeSystemProperties = includeSystemProperties; this.includeSystemProperties = includeSystemProperties;
} }
public void setEscapeChars(String escapeChars) { public void setEscapeChars(String escapeChars) {
this.escapeChars = escapeChars; this.escapeChars = escapeChars;
} }
public void setIncludeEnvironmentVariables(boolean includeEnvironmentVariables) { public void setIncludeEnvironmentVariables(boolean includeEnvironmentVariables) {
this.includeEnvironmentVariables = includeEnvironmentVariables; this.includeEnvironmentVariables = includeEnvironmentVariables;
} }
public void setExclude(String exclude) { public void setExclude(String exclude) {
this.exclude = exclude; this.exclude = exclude;
} }
public void setInclude(String include) { public void setInclude(String include) {
this.include = include; this.include = include;
} }
public void setQuiet(boolean quiet) {
this.quiet = quiet;
}
public void setQuiet(boolean quiet) {
this.quiet = quiet;
}
/** /**
* Sets property files for which keys properties should be included. * Sets property files for which keys properties should be included.
*
* @param includePropertyKeysFromFiles * @param includePropertyKeysFromFiles
*/ */
public void setIncludePropertyKeysFromFiles( public void setIncludePropertyKeysFromFiles(String[] includePropertyKeysFromFiles) {
String[] includePropertyKeysFromFiles) { if (includePropertyKeysFromFiles != null) {
if (includePropertyKeysFromFiles!=null) { this.includePropertyKeysFromFiles = Arrays
this.includePropertyKeysFromFiles = Arrays.copyOf( .copyOf(includePropertyKeysFromFiles, includePropertyKeysFromFiles.length);
includePropertyKeysFromFiles,
includePropertyKeysFromFiles.length);
} }
} }
}
}

177
dhp-build/dhp-build-properties-maven-plugin/src/test/java/eu/dnetlib/maven/plugin/properties/GenerateOoziePropertiesMojoTest.java
View File

@ -1,100 +1,97 @@
package eu.dnetlib.maven.plugin.properties;
import org.junit.jupiter.api.BeforeEach; package eu.dnetlib.maven.plugin.properties;
import org.junit.jupiter.api.Test;
import static eu.dnetlib.maven.plugin.properties.GenerateOoziePropertiesMojo.PROPERTY_NAME_SANDBOX_NAME; import static eu.dnetlib.maven.plugin.properties.GenerateOoziePropertiesMojo.PROPERTY_NAME_SANDBOX_NAME;
import static eu.dnetlib.maven.plugin.properties.GenerateOoziePropertiesMojo.PROPERTY_NAME_WF_SOURCE_DIR; import static eu.dnetlib.maven.plugin.properties.GenerateOoziePropertiesMojo.PROPERTY_NAME_WF_SOURCE_DIR;
import static org.junit.jupiter.api.Assertions.*; import static org.junit.jupiter.api.Assertions.*;
/** import org.junit.jupiter.api.BeforeEach;
* @author mhorst, claudio.atzori import org.junit.jupiter.api.Test;
*
*/ /** @author mhorst, claudio.atzori */
public class GenerateOoziePropertiesMojoTest { public class GenerateOoziePropertiesMojoTest {
private GenerateOoziePropertiesMojo mojo = new GenerateOoziePropertiesMojo(); private final GenerateOoziePropertiesMojo mojo = new GenerateOoziePropertiesMojo();
@BeforeEach
public void clearSystemProperties() {
System.clearProperty(PROPERTY_NAME_SANDBOX_NAME);
System.clearProperty(PROPERTY_NAME_WF_SOURCE_DIR);
}
@Test
public void testExecuteEmpty() throws Exception {
// execute
mojo.execute();
// assert
assertNull(System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
}
@Test @BeforeEach
public void testExecuteSandboxNameAlreadySet() throws Exception { public void clearSystemProperties() {
// given System.clearProperty(PROPERTY_NAME_SANDBOX_NAME);
String workflowSourceDir = "eu/dnetlib/dhp/wf/transformers"; System.clearProperty(PROPERTY_NAME_WF_SOURCE_DIR);
String sandboxName = "originalSandboxName"; }
System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir);
System.setProperty(PROPERTY_NAME_SANDBOX_NAME, sandboxName); @Test
public void testExecuteEmpty() throws Exception {
// execute // execute
mojo.execute(); mojo.execute();
// assert // assert
assertEquals(sandboxName, System.getProperty(PROPERTY_NAME_SANDBOX_NAME)); assertNull(System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
} }
@Test @Test
public void testExecuteEmptyWorkflowSourceDir() throws Exception { public void testExecuteSandboxNameAlreadySet() throws Exception {
// given // given
String workflowSourceDir = ""; String workflowSourceDir = "eu/dnetlib/dhp/wf/transformers";
System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir); String sandboxName = "originalSandboxName";
System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir);
// execute System.setProperty(PROPERTY_NAME_SANDBOX_NAME, sandboxName);
mojo.execute();
// execute
// assert mojo.execute();
assertNull(System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
} // assert
assertEquals(sandboxName, System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
@Test }
public void testExecuteNullSandboxNameGenerated() throws Exception {
// given @Test
String workflowSourceDir = "eu/dnetlib/dhp/"; public void testExecuteEmptyWorkflowSourceDir() throws Exception {
System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir); // given
String workflowSourceDir = "";
// execute System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir);
mojo.execute();
// execute
// assert mojo.execute();
assertNull(System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
} // assert
assertNull(System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
@Test }
public void testExecute() throws Exception {
// given @Test
String workflowSourceDir = "eu/dnetlib/dhp/wf/transformers"; public void testExecuteNullSandboxNameGenerated() throws Exception {
System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir); // given
String workflowSourceDir = "eu/dnetlib/dhp/";
// execute System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir);
mojo.execute();
// execute
// assert mojo.execute();
assertEquals("wf/transformers", System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
} // assert
assertNull(System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
@Test }
public void testExecuteWithoutRoot() throws Exception {
// given @Test
String workflowSourceDir = "wf/transformers"; public void testExecute() throws Exception {
System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir); // given
String workflowSourceDir = "eu/dnetlib/dhp/wf/transformers";
// execute System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir);
mojo.execute();
// execute
// assert mojo.execute();
assertEquals("wf/transformers", System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
} // assert
assertEquals("wf/transformers", System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
}
@Test
public void testExecuteWithoutRoot() throws Exception {
// given
String workflowSourceDir = "wf/transformers";
System.setProperty(PROPERTY_NAME_WF_SOURCE_DIR, workflowSourceDir);
// execute
mojo.execute();
// assert
assertEquals("wf/transformers", System.getProperty(PROPERTY_NAME_SANDBOX_NAME));
}
} }

695
dhp-build/dhp-build-properties-maven-plugin/src/test/java/eu/dnetlib/maven/plugin/properties/WritePredefinedProjectPropertiesTest.java
View File

@ -1,5 +1,14 @@
package eu.dnetlib.maven.plugin.properties; package eu.dnetlib.maven.plugin.properties;
import static eu.dnetlib.maven.plugin.properties.WritePredefinedProjectProperties.PROPERTY_PREFIX_ENV;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.lenient;
import java.io.*;
import java.util.Properties;
import org.apache.maven.plugin.MojoExecutionException; import org.apache.maven.plugin.MojoExecutionException;
import org.apache.maven.project.MavenProject; import org.apache.maven.project.MavenProject;
import org.junit.jupiter.api.*; import org.junit.jupiter.api.*;
@ -9,347 +18,357 @@ import org.mockito.Mock;
import org.mockito.MockitoAnnotations; import org.mockito.MockitoAnnotations;
import org.mockito.junit.jupiter.MockitoExtension; import org.mockito.junit.jupiter.MockitoExtension;
import java.io.*; /** @author mhorst, claudio.atzori */
import java.util.Properties;
import static eu.dnetlib.maven.plugin.properties.WritePredefinedProjectProperties.PROPERTY_PREFIX_ENV;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.lenient;
/**
* @author mhorst, claudio.atzori
*
*/
@ExtendWith(MockitoExtension.class) @ExtendWith(MockitoExtension.class)
public class WritePredefinedProjectPropertiesTest { public class WritePredefinedProjectPropertiesTest {
@Mock @Mock
private MavenProject mavenProject; private MavenProject mavenProject;
private WritePredefinedProjectProperties mojo;
@BeforeEach private WritePredefinedProjectProperties mojo;
public void init(@TempDir File testFolder) {
MockitoAnnotations.initMocks(this);
mojo = new WritePredefinedProjectProperties();
mojo.outputFile = getPropertiesFileLocation(testFolder);
mojo.project = mavenProject;
lenient().doReturn(new Properties()).when(mavenProject).getProperties();
}
// ----------------------------------- TESTS --------------------------------------------- @BeforeEach
public void init(@TempDir File testFolder) {
@Test MockitoAnnotations.initMocks(this);
public void testExecuteEmpty() throws Exception { mojo = new WritePredefinedProjectProperties();
// execute mojo.outputFile = getPropertiesFileLocation(testFolder);
mojo.execute(); mojo.project = mavenProject;
lenient().doReturn(new Properties()).when(mavenProject).getProperties();
// assert }
assertTrue(mojo.outputFile.exists());
Properties storedProperties = getStoredProperties(mojo.outputFile.getParentFile()); // ----------------------------------- TESTS ---------------------------------------------
assertEquals(0, storedProperties.size());
} @Test
public void testExecuteEmpty() throws Exception {
@Test // execute
public void testExecuteWithProjectProperties() throws Exception { mojo.execute();
// given
String key = "projectPropertyKey"; // assert
String value = "projectPropertyValue"; assertTrue(mojo.outputFile.exists());
Properties projectProperties = new Properties(); Properties storedProperties = getStoredProperties(mojo.outputFile.getParentFile());
projectProperties.setProperty(key, value); assertEquals(0, storedProperties.size());
doReturn(projectProperties).when(mavenProject).getProperties(); }
// execute @Test
mojo.execute(); public void testExecuteWithProjectProperties() throws Exception {
// given
// assert String key = "projectPropertyKey";
assertTrue(mojo.outputFile.exists()); String value = "projectPropertyValue";
Properties storedProperties = getStoredProperties(mojo.outputFile.getParentFile()); Properties projectProperties = new Properties();
assertEquals(1, storedProperties.size()); projectProperties.setProperty(key, value);
assertTrue(storedProperties.containsKey(key)); doReturn(projectProperties).when(mavenProject).getProperties();
assertEquals(value, storedProperties.getProperty(key));
} // execute
mojo.execute();
@Test()
public void testExecuteWithProjectPropertiesAndInvalidOutputFile(@TempDir File testFolder) { // assert
// given assertTrue(mojo.outputFile.exists());
String key = "projectPropertyKey"; Properties storedProperties = getStoredProperties(mojo.outputFile.getParentFile());
String value = "projectPropertyValue"; assertEquals(1, storedProperties.size());
Properties projectProperties = new Properties(); assertTrue(storedProperties.containsKey(key));
projectProperties.setProperty(key, value); assertEquals(value, storedProperties.getProperty(key));
doReturn(projectProperties).when(mavenProject).getProperties(); }
mojo.outputFile = testFolder;
@Test()
// execute public void testExecuteWithProjectPropertiesAndInvalidOutputFile(@TempDir File testFolder) {
Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute()); // given
} String key = "projectPropertyKey";
String value = "projectPropertyValue";
@Test Properties projectProperties = new Properties();
public void testExecuteWithProjectPropertiesExclusion(@TempDir File testFolder) throws Exception { projectProperties.setProperty(key, value);
// given doReturn(projectProperties).when(mavenProject).getProperties();
String key = "projectPropertyKey"; mojo.outputFile = testFolder;
String value = "projectPropertyValue";
String excludedKey = "excludedPropertyKey"; // execute
String excludedValue = "excludedPropertyValue"; Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute());
Properties projectProperties = new Properties(); }
projectProperties.setProperty(key, value);
projectProperties.setProperty(excludedKey, excludedValue); @Test
doReturn(projectProperties).when(mavenProject).getProperties(); public void testExecuteWithProjectPropertiesExclusion(@TempDir File testFolder) throws Exception {
mojo.setExclude(excludedKey); // given
String key = "projectPropertyKey";
// execute String value = "projectPropertyValue";
mojo.execute(); String excludedKey = "excludedPropertyKey";
String excludedValue = "excludedPropertyValue";
// assert Properties projectProperties = new Properties();
assertTrue(mojo.outputFile.exists()); projectProperties.setProperty(key, value);
Properties storedProperties = getStoredProperties(testFolder); projectProperties.setProperty(excludedKey, excludedValue);
assertEquals(1, storedProperties.size()); doReturn(projectProperties).when(mavenProject).getProperties();
assertTrue(storedProperties.containsKey(key)); mojo.setExclude(excludedKey);
assertEquals(value, storedProperties.getProperty(key));
} // execute
mojo.execute();
@Test
public void testExecuteWithProjectPropertiesInclusion(@TempDir File testFolder) throws Exception { // assert
// given assertTrue(mojo.outputFile.exists());
String key = "projectPropertyKey"; Properties storedProperties = getStoredProperties(testFolder);
String value = "projectPropertyValue"; assertEquals(1, storedProperties.size());
String includedKey = "includedPropertyKey"; assertTrue(storedProperties.containsKey(key));
String includedValue = "includedPropertyValue"; assertEquals(value, storedProperties.getProperty(key));
Properties projectProperties = new Properties(); }
projectProperties.setProperty(key, value);
projectProperties.setProperty(includedKey, includedValue); @Test
doReturn(projectProperties).when(mavenProject).getProperties(); public void testExecuteWithProjectPropertiesInclusion(@TempDir File testFolder) throws Exception {
mojo.setInclude(includedKey); // given
String key = "projectPropertyKey";
// execute String value = "projectPropertyValue";
mojo.execute(); String includedKey = "includedPropertyKey";
String includedValue = "includedPropertyValue";
// assert Properties projectProperties = new Properties();
assertTrue(mojo.outputFile.exists()); projectProperties.setProperty(key, value);
Properties storedProperties = getStoredProperties(testFolder); projectProperties.setProperty(includedKey, includedValue);
assertEquals(1, storedProperties.size()); doReturn(projectProperties).when(mavenProject).getProperties();
assertTrue(storedProperties.containsKey(includedKey)); mojo.setInclude(includedKey);
assertEquals(includedValue, storedProperties.getProperty(includedKey));
} // execute
mojo.execute();
@Test
public void testExecuteIncludingPropertyKeysFromFile(@TempDir File testFolder) throws Exception { // assert
// given assertTrue(mojo.outputFile.exists());
String key = "projectPropertyKey"; Properties storedProperties = getStoredProperties(testFolder);
String value = "projectPropertyValue"; assertEquals(1, storedProperties.size());
String includedKey = "includedPropertyKey"; assertTrue(storedProperties.containsKey(includedKey));
String includedValue = "includedPropertyValue"; assertEquals(includedValue, storedProperties.getProperty(includedKey));
Properties projectProperties = new Properties(); }
projectProperties.setProperty(key, value);
projectProperties.setProperty(includedKey, includedValue); @Test
doReturn(projectProperties).when(mavenProject).getProperties(); public void testExecuteIncludingPropertyKeysFromFile(@TempDir File testFolder) throws Exception {
// given
File includedPropertiesFile = new File(testFolder, "included.properties"); String key = "projectPropertyKey";
Properties includedProperties = new Properties(); String value = "projectPropertyValue";
includedProperties.setProperty(includedKey, "irrelevantValue"); String includedKey = "includedPropertyKey";
includedProperties.store(new FileWriter(includedPropertiesFile), null); String includedValue = "includedPropertyValue";
Properties projectProperties = new Properties();
mojo.setIncludePropertyKeysFromFiles(new String[] {includedPropertiesFile.getAbsolutePath()}); projectProperties.setProperty(key, value);
projectProperties.setProperty(includedKey, includedValue);
// execute doReturn(projectProperties).when(mavenProject).getProperties();
mojo.execute();
File includedPropertiesFile = new File(testFolder, "included.properties");
// assert Properties includedProperties = new Properties();
assertTrue(mojo.outputFile.exists()); includedProperties.setProperty(includedKey, "irrelevantValue");
Properties storedProperties = getStoredProperties(testFolder); includedProperties.store(new FileWriter(includedPropertiesFile), null);
assertEquals(1, storedProperties.size());
assertTrue(storedProperties.containsKey(includedKey)); mojo.setIncludePropertyKeysFromFiles(new String[] {
assertEquals(includedValue, storedProperties.getProperty(includedKey)); includedPropertiesFile.getAbsolutePath()
} });
@Test // execute
public void testExecuteIncludingPropertyKeysFromClasspathResource(@TempDir File testFolder) throws Exception { mojo.execute();
// given
String key = "projectPropertyKey"; // assert
String value = "projectPropertyValue"; assertTrue(mojo.outputFile.exists());
String includedKey = "includedPropertyKey"; Properties storedProperties = getStoredProperties(testFolder);
String includedValue = "includedPropertyValue"; assertEquals(1, storedProperties.size());
Properties projectProperties = new Properties(); assertTrue(storedProperties.containsKey(includedKey));
projectProperties.setProperty(key, value); assertEquals(includedValue, storedProperties.getProperty(includedKey));
projectProperties.setProperty(includedKey, includedValue); }
doReturn(projectProperties).when(mavenProject).getProperties();
@Test
mojo.setIncludePropertyKeysFromFiles(new String[] {"/eu/dnetlib/maven/plugin/properties/included.properties"}); public void testExecuteIncludingPropertyKeysFromClasspathResource(@TempDir File testFolder)
throws Exception {
// execute // given
mojo.execute(); String key = "projectPropertyKey";
String value = "projectPropertyValue";
// assert String includedKey = "includedPropertyKey";
assertTrue(mojo.outputFile.exists()); String includedValue = "includedPropertyValue";
Properties storedProperties = getStoredProperties(testFolder); Properties projectProperties = new Properties();
assertEquals(1, storedProperties.size()); projectProperties.setProperty(key, value);
assertTrue(storedProperties.containsKey(includedKey)); projectProperties.setProperty(includedKey, includedValue);
assertEquals(includedValue, storedProperties.getProperty(includedKey)); doReturn(projectProperties).when(mavenProject).getProperties();
}
mojo
@Test .setIncludePropertyKeysFromFiles(
public void testExecuteIncludingPropertyKeysFromBlankLocation() { new String[] {
// given "/eu/dnetlib/maven/plugin/properties/included.properties"
String key = "projectPropertyKey"; });
String value = "projectPropertyValue";
String includedKey = "includedPropertyKey"; // execute
String includedValue = "includedPropertyValue"; mojo.execute();
Properties projectProperties = new Properties();
projectProperties.setProperty(key, value); // assert
projectProperties.setProperty(includedKey, includedValue); assertTrue(mojo.outputFile.exists());
doReturn(projectProperties).when(mavenProject).getProperties(); Properties storedProperties = getStoredProperties(testFolder);
assertEquals(1, storedProperties.size());
mojo.setIncludePropertyKeysFromFiles(new String[] {""}); assertTrue(storedProperties.containsKey(includedKey));
assertEquals(includedValue, storedProperties.getProperty(includedKey));
// execute }
Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute());
} @Test
public void testExecuteIncludingPropertyKeysFromBlankLocation() {
@Test // given
public void testExecuteIncludingPropertyKeysFromXmlFile(@TempDir File testFolder) throws Exception { String key = "projectPropertyKey";
// given String value = "projectPropertyValue";
String key = "projectPropertyKey"; String includedKey = "includedPropertyKey";
String value = "projectPropertyValue"; String includedValue = "includedPropertyValue";
String includedKey = "includedPropertyKey"; Properties projectProperties = new Properties();
String includedValue = "includedPropertyValue"; projectProperties.setProperty(key, value);
Properties projectProperties = new Properties(); projectProperties.setProperty(includedKey, includedValue);
projectProperties.setProperty(key, value); doReturn(projectProperties).when(mavenProject).getProperties();
projectProperties.setProperty(includedKey, includedValue);
doReturn(projectProperties).when(mavenProject).getProperties(); mojo.setIncludePropertyKeysFromFiles(new String[] {
""
File includedPropertiesFile = new File(testFolder, "included.xml"); });
Properties includedProperties = new Properties();
includedProperties.setProperty(includedKey, "irrelevantValue"); // execute
includedProperties.storeToXML(new FileOutputStream(includedPropertiesFile), null); Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute());
}
mojo.setIncludePropertyKeysFromFiles(new String[] {includedPropertiesFile.getAbsolutePath()});
@Test
// execute public void testExecuteIncludingPropertyKeysFromXmlFile(@TempDir File testFolder)
mojo.execute(); throws Exception {
// given
// assert String key = "projectPropertyKey";
assertTrue(mojo.outputFile.exists()); String value = "projectPropertyValue";
Properties storedProperties = getStoredProperties(testFolder); String includedKey = "includedPropertyKey";
assertEquals(1, storedProperties.size()); String includedValue = "includedPropertyValue";
assertTrue(storedProperties.containsKey(includedKey)); Properties projectProperties = new Properties();
assertEquals(includedValue, storedProperties.getProperty(includedKey)); projectProperties.setProperty(key, value);
} projectProperties.setProperty(includedKey, includedValue);
doReturn(projectProperties).when(mavenProject).getProperties();
@Test
public void testExecuteIncludingPropertyKeysFromInvalidXmlFile(@TempDir File testFolder) throws Exception { File includedPropertiesFile = new File(testFolder, "included.xml");
// given Properties includedProperties = new Properties();
String key = "projectPropertyKey"; includedProperties.setProperty(includedKey, "irrelevantValue");
String value = "projectPropertyValue"; includedProperties.storeToXML(new FileOutputStream(includedPropertiesFile), null);
String includedKey = "includedPropertyKey";
String includedValue = "includedPropertyValue"; mojo.setIncludePropertyKeysFromFiles(new String[] {
Properties projectProperties = new Properties(); includedPropertiesFile.getAbsolutePath()
projectProperties.setProperty(key, value); });
projectProperties.setProperty(includedKey, includedValue);
doReturn(projectProperties).when(mavenProject).getProperties(); // execute
mojo.execute();
File includedPropertiesFile = new File(testFolder, "included.xml");
Properties includedProperties = new Properties(); // assert
includedProperties.setProperty(includedKey, "irrelevantValue"); assertTrue(mojo.outputFile.exists());
includedProperties.store(new FileOutputStream(includedPropertiesFile), null); Properties storedProperties = getStoredProperties(testFolder);
assertEquals(1, storedProperties.size());
mojo.setIncludePropertyKeysFromFiles(new String[] {includedPropertiesFile.getAbsolutePath()}); assertTrue(storedProperties.containsKey(includedKey));
assertEquals(includedValue, storedProperties.getProperty(includedKey));
// execute }
Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute());
} @Test
public void testExecuteIncludingPropertyKeysFromInvalidXmlFile(@TempDir File testFolder)
@Test throws Exception {
public void testExecuteWithQuietModeOn(@TempDir File testFolder) throws Exception { // given
// given String key = "projectPropertyKey";
mojo.setQuiet(true); String value = "projectPropertyValue";
mojo.setIncludePropertyKeysFromFiles(new String[] {"invalid location"}); String includedKey = "includedPropertyKey";
String includedValue = "includedPropertyValue";
// execute Properties projectProperties = new Properties();
mojo.execute(); projectProperties.setProperty(key, value);
projectProperties.setProperty(includedKey, includedValue);
// assert doReturn(projectProperties).when(mavenProject).getProperties();
assertTrue(mojo.outputFile.exists());
Properties storedProperties = getStoredProperties(testFolder); File includedPropertiesFile = new File(testFolder, "included.xml");
assertEquals(0, storedProperties.size()); Properties includedProperties = new Properties();
} includedProperties.setProperty(includedKey, "irrelevantValue");
includedProperties.store(new FileOutputStream(includedPropertiesFile), null);
@Test
public void testExecuteIncludingPropertyKeysFromInvalidFile() { mojo.setIncludePropertyKeysFromFiles(new String[] {
// given includedPropertiesFile.getAbsolutePath()
mojo.setIncludePropertyKeysFromFiles(new String[] {"invalid location"}); });
// execute // execute
Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute()); Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute());
} }
@Test @Test
public void testExecuteWithEnvironmentProperties(@TempDir File testFolder) throws Exception { public void testExecuteWithQuietModeOn(@TempDir File testFolder) throws Exception {
// given // given
mojo.setIncludeEnvironmentVariables(true); mojo.setQuiet(true);
mojo.setIncludePropertyKeysFromFiles(new String[] {
// execute "invalid location"
mojo.execute(); });
// assert // execute
assertTrue(mojo.outputFile.exists()); mojo.execute();
Properties storedProperties = getStoredProperties(testFolder);
assertTrue(storedProperties.size() > 0); // assert
for (Object currentKey : storedProperties.keySet()) { assertTrue(mojo.outputFile.exists());
assertTrue(((String)currentKey).startsWith(PROPERTY_PREFIX_ENV)); Properties storedProperties = getStoredProperties(testFolder);
} assertEquals(0, storedProperties.size());
} }
@Test @Test
public void testExecuteWithSystemProperties(@TempDir File testFolder) throws Exception { public void testExecuteIncludingPropertyKeysFromInvalidFile() {
// given // given
String key = "systemPropertyKey"; mojo.setIncludePropertyKeysFromFiles(new String[] {
String value = "systemPropertyValue"; "invalid location"
System.setProperty(key, value); });
mojo.setIncludeSystemProperties(true);
// execute
// execute Assertions.assertThrows(MojoExecutionException.class, () -> mojo.execute());
mojo.execute(); }
// assert @Test
assertTrue(mojo.outputFile.exists()); public void testExecuteWithEnvironmentProperties(@TempDir File testFolder) throws Exception {
Properties storedProperties = getStoredProperties(testFolder); // given
assertTrue(storedProperties.size() > 0); mojo.setIncludeEnvironmentVariables(true);
assertTrue(storedProperties.containsKey(key));
assertEquals(value, storedProperties.getProperty(key)); // execute
} mojo.execute();
@Test // assert
public void testExecuteWithSystemPropertiesAndEscapeChars(@TempDir File testFolder) throws Exception { assertTrue(mojo.outputFile.exists());
// given Properties storedProperties = getStoredProperties(testFolder);
String key = "systemPropertyKey "; assertTrue(storedProperties.size() > 0);
String value = "systemPropertyValue"; for (Object currentKey : storedProperties.keySet()) {
System.setProperty(key, value); assertTrue(((String) currentKey).startsWith(PROPERTY_PREFIX_ENV));
mojo.setIncludeSystemProperties(true); }
String escapeChars = "cr,lf,tab,|"; }
mojo.setEscapeChars(escapeChars);
@Test
// execute public void testExecuteWithSystemProperties(@TempDir File testFolder) throws Exception {
mojo.execute(); // given
String key = "systemPropertyKey";
// assert String value = "systemPropertyValue";
assertTrue(mojo.outputFile.exists()); System.setProperty(key, value);
Properties storedProperties = getStoredProperties(testFolder); mojo.setIncludeSystemProperties(true);
assertTrue(storedProperties.size() > 0);
assertFalse(storedProperties.containsKey(key)); // execute
assertTrue(storedProperties.containsKey(key.trim())); mojo.execute();
assertEquals(value, storedProperties.getProperty(key.trim()));
} // assert
assertTrue(mojo.outputFile.exists());
// ----------------------------------- PRIVATE ------------------------------------------- Properties storedProperties = getStoredProperties(testFolder);
assertTrue(storedProperties.size() > 0);
private File getPropertiesFileLocation(File testFolder) { assertTrue(storedProperties.containsKey(key));
return new File(testFolder, "test.properties"); assertEquals(value, storedProperties.getProperty(key));
} }
private Properties getStoredProperties(File testFolder) throws FileNotFoundException, IOException { @Test
Properties properties = new Properties(); public void testExecuteWithSystemPropertiesAndEscapeChars(@TempDir File testFolder)
properties.load(new FileInputStream(getPropertiesFileLocation(testFolder))); throws Exception {
return properties; // given
} String key = "systemPropertyKey ";
String value = "systemPropertyValue";
System.setProperty(key, value);
mojo.setIncludeSystemProperties(true);
String escapeChars = "cr,lf,tab,|";
mojo.setEscapeChars(escapeChars);
// execute
mojo.execute();
// assert
assertTrue(mojo.outputFile.exists());
Properties storedProperties = getStoredProperties(testFolder);
assertTrue(storedProperties.size() > 0);
assertFalse(storedProperties.containsKey(key));
assertTrue(storedProperties.containsKey(key.trim()));
assertEquals(value, storedProperties.getProperty(key.trim()));
}
// ----------------------------------- PRIVATE -------------------------------------------
private File getPropertiesFileLocation(File testFolder) {
return new File(testFolder, "test.properties");
}
private Properties getStoredProperties(File testFolder)
throws IOException {
Properties properties = new Properties();
properties.load(new FileInputStream(getPropertiesFileLocation(testFolder)));
return properties;
}
} }

50
dhp-build/dhp-code-style/pom.xml Normal file
View File

@ -0,0 +1,50 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>eu.dnetlib.dhp</groupId>
<artifactId>dhp-code-style</artifactId>
<version>1.1.7-SNAPSHOT</version>
<packaging>jar</packaging>
<description>This module contains resources supporting common code style conventions</description>
<distributionManagement>
<snapshotRepository>
<id>dnet45-snapshots</id>
<name>DNet45 Snapshots</name>
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/dnet45-snapshots</url>
<layout>default</layout>
</snapshotRepository>
<repository>
<id>dnet45-releases</id>
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/dnet45-releases</url>
</repository>
</distributionManagement>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
</plugins>
</pluginManagement>
</build>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
</project>

252
dhp-build/dhp-code-style/src/main/resources/eclipse/formatter_aosp.xml Normal file
View File

@ -0,0 +1,252 @@
<?xml version="1.0" encoding="UTF-8"?>
<profiles version="10">
<profile kind="CodeFormatterProfile" name="Android" version="10">
<setting id="org.eclipse.jdt.core.formatter.align_type_members_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_enum_constant" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_explicit_constructor_call" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_method_invocation" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_qualified_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_assignment" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_binary_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_compact_if" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_conditional_expression" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_enum_constants" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_expressions_in_array_initializer" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_multiple_fields" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_method_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_selector_in_method_invocation" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superclass_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_enum_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_method_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_imports" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_package" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_field" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_first_class_body_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_imports" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_member_type" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_method" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_new_chunk" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_package" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_type_declarations" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_annotation_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_anonymous_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_array_initializer" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block_in_case" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_constructor_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_constant" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_method_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_switch" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.comment.clear_blank_lines" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_header" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_html" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_source_code" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_parameter_description" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_root_tags" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_before_root_tags" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_for_parameter" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.line_length" value="120"/>
<setting id="org.eclipse.jdt.core.formatter.compact_else_if" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation" value="2"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation_for_array_initializer" value="2"/>
<setting id="org.eclipse.jdt.core.formatter.format_guardian_clause_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_annotation_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_constant_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_type_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_breaks_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_empty_lines" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_block" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_body" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_switch" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indentation.size" value="4"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_at_end_of_file_if_missing" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_catch_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_closing_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_else_in_if_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_finally_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_while_in_do_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_annotation_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_anonymous_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_constant" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_method_body" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation_type_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_binary_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_paren_in_cast" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_labeled_statement" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_allocation_expression" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_annotation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_constant_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_explicitconstructorcall_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_increments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_inits" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_invocation_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_field_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_local_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_parameterized_type_reference" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_superinterfaces" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_ellipsis" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_at_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_binary_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_case" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_default" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_labeled_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_constant_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_explicitconstructorcall_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_increments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_inits" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_invocation_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_field_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_local_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_superinterfaces" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_ellipsis" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_anonymous_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_constructor_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_constant" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_method_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_switch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_catch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_if" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_switch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_synchronized" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_while" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_return" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_brackets_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_braces_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_brackets_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_else_statement_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_empty_array_initializer_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_imple_if_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_then_statement_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.lineSplit" value="100"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_beginning_of_method_body" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_empty_lines_to_preserve" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.put_empty_statement_on_new_line" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.char" value="tab"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.size" value="4"/>
<setting id="org.eclipse.jdt.core.formatter.use_tabs_only_for_leading_indentations" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.join_wrapped_lines" value="false"/>
</profile>
</profiles>

727
dhp-build/dhp-code-style/src/main/resources/eclipse/formatter_dnet.xml Normal file
View File

@ -0,0 +1,727 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<profiles version="18">
<profile kind="CodeFormatterProfile" name="Android_custom" version="18">
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_ellipsis" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_at_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_for_statment" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.comment.new_lines_at_block_boundaries" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_logical_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_for_parameter" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_package" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_method_invocation" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_imports" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_before_root_tags" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_switch_statement" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_javadoc_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indentation.size" value="4"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_enum_constant_declaration" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_increments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_arrow_in_switch_default" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_inits" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.align_with_spaces" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.disabling_tag" value="@formatter:off"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_before_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_switch_case_expressions" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_enum_constants" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_imports" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_end_of_method_body" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_package" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_local_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_if_while_statement" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_enum_constant" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_root_tags" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_or_operator_multicatch" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.enabling_tag" value="@formatter:on"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.count_line_length_from_starting_position" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_return" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_method_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_arrow_in_switch_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_multiplicative_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.keep_then_statement_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_field" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_explicitconstructorcall_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_type_declarations" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_method" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameterized_type_references" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_anonymous_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_logical_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_annotation_declaration_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_enum_constant" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_multiplicative_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.never_indent_line_comments_on_first_column" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_inits" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_block" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_anonymous_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_invocation_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_switch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.align_tags_descriptions_grouped" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.comment.line_length" value="120"/>
<setting id="org.eclipse.jdt.core.formatter.use_on_off_tags" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_method_body_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_brackets_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_loop_body_block_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_constant" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.preserve_white_space_between_code_and_line_comments" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_local_variable" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_method_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_abstract_method" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.keep_enum_constant_declaration_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.align_variable_declarations_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_union_type_in_multicatch" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_type_declaration_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_beginning_of_method_body" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_else_statement_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_catch_clause" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_additive_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_parameterized_type_reference" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_field_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_explicit_constructor_call" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_relational_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_multiplicative_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.keep_anonymous_type_declaration_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_switch_case_expressions" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_shift_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_annotation_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_superinterfaces" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_default" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_constructor_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_lambda_body" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_end_of_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.compact_else_if" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_catch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_bitwise_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.put_empty_statement_on_new_line" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_type_parameters" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_invocation_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_method_invocation" value="32"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_compact_loops" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_block_comment" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_catch_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_try" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_for_body_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_at_end_of_file_if_missing" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_javadoc_comment" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_relational_operator" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_expressions_in_array_initializer" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.format_line_comment_starting_on_first_column" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_empty_lines_to_preserve" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_annotation" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_ellipsis" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_additive_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_try_resources" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_string_concatenation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_line_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_labeled_statement" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.text_block_indentation" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.align_type_members_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_assignment" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_module_statements" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_type_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_after_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.comment.align_tags_names_descriptions" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.keep_if_then_body_block_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_first_class_body_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_conditional_expression" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_closing_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.format_guardian_clause_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_if" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.align_assignment_statements_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_type" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block_in_case" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_arrow_in_switch_default" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_between_different_tags" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_conditional_expression_chain" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_header" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_additive_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_while" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_method_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.join_wrapped_lines" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_conditional_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_shift_operator" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.align_fields_grouping_blank_lines" value="2147483647"/>
<setting id="org.eclipse.jdt.core.formatter.comment.new_lines_at_javadoc_boundaries" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_bitwise_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_annotation_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_resources_in_try" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.use_tabs_only_for_leading_indentations" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_try_clause" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_selector_in_method_invocation" value="48"/>
<setting id="org.eclipse.jdt.core.formatter.never_indent_block_comments_on_first_column" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_code_block_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.size" value="4"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_bitwise_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_allocation_expression" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_source_code" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_try" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_try_resources" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_field" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation_for_array_initializer" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_method" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superclass_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_enum_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_throw" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_assignment_operator" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_labeled_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_not_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_switch" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_superinterfaces" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_type_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_html" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation_type_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_method_delcaration" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_compact_if" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.keep_lambda_body_block_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.indent_empty_lines" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_type_arguments" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_annotation" value="48"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_empty_array_initializer_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_switch" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_else_in_if_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_new_chunk" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_label" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_arrow_in_switch_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_member_type" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_while_in_do_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_logical_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_qualified_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_breaks_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_bitwise_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_relational_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_try" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_block_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_lambda_arrow" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_tag_description" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_imple_if_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_method_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_brackets_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_string_concatenation" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_last_class_body_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_body" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_multiple_fields" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_constant_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_while_body_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_array_initializer" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_logical_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_shift_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_method_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_statement_group_in_switch" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_annotation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_constant_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_lambda_declaration" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_shift_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_braces_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_case" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_local_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_do_while_body_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_enum_declaration_on_one_line" value="one_line_never"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_outer_expressions_when_nested" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_paren_in_cast" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_constant" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_multiplicative_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_package" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_synchronized" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_increments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_expressions_in_for_loop_header" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_additive_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_getter_setter_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_explicitconstructorcall_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_constant_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_string_concatenation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_lambda_arrow" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_constructor_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.join_lines_in_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_parameter_description" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_beginning_of_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_finally_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.char" value="tab"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_relational_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_field_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_string_concatenation" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_import_groups" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.lineSplit" value="120"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_switch" value="insert"/>
</profile>
<profile kind="CodeFormatterProfile" name="Dnet" version="18">
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_ellipsis" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_at_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_for_statment" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.comment.new_lines_at_block_boundaries" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_logical_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_for_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_package" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_method_invocation" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_imports" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_before_root_tags" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_switch_statement" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_javadoc_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indentation.size" value="4"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_enum_constant_declaration" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_increments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_arrow_in_switch_default" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_inits" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.align_with_spaces" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.disabling_tag" value="@formatter:off"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_before_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_switch_case_expressions" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_enum_constants" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_imports" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_end_of_method_body" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_package" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_local_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_if_while_statement" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_enum_constant" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_root_tags" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_or_operator_multicatch" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.enabling_tag" value="@formatter:on"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.count_line_length_from_starting_position" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_return" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_method_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_parameter" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_arrow_in_switch_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_multiplicative_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.keep_then_statement_on_same_line" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_field" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_explicitconstructorcall_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_type_declarations" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_method" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameterized_type_references" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_anonymous_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_logical_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_annotation_declaration_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_enum_constant" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_multiplicative_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.never_indent_line_comments_on_first_column" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_inits" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_block" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_anonymous_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_invocation_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_switch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.align_tags_descriptions_grouped" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.comment.line_length" value="140"/>
<setting id="org.eclipse.jdt.core.formatter.use_on_off_tags" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_method_body_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_brackets_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_loop_body_block_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_constant" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.preserve_white_space_between_code_and_line_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_local_variable" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_method_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_abstract_method" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.keep_enum_constant_declaration_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.align_variable_declarations_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_union_type_in_multicatch" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_type_declaration_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_beginning_of_method_body" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_else_statement_on_same_line" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_catch_clause" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_additive_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_parameterized_type_reference" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_field_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_explicit_constructor_call" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_relational_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_multiplicative_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.keep_anonymous_type_declaration_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_switch_case_expressions" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_shift_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_annotation_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_superinterfaces" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_default" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_constructor_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_lambda_body" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_end_of_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.compact_else_if" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_catch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_bitwise_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.put_empty_statement_on_new_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_type_parameters" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_invocation_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_method_invocation" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_compact_loops" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_block_comment" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_catch_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_try" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_for_body_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_at_end_of_file_if_missing" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_javadoc_comment" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_relational_operator" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_expressions_in_array_initializer" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.format_line_comment_starting_on_first_column" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_empty_lines_to_preserve" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_annotation" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_ellipsis" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_additive_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_try_resources" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_string_concatenation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_line_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_labeled_statement" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.text_block_indentation" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.align_type_members_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_assignment" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_module_statements" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_type_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_after_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.comment.align_tags_names_descriptions" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.keep_if_then_body_block_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_first_class_body_declaration" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_conditional_expression" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_closing_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.format_guardian_clause_on_one_line" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_if" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.align_assignment_statements_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_type" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block_in_case" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_arrow_in_switch_default" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_between_different_tags" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_conditional_expression_chain" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_additive_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_while" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_method_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.join_wrapped_lines" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_conditional_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_shift_operator" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.align_fields_grouping_blank_lines" value="2147483647"/>
<setting id="org.eclipse.jdt.core.formatter.comment.new_lines_at_javadoc_boundaries" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_bitwise_operator" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_annotation_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_resources_in_try" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.use_tabs_only_for_leading_indentations" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_try_clause" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_selector_in_method_invocation" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.never_indent_block_comments_on_first_column" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_code_block_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.size" value="4"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_bitwise_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_allocation_expression" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_source_code" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_try" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_try_resources" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_field" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation_for_array_initializer" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_method" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superclass_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_enum_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_throw" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_assignment_operator" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_labeled_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_not_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_switch" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_superinterfaces" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_type_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_html" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation_type_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_method_delcaration" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_compact_if" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.keep_lambda_body_block_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.indent_empty_lines" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_type_arguments" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_annotation" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_empty_array_initializer_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_switch" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_else_in_if_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_new_chunk" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_label" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_arrow_in_switch_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_member_type" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_while_in_do_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_logical_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_qualified_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_breaks_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_bitwise_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_relational_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_try" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_block_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_lambda_arrow" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_tag_description" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_imple_if_on_one_line" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_method_declaration" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_brackets_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_string_concatenation" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_last_class_body_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_body" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_multiple_fields" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_constant_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_while_body_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_array_initializer" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_logical_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_shift_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_method_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_statement_group_in_switch" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_annotation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_constant_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.parentheses_positions_in_lambda_declaration" value="common_lines"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_shift_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_braces_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_case" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_local_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_do_while_body_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_enum_declaration_on_one_line" value="one_line_if_empty"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_outer_expressions_when_nested" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_paren_in_cast" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_constant" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_multiplicative_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_package" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_synchronized" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_increments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_expressions_in_for_loop_header" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_additive_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.keep_simple_getter_setter_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_explicitconstructorcall_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_constant_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_string_concatenation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_lambda_arrow" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_constructor_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.join_lines_in_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_parameter_description" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_beginning_of_code_block" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_finally_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.char" value="tab"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_relational_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_field_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_string_concatenation" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_import_groups" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.lineSplit" value="160"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_switch" value="insert"/>
</profile>
</profiles>

337
dhp-build/dhp-code-style/src/main/resources/eclipse/formatter_google.xml Normal file
View File

@ -0,0 +1,337 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<profiles version="13">
<profile kind="CodeFormatterProfile" name="GoogleStyle" version="13">
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_before_root_tags" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.disabling_tag" value="@formatter:off"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_annotation" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_anonymous_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_case" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_brace_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.new_lines_at_block_boundaries" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_cascading_method_invocation_with_arguments.count_dependent" value="16|-1|16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_annotation_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_field" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.use_on_off_tags" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_prefer_two_fragments" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_else_in_if_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_else_statement_on_same_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_ellipsis" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.insert_new_line_for_parameter" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_comment_inline_tags" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_breaks_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_local_variable_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_multiple_fields" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_parameter" value="1040"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_expressions_in_array_initializer" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_type.count_dependent" value="1585|-1|1585"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_conditional_expression" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_multiple_fields.count_dependent" value="16|-1|16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_binary_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_array_initializer" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_finally_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_local_variable" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_catch_in_try_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_while" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_package" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_qualified_allocation_expression.count_dependent" value="16|4|80"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_method_declaration.count_dependent" value="16|4|48"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation" value="2"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_enum_declaration.count_dependent" value="16|4|49"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_method_invocation" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_superinterfaces" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_new_chunk" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_binary_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_package" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_cascading_method_invocation_with_arguments" value="16"/>
<setting id="org.eclipse.jdt.core.compiler.source" value="1.7"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_constructor_declaration.count_dependent" value="16|4|48"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_constant_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_line_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.join_wrapped_lines" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_explicit_constructor_call" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_non_simple_local_variable_annotation" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_invocation_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.align_type_members_on_columns" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_member_type" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_enum_constants.count_dependent" value="16|5|48"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_method_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_selector_in_method_invocation" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_case" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_parameter_description" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_switch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_block_comment" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.lineSplit" value="100"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_if" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_selector_in_method_invocation.count_dependent" value="16|4|48"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_brackets_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_explicitconstructorcall_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_constructor_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_first_class_body_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_method" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indentation.size" value="4"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.enabling_tag" value="@formatter:on"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_package" value="1585"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superclass_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_assignment" value="16"/>
<setting id="org.eclipse.jdt.core.compiler.problem.assertIdentifier" value="error"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.char" value="space"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_try_resources" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_prefix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_body" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_method" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_outer_expressions_when_nested" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_non_simple_type_annotation" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.format_guardian_clause_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_field_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_labeled_statement" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_annotation_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_method_body" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_method_declaration" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_try" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_constant" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation_type_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_switch" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_return" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_question_in_wildcard" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_try" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.preserve_white_space_between_code_and_line_comments" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_throw" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.compiler.problem.enumIdentifier" value="error"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_generic_type_arguments" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_switch" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment_new_line_at_start_of_html_paragraph" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_ellipsis" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comment_prefix" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_inits" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_method_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.compact_else_if" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_non_simple_parameter_annotation" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_or_operator_multicatch" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_increments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_method" value="1585"/>
<setting id="org.eclipse.jdt.core.formatter.format_line_comment_starting_on_first_column" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_field" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.indent_root_tags" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_constant" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_union_type_in_multicatch" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_explicitconstructorcall_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_switch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_superinterfaces" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.tabulation.size" value="2"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_opening_brace_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_brace_in_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_throws" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_if" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_method_invocation.count_dependent" value="16|5|80"/>
<setting id="org.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_javadoc_comment" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_parameter.count_dependent" value="1040|-1|1040"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_constructor_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_assignment_operator" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_package.count_dependent" value="1585|-1|1585"/>
<setting id="org.eclipse.jdt.core.formatter.indent_empty_lines" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_paren_in_cast" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.force_if_else_statement_brace" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_block_in_case" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_empty_lines_to_preserve" value="3"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_non_simple_package_annotation" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_qualified_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_annotation.count_dependent" value="16|-1|16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_type" value="1585"/>
<setting id="org.eclipse.jdt.core.compiler.compliance" value="1.7"/>
<setting id="org.eclipse.jdt.core.formatter.continuation_indentation_for_array_initializer" value="2"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_brackets_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_at_in_annotation_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_cast" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_unary_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_new_anonymous_class" value="20"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_local_variable.count_dependent" value="1585|-1|1585"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_anonymous_type_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.keep_empty_array_initializer_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_declaration" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_field.count_dependent" value="1585|-1|1585"/>
<setting id="org.eclipse.jdt.core.formatter.keep_imple_if_on_one_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_parameters" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_labeled_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_at_end_of_file_if_missing" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_constructor_declaration.count_dependent" value="16|5|80"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_type_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_enum_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_binary_expression" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_while" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_type" value="insert"/>
<setting id="org.eclipse.jdt.core.compiler.codegen.inlineJsrBytecode" value="enabled"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_try" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.put_empty_statement_on_new_line" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_label" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_parameter" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_javadoc_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_enum_constant" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_before_while_in_do_statement" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_enum_constant.count_dependent" value="16|-1|16"/>
<setting id="org.eclipse.jdt.core.formatter.comment.line_length" value="100"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_package" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_import_groups" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_constant_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_constructor_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.number_of_blank_lines_at_beginning_of_method_body" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_type_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation_type_member_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_before_binary_operator" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_between_type_declarations" value="2"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_synchronized" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_enum_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.indent_statements_compare_to_block" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.join_lines_in_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_question_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_field_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_compact_if" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_inits" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_cases" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_default" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_and_in_type_parameter" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_constructor_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_before_imports" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_field" value="1585"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_html" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_method_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_parameters" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_allocation_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_anonymous_type_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_colon_in_conditional" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_expressions_in_array_initializer.count_dependent" value="16|5|80"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_postfix_operator" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_source_code" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_synchronized" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_allocation_expression" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_throws" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_method_declaration" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_brace_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.compiler.codegen.targetPlatform" value="1.7"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_resources_in_try" value="80"/>
<setting id="org.eclipse.jdt.core.formatter.use_tabs_only_for_leading_indentations" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_annotation" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.comment.format_block_comments" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_enum_constant" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_enum_constants" value="0"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_annotation_declaration_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_new_line_in_empty_block" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_parenthesized_expression" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_catch" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_local_declarations" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_type_declaration.count_dependent" value="16|4|48"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_switch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_increments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_method.count_dependent" value="1585|-1|1585"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_invocation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_colon_in_assert" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.brace_position_for_type_declaration" value="end_of_line"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_array_initializer" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_between_empty_braces_in_array_initializer" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_binary_expression.count_dependent" value="16|-1|16"/>
<setting id="org.eclipse.jdt.core.formatter.wrap_non_simple_member_annotation" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_annotations_on_local_variable" value="1585"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_declaration" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_for" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_explicit_constructor_call.count_dependent" value="16|5|80"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_catch" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_parameterized_type_reference" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_field_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_annotation" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_generic_type_arguments.count_dependent" value="16|-1|16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_parameterized_type_reference" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression.count_dependent" value="16|5|80"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_invocation_arguments" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_parameters_in_method_declaration.count_dependent" value="16|5|80"/>
<setting id="org.eclipse.jdt.core.formatter.comment.new_lines_at_javadoc_boundaries" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.blank_lines_after_imports" value="1"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_local_declarations" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_constant_header" value="true"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_for" value="insert"/>
<setting id="org.eclipse.jdt.core.formatter.never_indent_line_comments_on_first_column" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_try_resources" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.alignment_for_for_statement" value="16"/>
<setting id="org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_arguments" value="do not insert"/>
<setting id="org.eclipse.jdt.core.formatter.never_indent_block_comments_on_first_column" value="false"/>
<setting id="org.eclipse.jdt.core.formatter.keep_then_statement_on_same_line" value="false"/>
</profile>
</profiles>

4
dhp-build/pom.xml
View File

@ -8,7 +8,11 @@
</parent> </parent>
<artifactId>dhp-build</artifactId> <artifactId>dhp-build</artifactId>
<packaging>pom</packaging> <packaging>pom</packaging>
<description>This module is a container for the build tools used in dnet-hadoop</description>
<modules> <modules>
<module>dhp-code-style</module>
<module>dhp-build-assembly-resources</module> <module>dhp-build-assembly-resources</module>
<module>dhp-build-properties-maven-plugin</module> <module>dhp-build-properties-maven-plugin</module>
</modules> </modules>

2
dhp-common/pom.xml
View File

@ -12,6 +12,8 @@
<artifactId>dhp-common</artifactId> <artifactId>dhp-common</artifactId>
<packaging>jar</packaging> <packaging>jar</packaging>
<description>This module contains common utilities meant to be used across the dnet-hadoop submodules</description>
<dependencies> <dependencies>
<dependency> <dependency>

3
dhp-common/src/main/java/eu/dnetlib/collector/worker/model/ApiDescriptor.java
View File

@ -1,9 +1,9 @@
package eu.dnetlib.collector.worker.model; package eu.dnetlib.collector.worker.model;
import java.util.HashMap; import java.util.HashMap;
import java.util.Map; import java.util.Map;
public class ApiDescriptor { public class ApiDescriptor {
private String id; private String id;
@ -45,5 +45,4 @@ public class ApiDescriptor {
public void setProtocol(final String protocol) { public void setProtocol(final String protocol) {
this.protocol = protocol; this.protocol = protocol;
} }
} }

22
dhp-common/src/main/java/eu/dnetlib/data/mdstore/manager/common/model/MDStore.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.data.mdstore.manager.common.model; package eu.dnetlib.data.mdstore.manager.common.model;
import java.io.Serializable; import java.io.Serializable;
@ -12,9 +13,7 @@ import javax.persistence.Table;
@Table(name = "mdstores") @Table(name = "mdstores")
public class MDStore implements Serializable { public class MDStore implements Serializable {
/** /** */
*
*/
private static final long serialVersionUID = 3160530489149700055L; private static final long serialVersionUID = 3160530489149700055L;
@Id @Id
@ -95,16 +94,18 @@ public class MDStore implements Serializable {
this.apiId = apiId; this.apiId = apiId;
} }
public static MDStore newInstance(final String format, final String layout, final String interpretation) { public static MDStore newInstance(
final String format, final String layout, final String interpretation) {
return newInstance(format, layout, interpretation, null, null, null); return newInstance(format, layout, interpretation, null, null, null);
} }
public static MDStore newInstance(final String format, public static MDStore newInstance(
final String layout, final String format,
final String interpretation, final String layout,
final String dsName, final String interpretation,
final String dsId, final String dsName,
final String apiId) { final String dsId,
final String apiId) {
final MDStore md = new MDStore(); final MDStore md = new MDStore();
md.setId("md-" + UUID.randomUUID()); md.setId("md-" + UUID.randomUUID());
md.setFormat(format); md.setFormat(format);
@ -115,5 +116,4 @@ public class MDStore implements Serializable {
md.setApiId(apiId); md.setApiId(apiId);
return md; return md;
} }
} }

5
dhp-common/src/main/java/eu/dnetlib/data/mdstore/manager/common/model/MDStoreCurrentVersion.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.data.mdstore.manager.common.model; package eu.dnetlib.data.mdstore.manager.common.model;
import java.io.Serializable; import java.io.Serializable;
@ -11,9 +12,7 @@ import javax.persistence.Table;
@Table(name = "mdstore_current_versions") @Table(name = "mdstore_current_versions")
public class MDStoreCurrentVersion implements Serializable { public class MDStoreCurrentVersion implements Serializable {
/** /** */
*
*/
private static final long serialVersionUID = -4757725888593745773L; private static final long serialVersionUID = -4757725888593745773L;
@Id @Id

6
dhp-common/src/main/java/eu/dnetlib/data/mdstore/manager/common/model/MDStoreVersion.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.data.mdstore.manager.common.model; package eu.dnetlib.data.mdstore.manager.common.model;
import java.io.Serializable; import java.io.Serializable;
@ -14,9 +15,7 @@ import javax.persistence.TemporalType;
@Table(name = "mdstore_versions") @Table(name = "mdstore_versions")
public class MDStoreVersion implements Serializable { public class MDStoreVersion implements Serializable {
/** /** */
*
*/
private static final long serialVersionUID = -4763494442274298339L; private static final long serialVersionUID = -4763494442274298339L;
@Id @Id
@ -97,5 +96,4 @@ public class MDStoreVersion implements Serializable {
public void setSize(final long size) { public void setSize(final long size) {
this.size = size; this.size = size;
} }
} }

6
dhp-common/src/main/java/eu/dnetlib/data/mdstore/manager/common/model/MDStoreWithInfo.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.data.mdstore.manager.common.model; package eu.dnetlib.data.mdstore.manager.common.model;
import java.io.Serializable; import java.io.Serializable;
@ -14,9 +15,7 @@ import javax.persistence.TemporalType;
@Table(name = "mdstores_with_info") @Table(name = "mdstores_with_info")
public class MDStoreWithInfo implements Serializable { public class MDStoreWithInfo implements Serializable {
/** /** */
*
*/
private static final long serialVersionUID = -8445784770687571492L; private static final long serialVersionUID = -8445784770687571492L;
@Id @Id
@ -141,5 +140,4 @@ public class MDStoreWithInfo implements Serializable {
public void setNumberOfVersions(final long numberOfVersions) { public void setNumberOfVersions(final long numberOfVersions) {
this.numberOfVersions = numberOfVersions; this.numberOfVersions = numberOfVersions;
} }
} }

135
dhp-common/src/main/java/eu/dnetlib/dhp/application/ArgumentApplicationParser.java
View File

@ -1,9 +1,5 @@
package eu.dnetlib.dhp.application;
import com.fasterxml.jackson.databind.ObjectMapper; package eu.dnetlib.dhp.application;
import org.apache.commons.cli.*;
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.io.IOUtils;
import java.io.ByteArrayInputStream; import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream; import java.io.ByteArrayOutputStream;
@ -12,76 +8,91 @@ import java.io.StringWriter;
import java.util.*; import java.util.*;
import java.util.zip.GZIPInputStream; import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream; import java.util.zip.GZIPOutputStream;
import java.util.zip.Inflater;
import org.apache.commons.cli.*;
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.io.IOUtils;
import com.fasterxml.jackson.databind.ObjectMapper;
public class ArgumentApplicationParser implements Serializable { public class ArgumentApplicationParser implements Serializable {
private final Options options = new Options(); private final Options options = new Options();
private final Map<String, String> objectMap = new HashMap<>(); private final Map<String, String> objectMap = new HashMap<>();
private final List<String> compressedValues = new ArrayList<>(); private final List<String> compressedValues = new ArrayList<>();
public ArgumentApplicationParser(final String json_configuration) throws Exception { public ArgumentApplicationParser(final String json_configuration) throws Exception {
final ObjectMapper mapper = new ObjectMapper(); final ObjectMapper mapper = new ObjectMapper();
final OptionsParameter[] configuration = mapper.readValue(json_configuration, OptionsParameter[].class); final OptionsParameter[] configuration = mapper.readValue(json_configuration, OptionsParameter[].class);
createOptionMap(configuration); createOptionMap(configuration);
} }
public ArgumentApplicationParser(final OptionsParameter[] configuration) { public ArgumentApplicationParser(final OptionsParameter[] configuration) {
createOptionMap(configuration); createOptionMap(configuration);
} }
private void createOptionMap(final OptionsParameter[] configuration) { private void createOptionMap(final OptionsParameter[] configuration) {
Arrays.stream(configuration).map(conf -> { Arrays
final Option o = new Option(conf.getParamName(), true, conf.getParamDescription()); .stream(configuration)
o.setLongOpt(conf.getParamLongName()); .map(
o.setRequired(conf.isParamRequired()); conf -> {
if (conf.isCompressed()) { final Option o = new Option(conf.getParamName(), true, conf.getParamDescription());
compressedValues.add(conf.getParamLongName()); o.setLongOpt(conf.getParamLongName());
} o.setRequired(conf.isParamRequired());
return o; if (conf.isCompressed()) {
}).forEach(options::addOption); compressedValues.add(conf.getParamLongName());
}
return o;
})
.forEach(options::addOption);
// HelpFormatter formatter = new HelpFormatter(); // HelpFormatter formatter = new HelpFormatter();
// formatter.printHelp("myapp", null, options, null, true); // formatter.printHelp("myapp", null, options, null, true);
}
} public static String decompressValue(final String abstractCompressed) {
try {
byte[] byteArray = Base64.decodeBase64(abstractCompressed.getBytes());
GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(byteArray));
final StringWriter stringWriter = new StringWriter();
IOUtils.copy(gis, stringWriter);
return stringWriter.toString();
} catch (Throwable e) {
System.out.println("Wrong value to decompress:" + abstractCompressed);
throw new RuntimeException(e);
}
}
public static String compressArgument(final String value) throws Exception {
ByteArrayOutputStream out = new ByteArrayOutputStream();
GZIPOutputStream gzip = new GZIPOutputStream(out);
gzip.write(value.getBytes());
gzip.close();
return java.util.Base64.getEncoder().encodeToString(out.toByteArray());
}
public static String decompressValue(final String abstractCompressed) { public void parseArgument(final String[] args) throws Exception {
try { CommandLineParser parser = new BasicParser();
byte[] byteArray = Base64.decodeBase64(abstractCompressed.getBytes()); CommandLine cmd = parser.parse(options, args);
GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(byteArray)); Arrays
final StringWriter stringWriter = new StringWriter(); .stream(cmd.getOptions())
IOUtils.copy(gis, stringWriter); .forEach(
return stringWriter.toString(); it -> objectMap
} catch (Throwable e) { .put(
System.out.println("Wrong value to decompress:" + abstractCompressed); it.getLongOpt(),
throw new RuntimeException(e); compressedValues.contains(it.getLongOpt())
} ? decompressValue(it.getValue())
} : it.getValue()));
}
public static String compressArgument(final String value) throws Exception{ public String get(final String key) {
ByteArrayOutputStream out = new ByteArrayOutputStream(); return objectMap.get(key);
GZIPOutputStream gzip = new GZIPOutputStream(out); }
gzip.write(value.getBytes());
gzip.close();
return java.util.Base64.getEncoder().encodeToString(out.toByteArray());
}
public void parseArgument(final String[] args) throws Exception { public Map<String, String> getObjectMap() {
CommandLineParser parser = new BasicParser(); return objectMap;
CommandLine cmd = parser.parse(options, args); }
Arrays.stream(cmd.getOptions()).forEach(it -> objectMap.put(it.getLongOpt(), compressedValues.contains(it.getLongOpt())? decompressValue(it.getValue()): it.getValue()));
}
public String get(final String key) {
return objectMap.get(key);
}
public Map<String, String> getObjectMap() {
return objectMap;
}
} }

52
dhp-common/src/main/java/eu/dnetlib/dhp/application/OptionsParameter.java
View File

@ -1,38 +1,38 @@
package eu.dnetlib.dhp.application;
package eu.dnetlib.dhp.application;
public class OptionsParameter { public class OptionsParameter {
private String paramName; private String paramName;
private String paramLongName; private String paramLongName;
private String paramDescription; private String paramDescription;
private boolean paramRequired; private boolean paramRequired;
private boolean compressed; private boolean compressed;
public OptionsParameter() { public OptionsParameter() {
} }
public String getParamName() { public String getParamName() {
return paramName; return paramName;
} }
public String getParamLongName() { public String getParamLongName() {
return paramLongName; return paramLongName;
} }
public String getParamDescription() { public String getParamDescription() {
return paramDescription; return paramDescription;
} }
public boolean isParamRequired() { public boolean isParamRequired() {
return paramRequired; return paramRequired;
} }
public boolean isCompressed() { public boolean isCompressed() {
return compressed; return compressed;
} }
public void setCompressed(boolean compressed) { public void setCompressed(boolean compressed) {
this.compressed = compressed; this.compressed = compressed;
} }
} }

88
dhp-common/src/main/java/eu/dnetlib/dhp/common/FunctionalInterfaceSupport.java
View File

@ -1,56 +1,54 @@
package eu.dnetlib.dhp.common; package eu.dnetlib.dhp.common;
import java.io.Serializable; import java.io.Serializable;
import java.util.function.Supplier; import java.util.function.Supplier;
/** /** Provides serializable and throwing extensions to standard functional interfaces. */
* Provides serializable and throwing extensions to standard functional interfaces.
*/
public class FunctionalInterfaceSupport { public class FunctionalInterfaceSupport {
private FunctionalInterfaceSupport() { private FunctionalInterfaceSupport() {
} }
/** /**
* Serializable supplier of any kind of objects. To be used withing spark processing pipelines when supplying * Serializable supplier of any kind of objects. To be used withing spark processing pipelines when supplying
* functions externally. * functions externally.
* *
* @param <T> * @param <T>
*/ */
@FunctionalInterface @FunctionalInterface
public interface SerializableSupplier<T> extends Supplier<T>, Serializable { public interface SerializableSupplier<T> extends Supplier<T>, Serializable {
} }
/** /**
* Extension of consumer accepting functions throwing an exception. * Extension of consumer accepting functions throwing an exception.
* *
* @param <T> * @param <T>
* @param <E> * @param <E>
*/ */
@FunctionalInterface @FunctionalInterface
public interface ThrowingConsumer<T, E extends Exception> { public interface ThrowingConsumer<T, E extends Exception> {
void accept(T t) throws E; void accept(T t) throws E;
} }
/** /**
* Extension of supplier accepting functions throwing an exception. * Extension of supplier accepting functions throwing an exception.
* *
* @param <T> * @param <T>
* @param <E> * @param <E>
*/ */
@FunctionalInterface @FunctionalInterface
public interface ThrowingSupplier<T, E extends Exception> { public interface ThrowingSupplier<T, E extends Exception> {
T get() throws E; T get() throws E;
} }
/** /**
* Extension of runnable accepting functions throwing an exception. * Extension of runnable accepting functions throwing an exception.
* *
* @param <E> * @param <E>
*/ */
@FunctionalInterface @FunctionalInterface
public interface ThrowingRunnable<E extends Exception> { public interface ThrowingRunnable<E extends Exception> {
void run() throws E; void run() throws E;
} }
}
}

103
dhp-common/src/main/java/eu/dnetlib/dhp/common/HdfsSupport.java
View File

@ -1,5 +1,12 @@
package eu.dnetlib.dhp.common; package eu.dnetlib.dhp.common;
import static eu.dnetlib.dhp.common.ThrowingSupport.rethrowAsRuntimeException;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus; import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.FileSystem;
@ -7,51 +14,61 @@ import org.apache.hadoop.fs.Path;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import java.util.Arrays; /** HDFS utility methods. */
import java.util.List;
import java.util.stream.Collectors;
import static eu.dnetlib.dhp.common.ThrowingSupport.rethrowAsRuntimeException;
/**
* HDFS utility methods.
*/
public class HdfsSupport { public class HdfsSupport {
private static final Logger logger = LoggerFactory.getLogger(HdfsSupport.class); private static final Logger logger = LoggerFactory.getLogger(HdfsSupport.class);
private HdfsSupport() { private HdfsSupport() {
} }
/** /**
* Removes a path (file or dir) from HDFS. * Checks a path (file or dir) exists on HDFS.
* *
* @param path Path to be removed * @param path Path to be checked
* @param configuration Configuration of hadoop env * @param configuration Configuration of hadoop env
*/ */
public static void remove(String path, Configuration configuration) { public static boolean exists(String path, Configuration configuration) {
logger.info("Removing path: {}", path); logger.info("Removing path: {}", path);
rethrowAsRuntimeException(() -> { return rethrowAsRuntimeException(
Path f = new Path(path); () -> {
FileSystem fileSystem = FileSystem.get(configuration); Path f = new Path(path);
if (fileSystem.exists(f)) { FileSystem fileSystem = FileSystem.get(configuration);
fileSystem.delete(f, true); return fileSystem.exists(f);
} });
}); }
}
/** /**
* Lists hadoop files located below path or alternatively lists subdirs under path. * Removes a path (file or dir) from HDFS.
* *
* @param path Path to be listed for hadoop files * @param path Path to be removed
* @param configuration Configuration of hadoop env * @param configuration Configuration of hadoop env
* @return List with string locations of hadoop files */
*/ public static void remove(String path, Configuration configuration) {
public static List<String> listFiles(String path, Configuration configuration) { logger.info("Removing path: {}", path);
logger.info("Listing files in path: {}", path); rethrowAsRuntimeException(
return rethrowAsRuntimeException(() -> Arrays () -> {
.stream(FileSystem.get(configuration).listStatus(new Path(path))) Path f = new Path(path);
.filter(FileStatus::isDirectory) FileSystem fileSystem = FileSystem.get(configuration);
.map(x -> x.getPath().toString()) if (fileSystem.exists(f)) {
.collect(Collectors.toList())); fileSystem.delete(f, true);
} }
});
}
/**
* Lists hadoop files located below path or alternatively lists subdirs under path.
*
* @param path Path to be listed for hadoop files
* @param configuration Configuration of hadoop env
* @return List with string locations of hadoop files
*/
public static List<String> listFiles(String path, Configuration configuration) {
logger.info("Listing files in path: {}", path);
return rethrowAsRuntimeException(
() -> Arrays
.stream(FileSystem.get(configuration).listStatus(new Path(path)))
.filter(FileStatus::isDirectory)
.map(x -> x.getPath().toString())
.collect(Collectors.toList()));
}
} }

126
dhp-common/src/main/java/eu/dnetlib/dhp/common/SparkSessionSupport.java
View File

@ -1,71 +1,75 @@
package eu.dnetlib.dhp.common;
import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingConsumer; package eu.dnetlib.dhp.common;
import org.apache.spark.SparkConf;
import org.apache.spark.sql.SparkSession;
import java.util.Objects; import java.util.Objects;
import java.util.function.Function; import java.util.function.Function;
/** import org.apache.spark.SparkConf;
* SparkSession utility methods. import org.apache.spark.sql.SparkSession;
*/
import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingConsumer;
/** SparkSession utility methods. */
public class SparkSessionSupport { public class SparkSessionSupport {
private SparkSessionSupport() { private SparkSessionSupport() {
} }
/** /**
* Runs a given function using SparkSession created using default builder and supplied SparkConf. Stops SparkSession * Runs a given function using SparkSession created using default builder and supplied SparkConf. Stops SparkSession
* when SparkSession is managed. Allows to reuse SparkSession created externally. * when SparkSession is managed. Allows to reuse SparkSession created externally.
* *
* @param conf SparkConf instance * @param conf SparkConf instance
* @param isSparkSessionManaged When true will stop SparkSession * @param isSparkSessionManaged When true will stop SparkSession
* @param fn Consumer to be applied to constructed SparkSession * @param fn Consumer to be applied to constructed SparkSession
*/ */
public static void runWithSparkSession(SparkConf conf, public static void runWithSparkSession(
Boolean isSparkSessionManaged, SparkConf conf, Boolean isSparkSessionManaged, ThrowingConsumer<SparkSession, Exception> fn) {
ThrowingConsumer<SparkSession, Exception> fn) { runWithSparkSession(
runWithSparkSession(c -> SparkSession.builder().config(c).getOrCreate(), conf, isSparkSessionManaged, fn); c -> SparkSession.builder().config(c).getOrCreate(), conf, isSparkSessionManaged, fn);
} }
/** /**
* Runs a given function using SparkSession created with hive support and using default builder and supplied SparkConf. * Runs a given function using SparkSession created with hive support and using default builder and supplied
* Stops SparkSession when SparkSession is managed. Allows to reuse SparkSession created externally. * SparkConf. Stops SparkSession when SparkSession is managed. Allows to reuse SparkSession created externally.
* *
* @param conf SparkConf instance * @param conf SparkConf instance
* @param isSparkSessionManaged When true will stop SparkSession * @param isSparkSessionManaged When true will stop SparkSession
* @param fn Consumer to be applied to constructed SparkSession * @param fn Consumer to be applied to constructed SparkSession
*/ */
public static void runWithSparkHiveSession(SparkConf conf, public static void runWithSparkHiveSession(
Boolean isSparkSessionManaged, SparkConf conf, Boolean isSparkSessionManaged, ThrowingConsumer<SparkSession, Exception> fn) {
ThrowingConsumer<SparkSession, Exception> fn) { runWithSparkSession(
runWithSparkSession(c -> SparkSession.builder().config(c).enableHiveSupport().getOrCreate(), conf, isSparkSessionManaged, fn); c -> SparkSession.builder().config(c).enableHiveSupport().getOrCreate(),
} conf,
isSparkSessionManaged,
fn);
}
/** /**
* Runs a given function using SparkSession created using supplied builder and supplied SparkConf. Stops SparkSession * Runs a given function using SparkSession created using supplied builder and supplied SparkConf. Stops
* when SparkSession is managed. Allows to reuse SparkSession created externally. * SparkSession when SparkSession is managed. Allows to reuse SparkSession created externally.
* *
* @param sparkSessionBuilder Builder of SparkSession * @param sparkSessionBuilder Builder of SparkSession
* @param conf SparkConf instance * @param conf SparkConf instance
* @param isSparkSessionManaged When true will stop SparkSession * @param isSparkSessionManaged When true will stop SparkSession
* @param fn Consumer to be applied to constructed SparkSession * @param fn Consumer to be applied to constructed SparkSession
*/ */
public static void runWithSparkSession(Function<SparkConf, SparkSession> sparkSessionBuilder, public static void runWithSparkSession(
SparkConf conf, Function<SparkConf, SparkSession> sparkSessionBuilder,
Boolean isSparkSessionManaged, SparkConf conf,
ThrowingConsumer<SparkSession, Exception> fn) { Boolean isSparkSessionManaged,
SparkSession spark = null; ThrowingConsumer<SparkSession, Exception> fn) {
try { SparkSession spark = null;
spark = sparkSessionBuilder.apply(conf); try {
fn.accept(spark); spark = sparkSessionBuilder.apply(conf);
} catch (Exception e) { fn.accept(spark);
throw new RuntimeException(e); } catch (Exception e) {
} finally { throw new RuntimeException(e);
if (Objects.nonNull(spark) && isSparkSessionManaged) { } finally {
spark.stop(); if (Objects.nonNull(spark) && isSparkSessionManaged) {
} spark.stop();
} }
} }
} }
}

130
dhp-common/src/main/java/eu/dnetlib/dhp/common/ThrowingSupport.java
View File

@ -1,76 +1,76 @@
package eu.dnetlib.dhp.common; package eu.dnetlib.dhp.common;
import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingRunnable; import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingRunnable;
import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingSupplier; import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingSupplier;
/** /** Exception handling utility methods. */
* Exception handling utility methods.
*/
public class ThrowingSupport { public class ThrowingSupport {
private ThrowingSupport() { private ThrowingSupport() {
} }
/** /**
* Executes given runnable and rethrows any exceptions as RuntimeException. * Executes given runnable and rethrows any exceptions as RuntimeException.
* *
* @param fn Runnable to be executed * @param fn Runnable to be executed
* @param <E> Type of exception thrown * @param <E> Type of exception thrown
*/ */
public static <E extends Exception> void rethrowAsRuntimeException(ThrowingRunnable<E> fn) { public static <E extends Exception> void rethrowAsRuntimeException(ThrowingRunnable<E> fn) {
try { try {
fn.run(); fn.run();
} catch (Exception e) { } catch (Exception e) {
throw new RuntimeException(e); throw new RuntimeException(e);
} }
} }
/** /**
* Executes given runnable and rethrows any exceptions as RuntimeException with custom message. * Executes given runnable and rethrows any exceptions as RuntimeException with custom message.
* *
* @param fn Runnable to be executed * @param fn Runnable to be executed
* @param msg Message to be set for rethrown exception * @param msg Message to be set for rethrown exception
* @param <E> Type of exception thrown * @param <E> Type of exception thrown
*/ */
public static <E extends Exception> void rethrowAsRuntimeException(ThrowingRunnable<E> fn, String msg) { public static <E extends Exception> void rethrowAsRuntimeException(
try { ThrowingRunnable<E> fn, String msg) {
fn.run(); try {
} catch (Exception e) { fn.run();
throw new RuntimeException(msg, e); } catch (Exception e) {
} throw new RuntimeException(msg, e);
} }
}
/** /**
* Executes given supplier and rethrows any exceptions as RuntimeException. * Executes given supplier and rethrows any exceptions as RuntimeException.
* *
* @param fn Supplier to be executed * @param fn Supplier to be executed
* @param <T> Type of returned value * @param <T> Type of returned value
* @param <E> Type of exception thrown * @param <E> Type of exception thrown
* @return Result of supplier execution * @return Result of supplier execution
*/ */
public static <T, E extends Exception> T rethrowAsRuntimeException(ThrowingSupplier<T, E> fn) { public static <T, E extends Exception> T rethrowAsRuntimeException(ThrowingSupplier<T, E> fn) {
try { try {
return fn.get(); return fn.get();
} catch (Exception e) { } catch (Exception e) {
throw new RuntimeException(e); throw new RuntimeException(e);
} }
} }
/** /**
* Executes given supplier and rethrows any exceptions as RuntimeException with custom message. * Executes given supplier and rethrows any exceptions as RuntimeException with custom message.
* *
* @param fn Supplier to be executed * @param fn Supplier to be executed
* @param msg Message to be set for rethrown exception * @param msg Message to be set for rethrown exception
* @param <T> Type of returned value * @param <T> Type of returned value
* @param <E> Type of exception thrown * @param <E> Type of exception thrown
* @return Result of supplier execution * @return Result of supplier execution
*/ */
public static <T, E extends Exception> T rethrowAsRuntimeException(ThrowingSupplier<T, E> fn, String msg) { public static <T, E extends Exception> T rethrowAsRuntimeException(
try { ThrowingSupplier<T, E> fn, String msg) {
return fn.get(); try {
} catch (Exception e) { return fn.get();
throw new RuntimeException(msg, e); } catch (Exception e) {
} throw new RuntimeException(msg, e);
} }
}
} }

192
dhp-common/src/main/java/eu/dnetlib/dhp/model/mdstore/MetadataRecord.java
View File

@ -1,137 +1,121 @@
package eu.dnetlib.dhp.model.mdstore;
import eu.dnetlib.dhp.utils.DHPUtils; package eu.dnetlib.dhp.model.mdstore;
import java.io.Serializable; import java.io.Serializable;
import eu.dnetlib.dhp.utils.DHPUtils;
/** /** This class models a record inside the new Metadata store collection on HDFS * */
* This class models a record inside the new Metadata store collection on HDFS *
*
*/
public class MetadataRecord implements Serializable { public class MetadataRecord implements Serializable {
/** /** The D-Net Identifier associated to the record */
* The D-Net Identifier associated to the record private String id;
*/
private String id;
/** /** The original Identifier of the record */
* The original Identifier of the record private String originalId;
*/
private String originalId;
/** The encoding of the record, should be JSON or XML */
private String encoding;
/** /**
* The encoding of the record, should be JSON or XML * The information about the provenance of the record see @{@link Provenance} for the model of this information
*/ */
private String encoding; private Provenance provenance;
/** /** The content of the metadata */
* The information about the provenance of the record see @{@link Provenance} private String body;
* for the model of this information
*/
private Provenance provenance;
/** /** the date when the record has been stored */
* The content of the metadata private long dateOfCollection;
*/
private String body;
/** /** the date when the record has been stored */
* the date when the record has been stored private long dateOfTransformation;
*/
private long dateOfCollection;
/** public MetadataRecord() {
* the date when the record has been stored this.dateOfCollection = System.currentTimeMillis();
*/ }
private long dateOfTransformation;
public MetadataRecord(
String originalId,
String encoding,
Provenance provenance,
String body,
long dateOfCollection) {
public MetadataRecord() { this.originalId = originalId;
this.dateOfCollection = System.currentTimeMillis(); this.encoding = encoding;
} this.provenance = provenance;
this.body = body;
this.dateOfCollection = dateOfCollection;
this.id = DHPUtils.generateIdentifier(originalId, this.provenance.getNsPrefix());
}
public MetadataRecord(String originalId, String encoding, Provenance provenance, String body, long dateOfCollection) { public String getId() {
return id;
}
this.originalId = originalId; public void setId(String id) {
this.encoding = encoding; this.id = id;
this.provenance = provenance; }
this.body = body;
this.dateOfCollection = dateOfCollection;
this.id = DHPUtils.generateIdentifier(originalId,this.provenance.getNsPrefix());
}
public String getId() { public String getOriginalId() {
return id; return originalId;
} }
public void setId(String id) { public void setOriginalId(String originalId) {
this.id = id; this.originalId = originalId;
} }
public String getEncoding() {
return encoding;
}
public String getOriginalId() { public void setEncoding(String encoding) {
return originalId; this.encoding = encoding;
} }
public void setOriginalId(String originalId) { public Provenance getProvenance() {
this.originalId = originalId; return provenance;
} }
public String getEncoding() { public void setProvenance(Provenance provenance) {
return encoding; this.provenance = provenance;
} }
public void setEncoding(String encoding) { public String getBody() {
this.encoding = encoding; return body;
} }
public Provenance getProvenance() { public void setBody(String body) {
return provenance; this.body = body;
} }
public void setProvenance(Provenance provenance) { public long getDateOfCollection() {
this.provenance = provenance; return dateOfCollection;
} }
public void setDateOfCollection(long dateOfCollection) {
this.dateOfCollection = dateOfCollection;
}
public String getBody() { public long getDateOfTransformation() {
return body; return dateOfTransformation;
} }
public void setBody(String body) { public void setDateOfTransformation(long dateOfTransformation) {
this.body = body; this.dateOfTransformation = dateOfTransformation;
} }
public long getDateOfCollection() { @Override
return dateOfCollection; public boolean equals(Object o) {
} if (!(o instanceof MetadataRecord)) {
return false;
}
return ((MetadataRecord) o).getId().equalsIgnoreCase(id);
}
public void setDateOfCollection(long dateOfCollection) { @Override
this.dateOfCollection = dateOfCollection; public int hashCode() {
} return id.hashCode();
}
public long getDateOfTransformation() {
return dateOfTransformation;
}
public void setDateOfTransformation(long dateOfTransformation) {
this.dateOfTransformation = dateOfTransformation;
}
@Override
public boolean equals(Object o) {
if (!(o instanceof MetadataRecord)) {
return false;
}
return ((MetadataRecord) o).getId().equalsIgnoreCase(id);
}
@Override
public int hashCode() {
return id.hashCode();
}
} }

68
dhp-common/src/main/java/eu/dnetlib/dhp/model/mdstore/Provenance.java
View File

@ -1,56 +1,52 @@
package eu.dnetlib.dhp.model.mdstore; package eu.dnetlib.dhp.model.mdstore;
import java.io.Serializable; import java.io.Serializable;
/** /**
* @author Sandro La Bruzzo * @author Sandro La Bruzzo
* * <p>
* Provenace class models the provenance of the record in the metadataStore * Provenace class models the provenance of the record in the metadataStore It contains the identifier and the
* It contains the identifier and the name of the datasource that gives the * name of the datasource that gives the record
* record
*
*/ */
public class Provenance implements Serializable { public class Provenance implements Serializable {
private String datasourceId; private String datasourceId;
private String datasourceName;
private String datasourceName; private String nsPrefix;
private String nsPrefix; public Provenance() {
}
public Provenance() { public Provenance(String datasourceId, String datasourceName, String nsPrefix) {
this.datasourceId = datasourceId;
this.datasourceName = datasourceName;
this.nsPrefix = nsPrefix;
}
} public String getDatasourceId() {
return datasourceId;
}
public Provenance(String datasourceId, String datasourceName, String nsPrefix) { public void setDatasourceId(String datasourceId) {
this.datasourceId = datasourceId; this.datasourceId = datasourceId;
this.datasourceName = datasourceName; }
this.nsPrefix = nsPrefix;
}
public String getDatasourceId() { public String getDatasourceName() {
return datasourceId; return datasourceName;
} }
public void setDatasourceId(String datasourceId) { public void setDatasourceName(String datasourceName) {
this.datasourceId = datasourceId; this.datasourceName = datasourceName;
} }
public String getDatasourceName() { public String getNsPrefix() {
return datasourceName; return nsPrefix;
} }
public void setDatasourceName(String datasourceName) { public void setNsPrefix(String nsPrefix) {
this.datasourceName = datasourceName; this.nsPrefix = nsPrefix;
} }
public String getNsPrefix() {
return nsPrefix;
}
public void setNsPrefix(String nsPrefix) {
this.nsPrefix = nsPrefix;
}
} }

15
dhp-common/src/main/java/eu/dnetlib/dhp/parser/utility/VtdException.java
View File

@ -1,12 +1,13 @@
package eu.dnetlib.dhp.parser.utility; package eu.dnetlib.dhp.parser.utility;
public class VtdException extends Exception { public class VtdException extends Exception {
public VtdException(final Exception e) { public VtdException(final Exception e) {
super(e); super(e);
} }
public VtdException(final Throwable e) { public VtdException(final Throwable e) {
super(e); super(e);
} }
} }

167
dhp-common/src/main/java/eu/dnetlib/dhp/parser/utility/VtdUtilityParser.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.dhp.parser.utility; package eu.dnetlib.dhp.parser.utility;
import java.util.ArrayList; import java.util.ArrayList;
@ -5,103 +6,105 @@ import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import com.ximpleware.AutoPilot; import com.ximpleware.AutoPilot;
import com.ximpleware.VTDNav; import com.ximpleware.VTDNav;
/** /** Created by sandro on 9/29/16. */
* Created by sandro on 9/29/16.
*/
public class VtdUtilityParser { public class VtdUtilityParser {
public static List<Node> getTextValuesWithAttributes(final AutoPilot ap, final VTDNav vn, final String xpath, final List<String> attributes) public static List<Node> getTextValuesWithAttributes(
throws VtdException { final AutoPilot ap, final VTDNav vn, final String xpath, final List<String> attributes)
final List<Node> results = new ArrayList<>(); throws VtdException {
try { final List<Node> results = new ArrayList<>();
ap.selectXPath(xpath); try {
ap.selectXPath(xpath);
while (ap.evalXPath() != -1) { while (ap.evalXPath() != -1) {
final Node currentNode = new Node(); final Node currentNode = new Node();
int t = vn.getText(); int t = vn.getText();
if (t >= 0) { if (t >= 0) {
currentNode.setTextValue(vn.toNormalizedString(t)); currentNode.setTextValue(vn.toNormalizedString(t));
} }
currentNode.setAttributes(getAttributes(vn, attributes)); currentNode.setAttributes(getAttributes(vn, attributes));
results.add(currentNode); results.add(currentNode);
} }
return results; return results;
} catch (Exception e) { } catch (Exception e) {
throw new VtdException(e); throw new VtdException(e);
} }
} }
private static Map<String, String> getAttributes(final VTDNav vn, final List<String> attributes) { private static Map<String, String> getAttributes(final VTDNav vn, final List<String> attributes) {
final Map<String, String> currentAttributes = new HashMap<>(); final Map<String, String> currentAttributes = new HashMap<>();
if (attributes != null) { if (attributes != null) {
attributes.forEach(attributeKey -> { attributes
try { .forEach(
int attr = vn.getAttrVal(attributeKey); attributeKey -> {
if (attr > -1) { try {
currentAttributes.put(attributeKey, vn.toNormalizedString(attr)); int attr = vn.getAttrVal(attributeKey);
} if (attr > -1) {
} catch (Throwable e) { currentAttributes.put(attributeKey, vn.toNormalizedString(attr));
throw new RuntimeException(e); }
} } catch (Throwable e) {
}); throw new RuntimeException(e);
} }
return currentAttributes; });
} }
return currentAttributes;
}
public static List<String> getTextValue(final AutoPilot ap, final VTDNav vn, final String xpath) throws VtdException { public static List<String> getTextValue(final AutoPilot ap, final VTDNav vn, final String xpath)
List<String> results = new ArrayList<>(); throws VtdException {
try { List<String> results = new ArrayList<>();
ap.selectXPath(xpath); try {
while (ap.evalXPath() != -1) { ap.selectXPath(xpath);
int t = vn.getText(); while (ap.evalXPath() != -1) {
if (t > -1) results.add(vn.toNormalizedString(t)); int t = vn.getText();
} if (t > -1)
return results; results.add(vn.toNormalizedString(t));
} catch (Exception e) { }
throw new VtdException(e); return results;
} } catch (Exception e) {
} throw new VtdException(e);
}
}
public static String getSingleValue(final AutoPilot ap, final VTDNav nav, final String xpath) throws VtdException { public static String getSingleValue(final AutoPilot ap, final VTDNav nav, final String xpath)
try { throws VtdException {
ap.selectXPath(xpath); try {
while (ap.evalXPath() != -1) { ap.selectXPath(xpath);
int it = nav.getText(); while (ap.evalXPath() != -1) {
if (it > -1) int it = nav.getText();
return nav.toNormalizedString(it); if (it > -1)
} return nav.toNormalizedString(it);
return null; }
} catch (Exception e) { return null;
throw new VtdException(e); } catch (Exception e) {
} throw new VtdException(e);
} }
}
public static class Node { public static class Node {
private String textValue; private String textValue;
private Map<String, String> attributes; private Map<String, String> attributes;
public String getTextValue() { public String getTextValue() {
return textValue; return textValue;
} }
public void setTextValue(final String textValue) { public void setTextValue(final String textValue) {
this.textValue = textValue; this.textValue = textValue;
} }
public Map<String, String> getAttributes() { public Map<String, String> getAttributes() {
return attributes; return attributes;
} }
public void setAttributes(final Map<String, String> attributes) {
this.attributes = attributes;
}
}
public void setAttributes(final Map<String, String> attributes) {
this.attributes = attributes;
}
}
} }

117
dhp-common/src/main/java/eu/dnetlib/dhp/utils/DHPUtils.java
View File

@ -1,11 +1,5 @@
package eu.dnetlib.dhp.utils;
import com.jayway.jsonpath.JsonPath; package eu.dnetlib.dhp.utils;
import net.minidev.json.JSONArray;
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.codec.binary.Base64OutputStream;
import org.apache.commons.codec.binary.Hex;
import org.apache.commons.lang3.StringUtils;
import java.io.ByteArrayInputStream; import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream; import java.io.ByteArrayOutputStream;
@ -14,61 +8,68 @@ import java.security.MessageDigest;
import java.util.zip.GZIPInputStream; import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream; import java.util.zip.GZIPOutputStream;
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.codec.binary.Base64OutputStream;
import org.apache.commons.codec.binary.Hex;
import com.jayway.jsonpath.JsonPath;
import net.minidev.json.JSONArray;
public class DHPUtils { public class DHPUtils {
public static String md5(final String s) { public static String md5(final String s) {
try { try {
final MessageDigest md = MessageDigest.getInstance("MD5"); final MessageDigest md = MessageDigest.getInstance("MD5");
md.update(s.getBytes("UTF-8")); md.update(s.getBytes(StandardCharsets.UTF_8));
return new String(Hex.encodeHex(md.digest())); return new String(Hex.encodeHex(md.digest()));
} catch (final Exception e) { } catch (final Exception e) {
System.err.println("Error creating id"); System.err.println("Error creating id");
return null; return null;
} }
} }
public static String generateIdentifier(final String originalId, final String nsPrefix) { public static String generateIdentifier(final String originalId, final String nsPrefix) {
return String.format("%s::%s",nsPrefix, DHPUtils.md5(originalId)); return String.format("%s::%s", nsPrefix, DHPUtils.md5(originalId));
} }
public static String compressString(final String input ) { public static String compressString(final String input) {
try ( ByteArrayOutputStream out = new ByteArrayOutputStream(); Base64OutputStream b64os = new Base64OutputStream(out)) { try (ByteArrayOutputStream out = new ByteArrayOutputStream();
GZIPOutputStream gzip = new GZIPOutputStream(b64os); Base64OutputStream b64os = new Base64OutputStream(out)) {
gzip.write(input.getBytes(StandardCharsets.UTF_8)); GZIPOutputStream gzip = new GZIPOutputStream(b64os);
gzip.close(); gzip.write(input.getBytes(StandardCharsets.UTF_8));
return out.toString(); gzip.close();
} catch (Throwable e ) { return out.toString();
return null; } catch (Throwable e) {
} return null;
} }
}
public static String decompressString(final String input) {
byte[] byteArray = Base64.decodeBase64(input.getBytes());
int len;
try (GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream((byteArray)));
ByteArrayOutputStream bos = new ByteArrayOutputStream(byteArray.length)) {
byte[] buffer = new byte[1024];
while ((len = gis.read(buffer)) != -1) {
bos.write(buffer, 0, len);
}
return bos.toString();
} catch (Exception e) {
return null;
}
}
public static String decompressString(final String input) { public static String getJPathString(final String jsonPath, final String json) {
byte[] byteArray = Base64.decodeBase64(input.getBytes()); try {
int len; Object o = JsonPath.read(json, jsonPath);
try (GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream((byteArray))); ByteArrayOutputStream bos = new ByteArrayOutputStream(byteArray.length)) { if (o instanceof String)
byte[] buffer = new byte[1024]; return (String) o;
while((len = gis.read(buffer)) != -1){ if (o instanceof JSONArray && ((JSONArray) o).size() > 0)
bos.write(buffer, 0, len); return (String) ((JSONArray) o).get(0);
} return o.toString();
return bos.toString(); } catch (Exception e) {
} catch (Exception e) { return "";
return null; }
} }
}
public static String getJPathString(final String jsonPath, final String json) {
try {
Object o = JsonPath.read(json, jsonPath);
if (o instanceof String)
return (String) o;
if (o instanceof JSONArray && ((JSONArray) o).size() > 0)
return (String) ((JSONArray) o).get(0);
return o.toString();
} catch (Exception e) {
return "";
}
}
} }

28
dhp-common/src/main/java/eu/dnetlib/dhp/utils/ISLookupClientFactory.java
View File

@ -1,24 +1,26 @@
package eu.dnetlib.dhp.utils; package eu.dnetlib.dhp.utils;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpService;
import org.apache.commons.logging.Log; import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory; import org.apache.commons.logging.LogFactory;
import org.apache.cxf.jaxws.JaxWsProxyFactoryBean; import org.apache.cxf.jaxws.JaxWsProxyFactoryBean;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpService;
public class ISLookupClientFactory { public class ISLookupClientFactory {
private static final Log log = LogFactory.getLog(ISLookupClientFactory.class); private static final Log log = LogFactory.getLog(ISLookupClientFactory.class);
public static ISLookUpService getLookUpService(final String isLookupUrl) { public static ISLookUpService getLookUpService(final String isLookupUrl) {
return getServiceStub(ISLookUpService.class, isLookupUrl); return getServiceStub(ISLookUpService.class, isLookupUrl);
} }
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
private static <T> T getServiceStub(final Class<T> clazz, final String endpoint) { private static <T> T getServiceStub(final Class<T> clazz, final String endpoint) {
log.info(String.format("creating %s stub from %s", clazz.getName(), endpoint)); log.info(String.format("creating %s stub from %s", clazz.getName(), endpoint));
final JaxWsProxyFactoryBean jaxWsProxyFactory = new JaxWsProxyFactoryBean(); final JaxWsProxyFactoryBean jaxWsProxyFactory = new JaxWsProxyFactoryBean();
jaxWsProxyFactory.setServiceClass(clazz); jaxWsProxyFactory.setServiceClass(clazz);
jaxWsProxyFactory.setAddress(endpoint); jaxWsProxyFactory.setAddress(endpoint);
return (T) jaxWsProxyFactory.create(); return (T) jaxWsProxyFactory.create();
} }
} }

35
dhp-common/src/main/java/eu/dnetlib/dhp/utils/saxon/AbstractExtensionFunction.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.dhp.utils.saxon; package eu.dnetlib.dhp.utils.saxon;
import net.sf.saxon.expr.XPathContext; import net.sf.saxon.expr.XPathContext;
@ -9,24 +10,24 @@ import net.sf.saxon.trans.XPathException;
public abstract class AbstractExtensionFunction extends ExtensionFunctionDefinition { public abstract class AbstractExtensionFunction extends ExtensionFunctionDefinition {
public static String DEFAULT_SAXON_EXT_NS_URI = "http://www.d-net.research-infrastructures.eu/saxon-extension"; public static String DEFAULT_SAXON_EXT_NS_URI = "http://www.d-net.research-infrastructures.eu/saxon-extension";
public abstract String getName(); public abstract String getName();
public abstract Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException;
@Override public abstract Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException;
public StructuredQName getFunctionQName() {
return new StructuredQName("dnet", DEFAULT_SAXON_EXT_NS_URI, getName());
}
@Override @Override
public ExtensionFunctionCall makeCallExpression() { public StructuredQName getFunctionQName() {
return new ExtensionFunctionCall() { return new StructuredQName("dnet", DEFAULT_SAXON_EXT_NS_URI, getName());
@Override }
public Sequence call(XPathContext context, Sequence[] arguments) throws XPathException {
return doCall(context, arguments);
}
};
}
} @Override
public ExtensionFunctionCall makeCallExpression() {
return new ExtensionFunctionCall() {
@Override
public Sequence call(XPathContext context, Sequence[] arguments) throws XPathException {
return doCall(context, arguments);
}
};
}
}

102
dhp-common/src/main/java/eu/dnetlib/dhp/utils/saxon/ExtractYear.java
View File

@ -1,5 +1,11 @@
package eu.dnetlib.dhp.utils.saxon; package eu.dnetlib.dhp.utils.saxon;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.GregorianCalendar;
import net.sf.saxon.expr.XPathContext; import net.sf.saxon.expr.XPathContext;
import net.sf.saxon.om.Item; import net.sf.saxon.om.Item;
import net.sf.saxon.om.Sequence; import net.sf.saxon.om.Sequence;
@ -7,61 +13,61 @@ import net.sf.saxon.trans.XPathException;
import net.sf.saxon.value.SequenceType; import net.sf.saxon.value.SequenceType;
import net.sf.saxon.value.StringValue; import net.sf.saxon.value.StringValue;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.GregorianCalendar;
public class ExtractYear extends AbstractExtensionFunction { public class ExtractYear extends AbstractExtensionFunction {
private static final String[] dateFormats = { "yyyy-MM-dd", "yyyy/MM/dd" }; private static final String[] dateFormats = {
"yyyy-MM-dd", "yyyy/MM/dd"
};
@Override @Override
public String getName() { public String getName() {
return "extractYear"; return "extractYear";
} }
@Override @Override
public Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException { public Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException {
if (arguments == null | arguments.length == 0) { if (arguments == null | arguments.length == 0) {
return new StringValue(""); return new StringValue("");
} }
final Item item = arguments[0].head(); final Item item = arguments[0].head();
if (item == null) { if (item == null) {
return new StringValue(""); return new StringValue("");
} }
return new StringValue(_year(item.getStringValue())); return new StringValue(_year(item.getStringValue()));
} }
@Override @Override
public int getMinimumNumberOfArguments() { public int getMinimumNumberOfArguments() {
return 0; return 0;
} }
@Override @Override
public int getMaximumNumberOfArguments() { public int getMaximumNumberOfArguments() {
return 1; return 1;
} }
@Override @Override
public SequenceType[] getArgumentTypes() { public SequenceType[] getArgumentTypes() {
return new SequenceType[] { SequenceType.OPTIONAL_ITEM }; return new SequenceType[] {
} SequenceType.OPTIONAL_ITEM
};
}
@Override @Override
public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) { public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) {
return SequenceType.SINGLE_STRING; return SequenceType.SINGLE_STRING;
} }
private String _year(String s) { private String _year(String s) {
Calendar c = new GregorianCalendar(); Calendar c = new GregorianCalendar();
for (String format : dateFormats) { for (String format : dateFormats) {
try { try {
c.setTime(new SimpleDateFormat(format).parse(s)); c.setTime(new SimpleDateFormat(format).parse(s));
String year = String.valueOf(c.get(Calendar.YEAR)); String year = String.valueOf(c.get(Calendar.YEAR));
return year; return year;
} catch (ParseException e) {} } catch (ParseException e) {
} }
return ""; }
} return "";
}
} }

97
dhp-common/src/main/java/eu/dnetlib/dhp/utils/saxon/NormalizeDate.java
View File

@ -1,66 +1,71 @@
package eu.dnetlib.dhp.utils.saxon; package eu.dnetlib.dhp.utils.saxon;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import net.sf.saxon.expr.XPathContext; import net.sf.saxon.expr.XPathContext;
import net.sf.saxon.om.Sequence; import net.sf.saxon.om.Sequence;
import net.sf.saxon.trans.XPathException; import net.sf.saxon.trans.XPathException;
import net.sf.saxon.value.SequenceType; import net.sf.saxon.value.SequenceType;
import net.sf.saxon.value.StringValue; import net.sf.saxon.value.StringValue;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
public class NormalizeDate extends AbstractExtensionFunction { public class NormalizeDate extends AbstractExtensionFunction {
private static final String[] normalizeDateFormats = { "yyyy-MM-dd'T'hh:mm:ss", "yyyy-MM-dd", "yyyy/MM/dd", "yyyy" }; private static final String[] normalizeDateFormats = {
"yyyy-MM-dd'T'hh:mm:ss", "yyyy-MM-dd", "yyyy/MM/dd", "yyyy"
};
private static final String normalizeOutFormat = new String("yyyy-MM-dd'T'hh:mm:ss'Z'"); private static final String normalizeOutFormat = "yyyy-MM-dd'T'hh:mm:ss'Z'";
@Override @Override
public String getName() { public String getName() {
return "normalizeDate"; return "normalizeDate";
} }
@Override @Override
public Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException { public Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException {
if (arguments == null | arguments.length == 0) { if (arguments == null | arguments.length == 0) {
return new StringValue(""); return new StringValue("");
} }
String s = arguments[0].head().getStringValue(); String s = arguments[0].head().getStringValue();
return new StringValue(_year(s)); return new StringValue(_year(s));
} }
@Override @Override
public int getMinimumNumberOfArguments() { public int getMinimumNumberOfArguments() {
return 0; return 0;
} }
@Override @Override
public int getMaximumNumberOfArguments() { public int getMaximumNumberOfArguments() {
return 1; return 1;
} }
@Override @Override
public SequenceType[] getArgumentTypes() { public SequenceType[] getArgumentTypes() {
return new SequenceType[] { SequenceType.OPTIONAL_ITEM }; return new SequenceType[] {
} SequenceType.OPTIONAL_ITEM
};
}
@Override @Override
public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) { public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) {
return SequenceType.SINGLE_STRING; return SequenceType.SINGLE_STRING;
} }
private String _year(String s) { private String _year(String s) {
final String date = s != null ? s.trim() : ""; final String date = s != null ? s.trim() : "";
for (String format : normalizeDateFormats) {
try {
Date parse = new SimpleDateFormat(format).parse(date);
String res = new SimpleDateFormat(normalizeOutFormat).format(parse);
return res;
} catch (ParseException e) {}
}
return "";
}
for (String format : normalizeDateFormats) {
try {
Date parse = new SimpleDateFormat(format).parse(date);
String res = new SimpleDateFormat(normalizeOutFormat).format(parse);
return res;
} catch (ParseException e) {
}
}
return "";
}
} }

83
dhp-common/src/main/java/eu/dnetlib/dhp/utils/saxon/PickFirst.java
View File

@ -1,60 +1,63 @@
package eu.dnetlib.dhp.utils.saxon; package eu.dnetlib.dhp.utils.saxon;
import org.apache.commons.lang3.StringUtils;
import net.sf.saxon.expr.XPathContext; import net.sf.saxon.expr.XPathContext;
import net.sf.saxon.om.Item; import net.sf.saxon.om.Item;
import net.sf.saxon.om.Sequence; import net.sf.saxon.om.Sequence;
import net.sf.saxon.trans.XPathException; import net.sf.saxon.trans.XPathException;
import net.sf.saxon.value.SequenceType; import net.sf.saxon.value.SequenceType;
import net.sf.saxon.value.StringValue; import net.sf.saxon.value.StringValue;
import org.apache.commons.lang3.StringUtils;
public class PickFirst extends AbstractExtensionFunction { public class PickFirst extends AbstractExtensionFunction {
@Override @Override
public String getName() { public String getName() {
return "pickFirst"; return "pickFirst";
} }
@Override @Override
public Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException { public Sequence doCall(XPathContext context, Sequence[] arguments) throws XPathException {
if (arguments == null | arguments.length == 0) { if (arguments == null | arguments.length == 0) {
return new StringValue(""); return new StringValue("");
} }
final String s1 = getValue(arguments[0]); final String s1 = getValue(arguments[0]);
final String s2 = getValue(arguments[1]); final String s2 = getValue(arguments[1]);
return new StringValue(StringUtils.isNotBlank(s1) ? s1 : StringUtils.isNotBlank(s2) ? s2 : ""); return new StringValue(StringUtils.isNotBlank(s1) ? s1 : StringUtils.isNotBlank(s2) ? s2 : "");
} }
private String getValue(final Sequence arg) throws XPathException { private String getValue(final Sequence arg) throws XPathException {
if (arg != null) { if (arg != null) {
final Item item = arg.head(); final Item item = arg.head();
if (item != null) { if (item != null) {
return item.getStringValue(); return item.getStringValue();
} }
} }
return ""; return "";
} }
@Override @Override
public int getMinimumNumberOfArguments() { public int getMinimumNumberOfArguments() {
return 0; return 0;
} }
@Override @Override
public int getMaximumNumberOfArguments() { public int getMaximumNumberOfArguments() {
return 2; return 2;
} }
@Override @Override
public SequenceType[] getArgumentTypes() { public SequenceType[] getArgumentTypes() {
return new SequenceType[] { SequenceType.OPTIONAL_ITEM }; return new SequenceType[] {
} SequenceType.OPTIONAL_ITEM
};
@Override }
public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) {
return SequenceType.SINGLE_STRING;
}
@Override
public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) {
return SequenceType.SINGLE_STRING;
}
} }

38
dhp-common/src/main/java/eu/dnetlib/dhp/utils/saxon/SaxonTransformerFactory.java
View File

@ -1,30 +1,32 @@
package eu.dnetlib.dhp.utils.saxon; package eu.dnetlib.dhp.utils.saxon;
import net.sf.saxon.Configuration; import java.io.StringReader;
import net.sf.saxon.TransformerFactoryImpl;
import javax.xml.transform.Transformer; import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException; import javax.xml.transform.TransformerException;
import javax.xml.transform.stream.StreamSource; import javax.xml.transform.stream.StreamSource;
import java.io.StringReader;
import net.sf.saxon.Configuration;
import net.sf.saxon.TransformerFactoryImpl;
public class SaxonTransformerFactory { public class SaxonTransformerFactory {
/** /**
* Creates the index record transformer from the given XSLT * Creates the index record transformer from the given XSLT
* @param xslt *
* @return * @param xslt
* @throws TransformerException * @return
*/ * @throws TransformerException
public static Transformer newInstance(final String xslt) throws TransformerException { */
public static Transformer newInstance(final String xslt) throws TransformerException {
final TransformerFactoryImpl factory = new TransformerFactoryImpl(); final TransformerFactoryImpl factory = new TransformerFactoryImpl();
final Configuration conf = factory.getConfiguration(); final Configuration conf = factory.getConfiguration();
conf.registerExtensionFunction(new ExtractYear()); conf.registerExtensionFunction(new ExtractYear());
conf.registerExtensionFunction(new NormalizeDate()); conf.registerExtensionFunction(new NormalizeDate());
conf.registerExtensionFunction(new PickFirst()); conf.registerExtensionFunction(new PickFirst());
return factory.newTransformer(new StreamSource(new StringReader(xslt)));
}
return factory.newTransformer(new StreamSource(new StringReader(xslt)));
}
} }

112
dhp-common/src/main/java/eu/dnetlib/message/Message.java
View File

@ -1,82 +1,76 @@
package eu.dnetlib.message;
import com.fasterxml.jackson.core.JsonProcessingException; package eu.dnetlib.message;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException; import java.io.IOException;
import java.util.Map; import java.util.Map;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class Message { public class Message {
private String workflowId; private String workflowId;
private String jobName; private String jobName;
private MessageType type; private MessageType type;
private Map<String, String> body; private Map<String, String> body;
public static Message fromJson(final String json) throws IOException {
final ObjectMapper jsonMapper = new ObjectMapper();
return jsonMapper.readValue(json, Message.class);
}
public static Message fromJson(final String json) throws IOException { public Message() {
final ObjectMapper jsonMapper = new ObjectMapper(); }
return jsonMapper.readValue(json, Message.class);
public Message(String workflowId, String jobName, MessageType type, Map<String, String> body) {
this.workflowId = workflowId;
this.jobName = jobName;
this.type = type;
this.body = body;
}
} public String getWorkflowId() {
return workflowId;
}
public void setWorkflowId(String workflowId) {
this.workflowId = workflowId;
}
public Message() { public String getJobName() {
return jobName;
}
public void setJobName(String jobName) {
this.jobName = jobName;
}
public MessageType getType() {
return type;
}
} public void setType(MessageType type) {
this.type = type;
}
public Message(String workflowId, String jobName, MessageType type, Map<String, String> body) { public Map<String, String> getBody() {
this.workflowId = workflowId; return body;
this.jobName = jobName; }
this.type = type;
this.body = body;
}
public String getWorkflowId() { public void setBody(Map<String, String> body) {
return workflowId; this.body = body;
} }
public void setWorkflowId(String workflowId) { @Override
this.workflowId = workflowId; public String toString() {
} final ObjectMapper jsonMapper = new ObjectMapper();
try {
public String getJobName() { return jsonMapper.writeValueAsString(this);
return jobName; } catch (JsonProcessingException e) {
} return null;
}
public void setJobName(String jobName) { }
this.jobName = jobName;
}
public MessageType getType() {
return type;
}
public void setType(MessageType type) {
this.type = type;
}
public Map<String, String> getBody() {
return body;
}
public void setBody(Map<String, String> body) {
this.body = body;
}
@Override
public String toString() {
final ObjectMapper jsonMapper = new ObjectMapper();
try {
return jsonMapper.writeValueAsString(this);
} catch (JsonProcessingException e) {
return null;
}
}
} }

69
dhp-common/src/main/java/eu/dnetlib/message/MessageConsumer.java
View File

@ -1,46 +1,47 @@
package eu.dnetlib.message; package eu.dnetlib.message;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.LinkedBlockingQueue;
import com.rabbitmq.client.AMQP; import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel; import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DefaultConsumer; import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope; import com.rabbitmq.client.Envelope;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.LinkedBlockingQueue;
public class MessageConsumer extends DefaultConsumer { public class MessageConsumer extends DefaultConsumer {
final LinkedBlockingQueue<Message> queueMessages; final LinkedBlockingQueue<Message> queueMessages;
/**
* Constructs a new instance and records its association to the passed-in channel.
*
* @param channel the channel to which this consumer is attached
* @param queueMessages
*/
public MessageConsumer(Channel channel, LinkedBlockingQueue<Message> queueMessages) {
super(channel);
this.queueMessages = queueMessages;
}
/** @Override
* Constructs a new instance and records its association to the passed-in channel. public void handleDelivery(
* String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
* @param channel the channel to which this consumer is attached throws IOException {
* @param queueMessages final String json = new String(body, StandardCharsets.UTF_8);
*/ Message message = Message.fromJson(json);
public MessageConsumer(Channel channel, LinkedBlockingQueue<Message> queueMessages) { try {
super(channel); this.queueMessages.put(message);
this.queueMessages = queueMessages; System.out.println("Receiving Message " + message);
} } catch (InterruptedException e) {
if (message.getType() == MessageType.REPORT)
throw new RuntimeException("Error on sending message");
@Override else {
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { // TODO LOGGING EXCEPTION
final String json = new String(body, StandardCharsets.UTF_8); }
Message message = Message.fromJson(json); } finally {
try { getChannel().basicAck(envelope.getDeliveryTag(), false);
this.queueMessages.put(message); }
System.out.println("Receiving Message "+message); }
} catch (InterruptedException e) {
if (message.getType()== MessageType.REPORT)
throw new RuntimeException("Error on sending message");
else {
//TODO LOGGING EXCEPTION
}
} finally {
getChannel().basicAck(envelope.getDeliveryTag(), false);
}
}
} }

191
dhp-common/src/main/java/eu/dnetlib/message/MessageManager.java
View File

@ -1,9 +1,5 @@
package eu.dnetlib.message;
import com.rabbitmq.client.Channel; package eu.dnetlib.message;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import sun.rmi.runtime.Log;
import java.io.IOException; import java.io.IOException;
import java.util.HashMap; import java.util.HashMap;
@ -11,107 +7,130 @@ import java.util.Map;
import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeoutException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class MessageManager { public class MessageManager {
private final String messageHost; private final String messageHost;
private final String username; private final String username;
private final String password; private final String password;
private Connection connection; private Connection connection;
private Map<String , Channel> channels = new HashMap<>(); private final Map<String, Channel> channels = new HashMap<>();
private boolean durable; private boolean durable;
private boolean autodelete; private boolean autodelete;
final private LinkedBlockingQueue<Message> queueMessages; private final LinkedBlockingQueue<Message> queueMessages;
public MessageManager(String messageHost, String username, String password, final LinkedBlockingQueue<Message> queueMessages) { public MessageManager(
this.queueMessages = queueMessages; String messageHost,
this.messageHost = messageHost; String username,
this.username = username; String password,
this.password = password; final LinkedBlockingQueue<Message> queueMessages) {
} this.queueMessages = queueMessages;
this.messageHost = messageHost;
this.username = username;
this.password = password;
}
public MessageManager(
String messageHost,
String username,
String password,
boolean durable,
boolean autodelete,
final LinkedBlockingQueue<Message> queueMessages) {
this.queueMessages = queueMessages;
this.messageHost = messageHost;
this.username = username;
this.password = password;
public MessageManager(String messageHost, String username, String password, boolean durable, boolean autodelete, final LinkedBlockingQueue<Message> queueMessages) { this.durable = durable;
this.queueMessages = queueMessages; this.autodelete = autodelete;
this.messageHost = messageHost; }
this.username = username;
this.password = password;
this.durable = durable; private Connection createConnection() throws IOException, TimeoutException {
this.autodelete = autodelete; ConnectionFactory factory = new ConnectionFactory();
} factory.setHost(this.messageHost);
factory.setUsername(this.username);
factory.setPassword(this.password);
return factory.newConnection();
}
private Connection createConnection() throws IOException, TimeoutException { private Channel createChannel(
ConnectionFactory factory = new ConnectionFactory(); final Connection connection,
factory.setHost(this.messageHost); final String queueName,
factory.setUsername(this.username); final boolean durable,
factory.setPassword(this.password); final boolean autodelete)
return factory.newConnection(); throws Exception {
} Map<String, Object> args = new HashMap<>();
args.put("x-message-ttl", 10000);
Channel channel = connection.createChannel();
channel.queueDeclare(queueName, durable, false, this.autodelete, args);
return channel;
}
private Channel createChannel(final Connection connection, final String queueName, final boolean durable, final boolean autodelete ) throws Exception { private Channel getOrCreateChannel(final String queueName, boolean durable, boolean autodelete)
Map<String, Object> args = new HashMap<>(); throws Exception {
args.put("x-message-ttl", 10000); if (channels.containsKey(queueName)) {
Channel channel = connection.createChannel(); return channels.get(queueName);
channel.queueDeclare(queueName, durable, false, this.autodelete, args); }
return channel;
}
private Channel getOrCreateChannel(final String queueName, boolean durable, boolean autodelete) throws Exception { if (this.connection == null) {
if (channels.containsKey(queueName)) { this.connection = createConnection();
return channels.get(queueName); }
} channels.put(queueName, createChannel(this.connection, queueName, durable, autodelete));
return channels.get(queueName);
}
if (this.connection == null) { public void close() throws IOException {
this.connection = createConnection(); channels
} .values()
channels.put(queueName, createChannel(this.connection, queueName, durable, autodelete)); .forEach(
return channels.get(queueName); ch -> {
} try {
ch.close();
} catch (Exception e) {
// TODO LOG
}
});
this.connection.close();
}
public boolean sendMessage(final Message message, String queueName) throws Exception {
try {
Channel channel = getOrCreateChannel(queueName, this.durable, this.autodelete);
channel.basicPublish("", queueName, null, message.toString().getBytes());
return true;
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
public void close() throws IOException { public boolean sendMessage(
channels.values().forEach(ch-> { final Message message, String queueName, boolean durable_var, boolean autodelete_var)
try { throws Exception {
ch.close(); try {
} catch (Exception e) { Channel channel = getOrCreateChannel(queueName, durable_var, autodelete_var);
//TODO LOG channel.basicPublish("", queueName, null, message.toString().getBytes());
} return true;
}); } catch (Throwable e) {
throw new RuntimeException(e);
}
}
this.connection.close(); public void startConsumingMessage(
} final String queueName, final boolean durable, final boolean autodelete) throws Exception {
public boolean sendMessage(final Message message, String queueName) throws Exception { Channel channel = createChannel(createConnection(), queueName, durable, autodelete);
try { channel.basicConsume(queueName, false, new MessageConsumer(channel, queueMessages));
Channel channel = getOrCreateChannel(queueName, this.durable, this.autodelete); }
channel.basicPublish("", queueName,null, message.toString().getBytes());
return true;
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
public boolean sendMessage(final Message message, String queueName, boolean durable_var, boolean autodelete_var) throws Exception {
try {
Channel channel = getOrCreateChannel(queueName, durable_var, autodelete_var);
channel.basicPublish("", queueName,null, message.toString().getBytes());
return true;
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
public void startConsumingMessage(final String queueName, final boolean durable, final boolean autodelete) throws Exception{
Channel channel = createChannel(createConnection(), queueName, durable, autodelete);
channel.basicConsume(queueName, false, new MessageConsumer(channel,queueMessages));
}
} }

6
dhp-common/src/main/java/eu/dnetlib/message/MessageType.java
View File

@ -1,8 +1,6 @@
package eu.dnetlib.message; package eu.dnetlib.message;
public enum MessageType { public enum MessageType {
ONGOING, REPORT
ONGOING,
REPORT
} }

31
dhp-common/src/main/java/eu/dnetlib/scholexplorer/relation/RelInfo.java
View File

@ -1,24 +1,25 @@
package eu.dnetlib.scholexplorer.relation; package eu.dnetlib.scholexplorer.relation;
import java.io.Serializable; import java.io.Serializable;
public class RelInfo implements Serializable { public class RelInfo implements Serializable {
private String original; private String original;
private String inverse; private String inverse;
public String getOriginal() { public String getOriginal() {
return original; return original;
} }
public void setOriginal(String original) { public void setOriginal(String original) {
this.original = original; this.original = original;
} }
public String getInverse() { public String getInverse() {
return inverse; return inverse;
} }
public void setInverse(String inverse) { public void setInverse(String inverse) {
this.inverse = inverse; this.inverse = inverse;
} }
} }

19
dhp-common/src/main/java/eu/dnetlib/scholexplorer/relation/RelationMapper.java
View File

@ -1,19 +1,20 @@
package eu.dnetlib.scholexplorer.relation;
import com.fasterxml.jackson.databind.ObjectMapper; package eu.dnetlib.scholexplorer.relation;
import org.apache.commons.io.IOUtils;
import java.io.Serializable; import java.io.Serializable;
import java.util.HashMap; import java.util.HashMap;
public class RelationMapper extends HashMap<String,RelInfo > implements Serializable { import org.apache.commons.io.IOUtils;
public static RelationMapper load() throws Exception { import com.fasterxml.jackson.databind.ObjectMapper;
final String json = IOUtils.toString(RelationMapper.class.getResourceAsStream("relations.json")); public class RelationMapper extends HashMap<String, RelInfo> implements Serializable {
ObjectMapper mapper = new ObjectMapper(); public static RelationMapper load() throws Exception {
return mapper.readValue(json, RelationMapper.class);
}
final String json = IOUtils.toString(RelationMapper.class.getResourceAsStream("relations.json"));
ObjectMapper mapper = new ObjectMapper();
return mapper.readValue(json, RelationMapper.class);
}
} }

107
dhp-common/src/test/java/eu/dnetlib/dhp/application/ArgumentApplicationParserTest.java
View File

@ -1,56 +1,67 @@
package eu.dnetlib.dhp.application;
import org.apache.commons.io.IOUtils; package eu.dnetlib.dhp.application;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertNotNull;
import org.apache.commons.io.IOUtils;
import org.junit.jupiter.api.Test;
public class ArgumentApplicationParserTest { public class ArgumentApplicationParserTest {
@Test @Test
public void testParseParameter() throws Exception { public void testParseParameter() throws Exception {
final String jsonConfiguration = IOUtils.toString(this.getClass().getResourceAsStream("/eu/dnetlib/application/parameters.json")); final String jsonConfiguration = IOUtils
assertNotNull(jsonConfiguration); .toString(
ArgumentApplicationParser parser = new ArgumentApplicationParser(jsonConfiguration); this.getClass().getResourceAsStream("/eu/dnetlib/application/parameters.json"));
parser.parseArgument(new String[]{"-p", "value0", assertNotNull(jsonConfiguration);
"-a", "value1", ArgumentApplicationParser parser = new ArgumentApplicationParser(jsonConfiguration);
"-n", "value2", parser
"-u", "value3", .parseArgument(
"-ru", "value4", new String[] {
"-rp", "value5", "-p",
"-rh", "value6", "value0",
"-ro", "value7", "-a",
"-rr", "value8", "value1",
"-w", "value9", "-n",
"-cc", ArgumentApplicationParser.compressArgument(jsonConfiguration) "value2",
}); "-u",
assertNotNull(parser.get("hdfsPath")); "value3",
assertNotNull(parser.get("apidescriptor")); "-ru",
assertNotNull(parser.get("namenode")); "value4",
assertNotNull(parser.get("userHDFS")); "-rp",
assertNotNull(parser.get("rabbitUser")); "value5",
assertNotNull(parser.get("rabbitPassWord")); "-rh",
assertNotNull(parser.get("rabbitHost")); "value6",
assertNotNull(parser.get("rabbitOngoingQueue")); "-ro",
assertNotNull(parser.get("rabbitReportQueue")); "value7",
assertNotNull(parser.get("workflowId")); "-rr",
assertEquals("value0", parser.get("hdfsPath")); "value8",
assertEquals("value1", parser.get("apidescriptor")); "-w",
assertEquals("value2", parser.get("namenode")); "value9",
assertEquals("value3", parser.get("userHDFS")); "-cc",
assertEquals("value4", parser.get("rabbitUser")); ArgumentApplicationParser.compressArgument(jsonConfiguration)
assertEquals("value5", parser.get("rabbitPassWord")); });
assertEquals("value6", parser.get("rabbitHost")); assertNotNull(parser.get("hdfsPath"));
assertEquals("value7", parser.get("rabbitOngoingQueue")); assertNotNull(parser.get("apidescriptor"));
assertEquals("value8", parser.get("rabbitReportQueue")); assertNotNull(parser.get("namenode"));
assertEquals("value9", parser.get("workflowId")); assertNotNull(parser.get("userHDFS"));
assertEquals(jsonConfiguration, parser.get("ccCoco")); assertNotNull(parser.get("rabbitUser"));
} assertNotNull(parser.get("rabbitPassWord"));
assertNotNull(parser.get("rabbitHost"));
assertNotNull(parser.get("rabbitOngoingQueue"));
assertNotNull(parser.get("rabbitReportQueue"));
assertNotNull(parser.get("workflowId"));
assertEquals("value0", parser.get("hdfsPath"));
assertEquals("value1", parser.get("apidescriptor"));
assertEquals("value2", parser.get("namenode"));
assertEquals("value3", parser.get("userHDFS"));
assertEquals("value4", parser.get("rabbitUser"));
assertEquals("value5", parser.get("rabbitPassWord"));
assertEquals("value6", parser.get("rabbitHost"));
assertEquals("value7", parser.get("rabbitOngoingQueue"));
assertEquals("value8", parser.get("rabbitReportQueue"));
assertEquals("value9", parser.get("workflowId"));
assertEquals(jsonConfiguration, parser.get("ccCoco"));
}
} }

110
dhp-common/src/test/java/eu/dnetlib/dhp/common/HdfsSupportTest.java
View File

@ -1,9 +1,7 @@
package eu.dnetlib.dhp.common; package eu.dnetlib.dhp.common;
import org.apache.hadoop.conf.Configuration; import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import java.io.IOException; import java.io.IOException;
import java.nio.file.Files; import java.nio.file.Files;
@ -12,67 +10,71 @@ import java.util.Arrays;
import java.util.List; import java.util.List;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import static org.junit.jupiter.api.Assertions.*; import org.apache.hadoop.conf.Configuration;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
public class HdfsSupportTest { public class HdfsSupportTest {
@Nested @Nested
class Remove { class Remove {
@Test @Test
public void shouldThrowARuntimeExceptionOnError() { public void shouldThrowARuntimeExceptionOnError() {
// when // when
assertThrows(RuntimeException.class, () -> assertThrows(RuntimeException.class, () -> HdfsSupport.remove(null, new Configuration()));
HdfsSupport.remove(null, new Configuration())); }
}
@Test @Test
public void shouldRemoveADirFromHDFS(@TempDir Path tempDir) { public void shouldRemoveADirFromHDFS(@TempDir Path tempDir) {
// when // when
HdfsSupport.remove(tempDir.toString(), new Configuration()); HdfsSupport.remove(tempDir.toString(), new Configuration());
// then // then
assertFalse(Files.exists(tempDir)); assertFalse(Files.exists(tempDir));
} }
@Test @Test
public void shouldRemoveAFileFromHDFS(@TempDir Path tempDir) throws IOException { public void shouldRemoveAFileFromHDFS(@TempDir Path tempDir) throws IOException {
// given // given
Path file = Files.createTempFile(tempDir, "p", "s"); Path file = Files.createTempFile(tempDir, "p", "s");
// when // when
HdfsSupport.remove(file.toString(), new Configuration()); HdfsSupport.remove(file.toString(), new Configuration());
// then // then
assertFalse(Files.exists(file)); assertFalse(Files.exists(file));
} }
} }
@Nested @Nested
class ListFiles { class ListFiles {
@Test @Test
public void shouldThrowARuntimeExceptionOnError() { public void shouldThrowARuntimeExceptionOnError() {
// when // when
assertThrows(RuntimeException.class, () -> assertThrows(RuntimeException.class, () -> HdfsSupport.listFiles(null, new Configuration()));
HdfsSupport.listFiles(null, new Configuration())); }
}
@Test @Test
public void shouldListFilesLocatedInPath(@TempDir Path tempDir) throws IOException { public void shouldListFilesLocatedInPath(@TempDir Path tempDir) throws IOException {
Path subDir1 = Files.createTempDirectory(tempDir, "list_me"); Path subDir1 = Files.createTempDirectory(tempDir, "list_me");
Path subDir2 = Files.createTempDirectory(tempDir, "list_me"); Path subDir2 = Files.createTempDirectory(tempDir, "list_me");
// when // when
List<String> paths = HdfsSupport.listFiles(tempDir.toString(), new Configuration()); List<String> paths = HdfsSupport.listFiles(tempDir.toString(), new Configuration());
// then // then
assertEquals(2, paths.size()); assertEquals(2, paths.size());
List<String> expecteds = Arrays.stream(new String[]{subDir1.toString(), subDir2.toString()}) List<String> expecteds = Arrays.stream(new String[] {
.sorted().collect(Collectors.toList()); subDir1.toString(), subDir2.toString()
List<String> actuals = paths.stream().sorted().collect(Collectors.toList()); })
assertTrue(actuals.get(0).contains(expecteds.get(0))); .sorted()
assertTrue(actuals.get(1).contains(expecteds.get(1))); .collect(Collectors.toList());
} List<String> actuals = paths.stream().sorted().collect(Collectors.toList());
} assertTrue(actuals.get(0).contains(expecteds.get(0)));
} assertTrue(actuals.get(1).contains(expecteds.get(1)));
}
}
}

80
dhp-common/src/test/java/eu/dnetlib/dhp/common/SparkSessionSupportTest.java
View File

@ -1,54 +1,58 @@
package eu.dnetlib.dhp.common; package eu.dnetlib.dhp.common;
import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingConsumer; import static org.mockito.Mockito.*;
import java.util.function.Function;
import org.apache.spark.SparkConf; import org.apache.spark.SparkConf;
import org.apache.spark.sql.SparkSession; import org.apache.spark.sql.SparkSession;
import org.junit.jupiter.api.Nested; import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import java.util.function.Function; import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.ThrowingConsumer;
import static org.mockito.Mockito.*;
public class SparkSessionSupportTest { public class SparkSessionSupportTest {
@Nested @Nested
class RunWithSparkSession { class RunWithSparkSession {
@Test @Test
public void shouldExecuteFunctionAndNotStopSparkSessionWhenSparkSessionIsNotManaged() throws Exception { public void shouldExecuteFunctionAndNotStopSparkSessionWhenSparkSessionIsNotManaged()
// given throws Exception {
SparkSession spark = mock(SparkSession.class); // given
SparkConf conf = mock(SparkConf.class); SparkSession spark = mock(SparkSession.class);
Function<SparkConf, SparkSession> sparkSessionBuilder = mock(Function.class); SparkConf conf = mock(SparkConf.class);
when(sparkSessionBuilder.apply(conf)).thenReturn(spark); Function<SparkConf, SparkSession> sparkSessionBuilder = mock(Function.class);
ThrowingConsumer<SparkSession, Exception> fn = mock(ThrowingConsumer.class); when(sparkSessionBuilder.apply(conf)).thenReturn(spark);
ThrowingConsumer<SparkSession, Exception> fn = mock(ThrowingConsumer.class);
// when // when
SparkSessionSupport.runWithSparkSession(sparkSessionBuilder, conf, false, fn); SparkSessionSupport.runWithSparkSession(sparkSessionBuilder, conf, false, fn);
// then // then
verify(sparkSessionBuilder).apply(conf); verify(sparkSessionBuilder).apply(conf);
verify(fn).accept(spark); verify(fn).accept(spark);
verify(spark, never()).stop(); verify(spark, never()).stop();
} }
@Test @Test
public void shouldExecuteFunctionAndStopSparkSessionWhenSparkSessionIsManaged() throws Exception { public void shouldExecuteFunctionAndStopSparkSessionWhenSparkSessionIsManaged()
// given throws Exception {
SparkSession spark = mock(SparkSession.class); // given
SparkConf conf = mock(SparkConf.class); SparkSession spark = mock(SparkSession.class);
Function<SparkConf, SparkSession> sparkSessionBuilder = mock(Function.class); SparkConf conf = mock(SparkConf.class);
when(sparkSessionBuilder.apply(conf)).thenReturn(spark); Function<SparkConf, SparkSession> sparkSessionBuilder = mock(Function.class);
ThrowingConsumer<SparkSession, Exception> fn = mock(ThrowingConsumer.class); when(sparkSessionBuilder.apply(conf)).thenReturn(spark);
ThrowingConsumer<SparkSession, Exception> fn = mock(ThrowingConsumer.class);
// when // when
SparkSessionSupport.runWithSparkSession(sparkSessionBuilder, conf, true, fn); SparkSessionSupport.runWithSparkSession(sparkSessionBuilder, conf, true, fn);
// then // then
verify(sparkSessionBuilder).apply(conf); verify(sparkSessionBuilder).apply(conf);
verify(fn).accept(spark); verify(fn).accept(spark);
verify(spark, times(1)).stop(); verify(spark, times(1)).stop();
} }
} }
} }

17
dhp-common/src/test/java/eu/dnetlib/dhp/model/mdstore/MetadataRecordTest.java
View File

@ -1,15 +1,16 @@
package eu.dnetlib.dhp.model.mdstore;
import org.junit.jupiter.api.Test; package eu.dnetlib.dhp.model.mdstore;
import static org.junit.jupiter.api.Assertions.assertTrue; import static org.junit.jupiter.api.Assertions.assertTrue;
import org.junit.jupiter.api.Test;
public class MetadataRecordTest { public class MetadataRecordTest {
@Test @Test
public void getTimestamp() { public void getTimestamp() {
MetadataRecord r = new MetadataRecord(); MetadataRecord r = new MetadataRecord();
assertTrue(r.getDateOfCollection() > 0); assertTrue(r.getDateOfCollection() > 0);
} }
} }

76
dhp-common/src/test/java/eu/dnetlib/message/MessageTest.java
View File

@ -1,55 +1,51 @@
package eu.dnetlib.message; package eu.dnetlib.message;
import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.*;
import java.io.IOException; import java.io.IOException;
import java.util.HashMap; import java.util.HashMap;
import java.util.Map; import java.util.Map;
import static org.junit.jupiter.api.Assertions.*; import org.junit.jupiter.api.Test;
public class MessageTest { public class MessageTest {
@Test @Test
public void fromJsonTest() throws IOException { public void fromJsonTest() throws IOException {
Message m = new Message(); Message m = new Message();
m.setWorkflowId("wId"); m.setWorkflowId("wId");
m.setType(MessageType.ONGOING); m.setType(MessageType.ONGOING);
m.setJobName("Collection"); m.setJobName("Collection");
Map<String,String> body= new HashMap<>(); Map<String, String> body = new HashMap<>();
body.put("parsedItem", "300"); body.put("parsedItem", "300");
body.put("ExecutionTime", "30s"); body.put("ExecutionTime", "30s");
m.setBody(body); m.setBody(body);
System.out.println("m = " + m); System.out.println("m = " + m);
Message m1 = Message.fromJson(m.toString()); Message m1 = Message.fromJson(m.toString());
assertEquals(m1.getWorkflowId(), m.getWorkflowId()); assertEquals(m1.getWorkflowId(), m.getWorkflowId());
assertEquals(m1.getType(), m.getType()); assertEquals(m1.getType(), m.getType());
assertEquals(m1.getJobName(), m.getJobName()); assertEquals(m1.getJobName(), m.getJobName());
assertNotNull(m1.getBody()); assertNotNull(m1.getBody());
m1.getBody().keySet().forEach(it -> assertEquals(m1.getBody().get(it), m.getBody().get(it))); m1.getBody().keySet().forEach(it -> assertEquals(m1.getBody().get(it), m.getBody().get(it)));
assertEquals(m1.getJobName(), m.getJobName()); assertEquals(m1.getJobName(), m.getJobName());
} }
@Test @Test
public void toStringTest() { public void toStringTest() {
final String expectedJson= "{\"workflowId\":\"wId\",\"jobName\":\"Collection\",\"type\":\"ONGOING\",\"body\":{\"ExecutionTime\":\"30s\",\"parsedItem\":\"300\"}}"; final String expectedJson = "{\"workflowId\":\"wId\",\"jobName\":\"Collection\",\"type\":\"ONGOING\",\"body\":{\"ExecutionTime\":\"30s\",\"parsedItem\":\"300\"}}";
Message m = new Message(); Message m = new Message();
m.setWorkflowId("wId"); m.setWorkflowId("wId");
m.setType(MessageType.ONGOING); m.setType(MessageType.ONGOING);
m.setJobName("Collection"); m.setJobName("Collection");
Map<String,String> body= new HashMap<>(); Map<String, String> body = new HashMap<>();
body.put("parsedItem", "300"); body.put("parsedItem", "300");
body.put("ExecutionTime", "30s"); body.put("ExecutionTime", "30s");
m.setBody(body); m.setBody(body);
assertEquals(expectedJson,m.toString()); assertEquals(expectedJson, m.toString());
}
}
}
}

13
dhp-common/src/test/java/eu/dnetlib/scholexplorer/relation/RelationMapperTest.java
View File

@ -1,15 +1,14 @@
package eu.dnetlib.scholexplorer.relation; package eu.dnetlib.scholexplorer.relation;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
public class RelationMapperTest { public class RelationMapperTest {
@Test @Test
public void testLoadRels() throws Exception{ public void testLoadRels() throws Exception {
RelationMapper relationMapper = RelationMapper.load(); RelationMapper relationMapper = RelationMapper.load();
relationMapper.keySet().forEach(System.out::println); relationMapper.keySet().forEach(System.out::println);
}
}
} }

2
dhp-schemas/pom.xml
View File

@ -12,7 +12,7 @@
<artifactId>dhp-schemas</artifactId> <artifactId>dhp-schemas</artifactId>
<packaging>jar</packaging> <packaging>jar</packaging>
<description>This module contains common schema classes meant to be used across the dnet-hadoop submodules</description>
<dependencies> <dependencies>

48
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/action/AtomicAction.java
View File

@ -1,38 +1,40 @@
package eu.dnetlib.dhp.schema.action; package eu.dnetlib.dhp.schema.action;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import eu.dnetlib.dhp.schema.oaf.Oaf;
import java.io.Serializable; import java.io.Serializable;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import eu.dnetlib.dhp.schema.oaf.Oaf;
@JsonDeserialize(using = AtomicActionDeserializer.class) @JsonDeserialize(using = AtomicActionDeserializer.class)
public class AtomicAction<T extends Oaf> implements Serializable { public class AtomicAction<T extends Oaf> implements Serializable {
private Class<T> clazz; private Class<T> clazz;
private T payload; private T payload;
public AtomicAction() { public AtomicAction() {
} }
public AtomicAction(Class<T> clazz, T payload) { public AtomicAction(Class<T> clazz, T payload) {
this.clazz = clazz; this.clazz = clazz;
this.payload = payload; this.payload = payload;
} }
public Class<T> getClazz() { public Class<T> getClazz() {
return clazz; return clazz;
} }
public void setClazz(Class<T> clazz) { public void setClazz(Class<T> clazz) {
this.clazz = clazz; this.clazz = clazz;
} }
public T getPayload() { public T getPayload() {
return payload; return payload;
} }
public void setPayload(T payload) { public void setPayload(T payload) {
this.payload = payload; this.payload = payload;
} }
} }

33
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/action/AtomicActionDeserializer.java
View File

@ -1,29 +1,32 @@
package eu.dnetlib.dhp.schema.action; package eu.dnetlib.dhp.schema.action;
import java.io.IOException;
import com.fasterxml.jackson.core.JsonParser; import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext; import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer; import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectMapper;
import eu.dnetlib.dhp.schema.oaf.Oaf;
import java.io.IOException; import eu.dnetlib.dhp.schema.oaf.Oaf;
public class AtomicActionDeserializer extends JsonDeserializer { public class AtomicActionDeserializer extends JsonDeserializer {
@Override @Override
public Object deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JsonProcessingException { public Object deserialize(JsonParser jp, DeserializationContext ctxt)
JsonNode node = jp.getCodec().readTree(jp); throws IOException {
String classTag = node.get("clazz").asText(); JsonNode node = jp.getCodec().readTree(jp);
JsonNode payload = node.get("payload"); String classTag = node.get("clazz").asText();
ObjectMapper mapper = new ObjectMapper(); JsonNode payload = node.get("payload");
ObjectMapper mapper = new ObjectMapper();
try { try {
final Class<?> clazz = Class.forName(classTag); final Class<?> clazz = Class.forName(classTag);
return new AtomicAction(clazz, (Oaf) mapper.readValue(payload.toString(), clazz)); return new AtomicAction(clazz, (Oaf) mapper.readValue(payload.toString(), clazz));
} catch (ClassNotFoundException e) { } catch (ClassNotFoundException e) {
throw new IOException(e); throw new IOException(e);
} }
} }
} }

28
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/common/EntityType.java
View File

@ -1,23 +1,21 @@
package eu.dnetlib.dhp.schema.common; package eu.dnetlib.dhp.schema.common;
import eu.dnetlib.dhp.schema.oaf.OafEntity; import eu.dnetlib.dhp.schema.oaf.OafEntity;
/** /** Actual entity types in the Graph */
* Actual entity types in the Graph
*/
public enum EntityType { public enum EntityType {
publication, dataset, otherresearchproduct, software, datasource, organization, project;
publication, dataset, otherresearchproduct, software, datasource, organization, project; /**
* Resolves the EntityType, given the relative class name
/** *
* Resolves the EntityType, given the relative class name * @param clazz the given class name
* @param clazz the given class name * @param <T> actual OafEntity subclass
* @param <T> actual OafEntity subclass * @return the EntityType associated to the given class
* @return the EntityType associated to the given class */
*/ public static <T extends OafEntity> EntityType fromClass(Class<T> clazz) {
public static <T extends OafEntity> EntityType fromClass(Class<T> clazz) {
return EntityType.valueOf(clazz.getSimpleName().toLowerCase());
}
return EntityType.valueOf(clazz.getSimpleName().toLowerCase());
}
} }

10
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/common/MainEntityType.java
View File

@ -1,9 +1,7 @@
package eu.dnetlib.dhp.schema.common; package eu.dnetlib.dhp.schema.common;
/** /** Main entity types in the Graph */
* Main entity types in the Graph
*/
public enum MainEntityType { public enum MainEntityType {
result, datasource, organization, project
result, datasource, organization, project }
}

41
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/common/ModelConstants.java Normal file
View File

@ -0,0 +1,41 @@
package eu.dnetlib.dhp.schema.common;
import eu.dnetlib.dhp.schema.oaf.Qualifier;
public class ModelConstants {
public static final String DNET_RESULT_TYPOLOGIES = "dnet:result_typologies";
public static final String DATASET_RESULTTYPE_CLASSID = "dataset";
public static final String PUBLICATION_RESULTTYPE_CLASSID = "publication";
public static final String SOFTWARE_RESULTTYPE_CLASSID = "software";
public static final String ORP_RESULTTYPE_CLASSID = "other";
public static Qualifier PUBLICATION_DEFAULT_RESULTTYPE = new Qualifier();
public static Qualifier DATASET_DEFAULT_RESULTTYPE = new Qualifier();
public static Qualifier SOFTWARE_DEFAULT_RESULTTYPE = new Qualifier();
public static Qualifier ORP_DEFAULT_RESULTTYPE = new Qualifier();
static {
PUBLICATION_DEFAULT_RESULTTYPE.setClassid(PUBLICATION_RESULTTYPE_CLASSID);
PUBLICATION_DEFAULT_RESULTTYPE.setClassname(PUBLICATION_RESULTTYPE_CLASSID);
PUBLICATION_DEFAULT_RESULTTYPE.setSchemeid(DNET_RESULT_TYPOLOGIES);
PUBLICATION_DEFAULT_RESULTTYPE.setSchemename(DNET_RESULT_TYPOLOGIES);
DATASET_DEFAULT_RESULTTYPE.setClassid(DATASET_RESULTTYPE_CLASSID);
DATASET_DEFAULT_RESULTTYPE.setClassname(DATASET_RESULTTYPE_CLASSID);
DATASET_DEFAULT_RESULTTYPE.setSchemeid(DNET_RESULT_TYPOLOGIES);
DATASET_DEFAULT_RESULTTYPE.setSchemename(DNET_RESULT_TYPOLOGIES);
SOFTWARE_DEFAULT_RESULTTYPE.setClassid(SOFTWARE_RESULTTYPE_CLASSID);
SOFTWARE_DEFAULT_RESULTTYPE.setClassname(SOFTWARE_RESULTTYPE_CLASSID);
SOFTWARE_DEFAULT_RESULTTYPE.setSchemeid(DNET_RESULT_TYPOLOGIES);
SOFTWARE_DEFAULT_RESULTTYPE.setSchemename(DNET_RESULT_TYPOLOGIES);
ORP_DEFAULT_RESULTTYPE.setClassid(ORP_RESULTTYPE_CLASSID);
ORP_DEFAULT_RESULTTYPE.setClassname(ORP_RESULTTYPE_CLASSID);
ORP_DEFAULT_RESULTTYPE.setSchemeid(DNET_RESULT_TYPOLOGIES);
ORP_DEFAULT_RESULTTYPE.setSchemename(DNET_RESULT_TYPOLOGIES);
}
}

312
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/common/ModelSupport.java
View File

@ -1,151 +1,205 @@
package eu.dnetlib.dhp.schema.common; package eu.dnetlib.dhp.schema.common;
import java.util.Map;
import java.util.Optional;
import java.util.function.Function;
import com.google.common.collect.Maps; import com.google.common.collect.Maps;
import eu.dnetlib.dhp.schema.oaf.*; import eu.dnetlib.dhp.schema.oaf.*;
import java.util.Map; /** Oaf model utility methods. */
/**
* Oaf model utility methods.
*/
public class ModelSupport { public class ModelSupport {
/** /** Defines the mapping between the actual entity type and the main entity type */
* Defines the mapping between the actual entity type and the main entity type private static final Map<EntityType, MainEntityType> entityMapping = Maps.newHashMap();
*/
private static Map<EntityType, MainEntityType> entityMapping = Maps.newHashMap();
static { static {
entityMapping.put(EntityType.publication, MainEntityType.result); entityMapping.put(EntityType.publication, MainEntityType.result);
entityMapping.put(EntityType.dataset, MainEntityType.result); entityMapping.put(EntityType.dataset, MainEntityType.result);
entityMapping.put(EntityType.otherresearchproduct, MainEntityType.result); entityMapping.put(EntityType.otherresearchproduct, MainEntityType.result);
entityMapping.put(EntityType.software, MainEntityType.result); entityMapping.put(EntityType.software, MainEntityType.result);
entityMapping.put(EntityType.datasource, MainEntityType.datasource); entityMapping.put(EntityType.datasource, MainEntityType.datasource);
entityMapping.put(EntityType.organization, MainEntityType.organization); entityMapping.put(EntityType.organization, MainEntityType.organization);
entityMapping.put(EntityType.project, MainEntityType.project); entityMapping.put(EntityType.project, MainEntityType.project);
} }
/** /**
* Defines the mapping between the actual entity types and the relative classes implementing them * Defines the mapping between the actual entity types and the relative classes implementing them
*/ */
public final static Map<EntityType, Class> entityTypes = Maps.newHashMap(); public static final Map<EntityType, Class> entityTypes = Maps.newHashMap();
static { static {
entityTypes.put(EntityType.datasource, Datasource.class); entityTypes.put(EntityType.datasource, Datasource.class);
entityTypes.put(EntityType.organization, Organization.class); entityTypes.put(EntityType.organization, Organization.class);
entityTypes.put(EntityType.project, Project.class); entityTypes.put(EntityType.project, Project.class);
entityTypes.put(EntityType.dataset, Dataset.class); entityTypes.put(EntityType.dataset, Dataset.class);
entityTypes.put(EntityType.otherresearchproduct, OtherResearchProduct.class); entityTypes.put(EntityType.otherresearchproduct, OtherResearchProduct.class);
entityTypes.put(EntityType.software, Software.class); entityTypes.put(EntityType.software, Software.class);
entityTypes.put(EntityType.publication, Publication.class); entityTypes.put(EntityType.publication, Publication.class);
} }
public final static Map<String, Class> oafTypes = Maps.newHashMap(); public static final Map<String, Class> oafTypes = Maps.newHashMap();
static { static {
oafTypes.put("datasource", Datasource.class); oafTypes.put("datasource", Datasource.class);
oafTypes.put("organization", Organization.class); oafTypes.put("organization", Organization.class);
oafTypes.put("project", Project.class); oafTypes.put("project", Project.class);
oafTypes.put("dataset", Dataset.class); oafTypes.put("dataset", Dataset.class);
oafTypes.put("otherresearchproduct", OtherResearchProduct.class); oafTypes.put("otherresearchproduct", OtherResearchProduct.class);
oafTypes.put("software", Software.class); oafTypes.put("software", Software.class);
oafTypes.put("publication", Publication.class); oafTypes.put("publication", Publication.class);
oafTypes.put("relation", Relation.class); oafTypes.put("relation", Relation.class);
} }
private static final String schemeTemplate = "dnet:%s_%s_relations"; private static final String schemeTemplate = "dnet:%s_%s_relations";
private ModelSupport() { private ModelSupport() {
} }
/** /**
* Checks subclass-superclass relationship. * Checks subclass-superclass relationship.
* *
* @param subClazzObject Subclass object instance * @param subClazzObject Subclass object instance
* @param superClazzObject Superclass object instance * @param superClazzObject Superclass object instance
* @param <X> Subclass type * @param <X> Subclass type
* @param <Y> Superclass type * @param <Y> Superclass type
* @return True if X is a subclass of Y * @return True if X is a subclass of Y
*/ */
public static <X extends Oaf, Y extends Oaf> Boolean isSubClass(X subClazzObject, Y superClazzObject) { public static <X extends Oaf, Y extends Oaf> Boolean isSubClass(
return isSubClass(subClazzObject.getClass(), superClazzObject.getClass()); X subClazzObject, Y superClazzObject) {
} return isSubClass(subClazzObject.getClass(), superClazzObject.getClass());
}
/** /**
* Checks subclass-superclass relationship. * Checks subclass-superclass relationship.
* *
* @param subClazzObject Subclass object instance * @param subClazzObject Subclass object instance
* @param superClazz Superclass class * @param superClazz Superclass class
* @param <X> Subclass type * @param <X> Subclass type
* @param <Y> Superclass type * @param <Y> Superclass type
* @return True if X is a subclass of Y * @return True if X is a subclass of Y
*/ */
public static <X extends Oaf, Y extends Oaf> Boolean isSubClass(X subClazzObject, Class<Y> superClazz) { public static <X extends Oaf, Y extends Oaf> Boolean isSubClass(
return isSubClass(subClazzObject.getClass(), superClazz); X subClazzObject, Class<Y> superClazz) {
} return isSubClass(subClazzObject.getClass(), superClazz);
}
/** /**
* Checks subclass-superclass relationship. * Checks subclass-superclass relationship.
* *
* @param subClazz Subclass class * @param subClazz Subclass class
* @param superClazz Superclass class * @param superClazz Superclass class
* @param <X> Subclass type * @param <X> Subclass type
* @param <Y> Superclass type * @param <Y> Superclass type
* @return True if X is a subclass of Y * @return True if X is a subclass of Y
*/ */
public static <X extends Oaf, Y extends Oaf> Boolean isSubClass(Class<X> subClazz, Class<Y> superClazz) { public static <X extends Oaf, Y extends Oaf> Boolean isSubClass(
return superClazz.isAssignableFrom(subClazz); Class<X> subClazz, Class<Y> superClazz) {
} return superClazz.isAssignableFrom(subClazz);
}
/** /**
* Lists all the OAF model classes * Lists all the OAF model classes
* *
* @param <T> * @param <T>
* @return * @return
*/ */
public static <T extends Oaf> Class<T>[] getOafModelClasses() { public static <T extends Oaf> Class<T>[] getOafModelClasses() {
return new Class[]{ return new Class[] {
Author.class, Author.class,
Context.class, Context.class,
Country.class, Country.class,
DataInfo.class, DataInfo.class,
Dataset.class, Dataset.class,
Datasource.class, Datasource.class,
ExternalReference.class, ExternalReference.class,
ExtraInfo.class, ExtraInfo.class,
Field.class, Field.class,
GeoLocation.class, GeoLocation.class,
Instance.class, Instance.class,
Journal.class, Journal.class,
KeyValue.class, KeyValue.class,
Oaf.class, Oaf.class,
OafEntity.class, OafEntity.class,
OAIProvenance.class, OAIProvenance.class,
Organization.class, Organization.class,
OriginDescription.class, OriginDescription.class,
OtherResearchProduct.class, OtherResearchProduct.class,
Project.class, Project.class,
Publication.class, Publication.class,
Qualifier.class, Qualifier.class,
Relation.class, Relation.class,
Result.class, Result.class,
Software.class, Software.class,
StructuredProperty.class StructuredProperty.class
}; };
} }
public static String getMainType(final EntityType type) { public static String getMainType(final EntityType type) {
return entityMapping.get(type).name(); return entityMapping.get(type).name();
} }
public static boolean isResult(EntityType type) { public static boolean isResult(EntityType type) {
return MainEntityType.result.name().equals(getMainType(type)); return MainEntityType.result.name().equals(getMainType(type));
} }
public static String getScheme(final String sourceType, final String targetType) { public static String getScheme(final String sourceType, final String targetType) {
return String.format(schemeTemplate, return String
entityMapping.get(EntityType.valueOf(sourceType)).name(), .format(
entityMapping.get(EntityType.valueOf(targetType)).name()); schemeTemplate,
} entityMapping.get(EntityType.valueOf(sourceType)).name(),
entityMapping.get(EntityType.valueOf(targetType)).name());
}
} public static <T extends Oaf> Function<T, String> idFn() {
return x -> {
if (isSubClass(x, Relation.class)) {
return idFnForRelation(x);
}
return idFnForOafEntity(x);
};
}
private static <T extends Oaf> String idFnForRelation(T t) {
Relation r = (Relation) t;
return Optional
.ofNullable(r.getSource())
.map(
source -> Optional
.ofNullable(r.getTarget())
.map(
target -> Optional
.ofNullable(r.getRelType())
.map(
relType -> Optional
.ofNullable(r.getSubRelType())
.map(
subRelType -> Optional
.ofNullable(r.getRelClass())
.map(
relClass -> String
.join(
source,
target,
relType,
subRelType,
relClass))
.orElse(
String
.join(
source,
target,
relType,
subRelType)))
.orElse(String.join(source, target, relType)))
.orElse(String.join(source, target)))
.orElse(source))
.orElse(null);
}
private static <T extends Oaf> String idFnForOafEntity(T t) {
return ((OafEntity) t).getId();
}
}

119
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Author.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
@ -6,81 +7,83 @@ import java.util.Objects;
public class Author implements Serializable { public class Author implements Serializable {
private String fullname; private String fullname;
private String name; private String name;
private String surname; private String surname;
private Integer rank; private Integer rank;
private List<StructuredProperty> pid; private List<StructuredProperty> pid;
private List<Field<String>> affiliation; private List<Field<String>> affiliation;
public String getFullname() { public String getFullname() {
return fullname; return fullname;
} }
public void setFullname(String fullname) { public void setFullname(String fullname) {
this.fullname = fullname; this.fullname = fullname;
} }
public String getName() { public String getName() {
return name; return name;
} }
public void setName(String name) { public void setName(String name) {
this.name = name; this.name = name;
} }
public String getSurname() { public String getSurname() {
return surname; return surname;
} }
public void setSurname(String surname) { public void setSurname(String surname) {
this.surname = surname; this.surname = surname;
} }
public Integer getRank() { public Integer getRank() {
return rank; return rank;
} }
public void setRank(Integer rank) { public void setRank(Integer rank) {
this.rank = rank; this.rank = rank;
} }
public List<StructuredProperty> getPid() { public List<StructuredProperty> getPid() {
return pid; return pid;
} }
public void setPid(List<StructuredProperty> pid) { public void setPid(List<StructuredProperty> pid) {
this.pid = pid; this.pid = pid;
} }
public List<Field<String>> getAffiliation() { public List<Field<String>> getAffiliation() {
return affiliation; return affiliation;
} }
public void setAffiliation(List<Field<String>> affiliation) { public void setAffiliation(List<Field<String>> affiliation) {
this.affiliation = affiliation; this.affiliation = affiliation;
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
Author author = (Author) o; if (o == null || getClass() != o.getClass())
return Objects.equals(fullname, author.fullname) && return false;
Objects.equals(name, author.name) && Author author = (Author) o;
Objects.equals(surname, author.surname) && return Objects.equals(fullname, author.fullname)
Objects.equals(rank, author.rank) && && Objects.equals(name, author.name)
Objects.equals(pid, author.pid) && && Objects.equals(surname, author.surname)
Objects.equals(affiliation, author.affiliation); && Objects.equals(rank, author.rank)
} && Objects.equals(pid, author.pid)
&& Objects.equals(affiliation, author.affiliation);
}
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(fullname, name, surname, rank, pid, affiliation); return Objects.hash(fullname, name, surname, rank, pid, affiliation);
} }
} }

59
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Context.java
View File

@ -1,45 +1,46 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List; import java.util.List;
public class Context implements Serializable { public class Context implements Serializable {
private String id; private String id;
private List<DataInfo> dataInfo; private List<DataInfo> dataInfo;
public String getId() { public String getId() {
return id; return id;
} }
public void setId(String id) { public void setId(String id) {
this.id = id; this.id = id;
} }
public List<DataInfo> getDataInfo() { public List<DataInfo> getDataInfo() {
return dataInfo; return dataInfo;
} }
public void setDataInfo(List<DataInfo> dataInfo) { public void setDataInfo(List<DataInfo> dataInfo) {
this.dataInfo = dataInfo; this.dataInfo = dataInfo;
} }
@Override @Override
public int hashCode() { public int hashCode() {
return id ==null? 0 : id.hashCode(); return id == null ? 0 : id.hashCode();
} }
@Override @Override
public boolean equals(Object obj) { public boolean equals(Object obj) {
if (this == obj) if (this == obj)
return true; return true;
if (obj == null) if (obj == null)
return false; return false;
if (getClass() != obj.getClass()) if (getClass() != obj.getClass())
return false; return false;
Context other = (Context) obj; Context other = (Context) obj;
return id.equals(other.getId()); return id.equals(other.getId());
} }
} }

42
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Country.java
View File

@ -1,30 +1,34 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.util.Objects; import java.util.Objects;
public class Country extends Qualifier { public class Country extends Qualifier {
private DataInfo dataInfo; private DataInfo dataInfo;
public DataInfo getDataInfo() { public DataInfo getDataInfo() {
return dataInfo; return dataInfo;
} }
public void setDataInfo(DataInfo dataInfo) { public void setDataInfo(DataInfo dataInfo) {
this.dataInfo = dataInfo; this.dataInfo = dataInfo;
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
if (!super.equals(o)) return false; if (o == null || getClass() != o.getClass())
Country country = (Country) o; return false;
return Objects.equals(dataInfo, country.dataInfo); if (!super.equals(o))
} return false;
Country country = (Country) o;
return Objects.equals(dataInfo, country.dataInfo);
}
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(super.hashCode(), dataInfo); return Objects.hash(super.hashCode(), dataInfo);
} }
} }

123
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/DataInfo.java
View File

@ -1,82 +1,85 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List;
import java.util.Objects; import java.util.Objects;
public class DataInfo implements Serializable { public class DataInfo implements Serializable {
private Boolean invisible = false; private Boolean invisible = false;
private Boolean inferred; private Boolean inferred;
private Boolean deletedbyinference; private Boolean deletedbyinference;
private String trust; private String trust;
private String inferenceprovenance; private String inferenceprovenance;
private Qualifier provenanceaction; private Qualifier provenanceaction;
public Boolean getInvisible() {
return invisible;
}
public Boolean getInvisible() { public void setInvisible(Boolean invisible) {
return invisible; this.invisible = invisible;
} }
public void setInvisible(Boolean invisible) { public Boolean getInferred() {
this.invisible = invisible; return inferred;
} }
public Boolean getInferred() { public void setInferred(Boolean inferred) {
return inferred; this.inferred = inferred;
} }
public void setInferred(Boolean inferred) { public Boolean getDeletedbyinference() {
this.inferred = inferred; return deletedbyinference;
} }
public Boolean getDeletedbyinference() { public void setDeletedbyinference(Boolean deletedbyinference) {
return deletedbyinference; this.deletedbyinference = deletedbyinference;
} }
public void setDeletedbyinference(Boolean deletedbyinference) { public String getTrust() {
this.deletedbyinference = deletedbyinference; return trust;
} }
public String getTrust() { public void setTrust(String trust) {
return trust; this.trust = trust;
} }
public void setTrust(String trust) { public String getInferenceprovenance() {
this.trust = trust; return inferenceprovenance;
} }
public String getInferenceprovenance() { public void setInferenceprovenance(String inferenceprovenance) {
return inferenceprovenance; this.inferenceprovenance = inferenceprovenance;
} }
public void setInferenceprovenance(String inferenceprovenance) { public Qualifier getProvenanceaction() {
this.inferenceprovenance = inferenceprovenance; return provenanceaction;
} }
public Qualifier getProvenanceaction() { public void setProvenanceaction(Qualifier provenanceaction) {
return provenanceaction; this.provenanceaction = provenanceaction;
} }
public void setProvenanceaction(Qualifier provenanceaction) { @Override
this.provenanceaction = provenanceaction; public boolean equals(Object o) {
} if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
DataInfo dataInfo = (DataInfo) o;
return Objects.equals(invisible, dataInfo.invisible)
&& Objects.equals(inferred, dataInfo.inferred)
&& Objects.equals(deletedbyinference, dataInfo.deletedbyinference)
&& Objects.equals(trust, dataInfo.trust)
&& Objects.equals(inferenceprovenance, dataInfo.inferenceprovenance)
&& Objects.equals(provenanceaction, dataInfo.provenanceaction);
}
@Override @Override
public boolean equals(Object o) { public int hashCode() {
if (this == o) return true; return Objects
if (o == null || getClass() != o.getClass()) return false; .hash(
DataInfo dataInfo = (DataInfo) o; invisible, inferred, deletedbyinference, trust, inferenceprovenance, provenanceaction);
return Objects.equals(invisible, dataInfo.invisible) && }
Objects.equals(inferred, dataInfo.inferred) &&
Objects.equals(deletedbyinference, dataInfo.deletedbyinference) &&
Objects.equals(trust, dataInfo.trust) &&
Objects.equals(inferenceprovenance, dataInfo.inferenceprovenance) &&
Objects.equals(provenanceaction, dataInfo.provenanceaction);
}
@Override
public int hashCode() {
return Objects.hash(invisible, inferred, deletedbyinference, trust, inferenceprovenance, provenanceaction);
}
} }

160
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Dataset.java
View File

@ -1,125 +1,115 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List; import java.util.List;
import java.util.Objects;
import eu.dnetlib.dhp.schema.common.ModelConstants;
public class Dataset extends Result implements Serializable { public class Dataset extends Result implements Serializable {
private Field<String> storagedate; private Field<String> storagedate;
private Field<String> device; private Field<String> device;
private Field<String> size; private Field<String> size;
private Field<String> version; private Field<String> version;
private Field<String> lastmetadataupdate; private Field<String> lastmetadataupdate;
private Field<String> metadataversionnumber; private Field<String> metadataversionnumber;
private List<GeoLocation> geolocation; private List<GeoLocation> geolocation;
public Field<String> getStoragedate() { public Dataset() {
return storagedate; setResulttype(ModelConstants.DATASET_DEFAULT_RESULTTYPE);
} }
public void setStoragedate(Field<String> storagedate) { public Field<String> getStoragedate() {
this.storagedate = storagedate; return storagedate;
} }
public Field<String> getDevice() { public void setStoragedate(Field<String> storagedate) {
return device; this.storagedate = storagedate;
} }
public void setDevice(Field<String> device) { public Field<String> getDevice() {
this.device = device; return device;
} }
public Field<String> getSize() { public void setDevice(Field<String> device) {
return size; this.device = device;
} }
public void setSize(Field<String> size) { public Field<String> getSize() {
this.size = size; return size;
} }
public Field<String> getVersion() { public void setSize(Field<String> size) {
return version; this.size = size;
} }
public void setVersion(Field<String> version) { public Field<String> getVersion() {
this.version = version; return version;
} }
public Field<String> getLastmetadataupdate() { public void setVersion(Field<String> version) {
return lastmetadataupdate; this.version = version;
} }
public void setLastmetadataupdate(Field<String> lastmetadataupdate) { public Field<String> getLastmetadataupdate() {
this.lastmetadataupdate = lastmetadataupdate; return lastmetadataupdate;
} }
public Field<String> getMetadataversionnumber() { public void setLastmetadataupdate(Field<String> lastmetadataupdate) {
return metadataversionnumber; this.lastmetadataupdate = lastmetadataupdate;
} }
public void setMetadataversionnumber(Field<String> metadataversionnumber) { public Field<String> getMetadataversionnumber() {
this.metadataversionnumber = metadataversionnumber; return metadataversionnumber;
} }
public List<GeoLocation> getGeolocation() { public void setMetadataversionnumber(Field<String> metadataversionnumber) {
return geolocation; this.metadataversionnumber = metadataversionnumber;
} }
public void setGeolocation(List<GeoLocation> geolocation) { public List<GeoLocation> getGeolocation() {
this.geolocation = geolocation; return geolocation;
} }
@Override public void setGeolocation(List<GeoLocation> geolocation) {
public void mergeFrom(OafEntity e) { this.geolocation = geolocation;
super.mergeFrom(e); }
if (!Dataset.class.isAssignableFrom(e.getClass())){ @Override
return; public void mergeFrom(OafEntity e) {
} super.mergeFrom(e);
final Dataset d = (Dataset) e; if (!Dataset.class.isAssignableFrom(e.getClass())) {
return;
}
storagedate = d.getStoragedate() != null && compareTrust(this, e)<0? d.getStoragedate() : storagedate; final Dataset d = (Dataset) e;
device= d.getDevice() != null && compareTrust(this, e)<0? d.getDevice() : device; storagedate = d.getStoragedate() != null && compareTrust(this, e) < 0 ? d.getStoragedate() : storagedate;
size= d.getSize() != null && compareTrust(this, e)<0? d.getSize() : size; device = d.getDevice() != null && compareTrust(this, e) < 0 ? d.getDevice() : device;
version= d.getVersion() != null && compareTrust(this, e)<0? d.getVersion() : version; size = d.getSize() != null && compareTrust(this, e) < 0 ? d.getSize() : size;
lastmetadataupdate= d.getLastmetadataupdate() != null && compareTrust(this, e)<0? d.getLastmetadataupdate() :lastmetadataupdate; version = d.getVersion() != null && compareTrust(this, e) < 0 ? d.getVersion() : version;
metadataversionnumber= d.getMetadataversionnumber() != null && compareTrust(this, e)<0? d.getMetadataversionnumber() : metadataversionnumber; lastmetadataupdate = d.getLastmetadataupdate() != null && compareTrust(this, e) < 0
? d.getLastmetadataupdate()
: lastmetadataupdate;
geolocation = mergeLists(geolocation, d.getGeolocation()); metadataversionnumber = d.getMetadataversionnumber() != null && compareTrust(this, e) < 0
? d.getMetadataversionnumber()
: metadataversionnumber;
mergeOAFDataInfo(d); geolocation = mergeLists(geolocation, d.getGeolocation());
}
@Override mergeOAFDataInfo(d);
public boolean equals(Object o) { }
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
if (!super.equals(o)) return false;
Dataset dataset = (Dataset) o;
return Objects.equals(storagedate, dataset.storagedate) &&
Objects.equals(device, dataset.device) &&
Objects.equals(size, dataset.size) &&
Objects.equals(version, dataset.version) &&
Objects.equals(lastmetadataupdate, dataset.lastmetadataupdate) &&
Objects.equals(metadataversionnumber, dataset.metadataversionnumber) &&
Objects.equals(geolocation, dataset.geolocation);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), storagedate, device, size, version, lastmetadataupdate, metadataversionnumber, geolocation);
}
} }

768
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Datasource.java
View File

@ -1,480 +1,472 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List; import java.util.List;
import java.util.Objects;
public class Datasource extends OafEntity implements Serializable { public class Datasource extends OafEntity implements Serializable {
private Qualifier datasourcetype; private Qualifier datasourcetype;
private Qualifier openairecompatibility; private Qualifier openairecompatibility;
private Field<String> officialname; private Field<String> officialname;
private Field<String> englishname; private Field<String> englishname;
private Field<String> websiteurl; private Field<String> websiteurl;
private Field<String> logourl; private Field<String> logourl;
private Field<String> contactemail; private Field<String> contactemail;
private Field<String> namespaceprefix; private Field<String> namespaceprefix;
private Field<String> latitude; private Field<String> latitude;
private Field<String> longitude; private Field<String> longitude;
private Field<String> dateofvalidation; private Field<String> dateofvalidation;
private Field<String> description; private Field<String> description;
private List<StructuredProperty> subjects; private List<StructuredProperty> subjects;
// opendoar specific fields (od*) // opendoar specific fields (od*)
private Field<String> odnumberofitems; private Field<String> odnumberofitems;
private Field<String> odnumberofitemsdate; private Field<String> odnumberofitemsdate;
private Field<String> odpolicies; private Field<String> odpolicies;
private List<Field<String>> odlanguages; private List<Field<String>> odlanguages;
private List<Field<String>> odcontenttypes; private List<Field<String>> odcontenttypes;
private List<Field<String>> accessinfopackage; private List<Field<String>> accessinfopackage;
// re3data fields // re3data fields
private Field<String> releasestartdate; private Field<String> releasestartdate;
private Field<String> releaseenddate; private Field<String> releaseenddate;
private Field<String> missionstatementurl; private Field<String> missionstatementurl;
private Field<Boolean> dataprovider; private Field<Boolean> dataprovider;
private Field<Boolean> serviceprovider; private Field<Boolean> serviceprovider;
// {open, restricted or closed} // {open, restricted or closed}
private Field<String> databaseaccesstype; private Field<String> databaseaccesstype;
// {open, restricted or closed} // {open, restricted or closed}
private Field<String> datauploadtype; private Field<String> datauploadtype;
// {feeRequired, registration, other} // {feeRequired, registration, other}
private Field<String> databaseaccessrestriction; private Field<String> databaseaccessrestriction;
// {feeRequired, registration, other} // {feeRequired, registration, other}
private Field<String> datauploadrestriction; private Field<String> datauploadrestriction;
private Field<Boolean> versioning; private Field<Boolean> versioning;
private Field<String> citationguidelineurl; private Field<String> citationguidelineurl;
//{yes, no, uknown} // {yes, no, uknown}
private Field<String> qualitymanagementkind; private Field<String> qualitymanagementkind;
private Field<String> pidsystems; private Field<String> pidsystems;
private Field<String> certificates; private Field<String> certificates;
private List<KeyValue> policies; private List<KeyValue> policies;
private Journal journal; private Journal journal;
public Qualifier getDatasourcetype() { public Qualifier getDatasourcetype() {
return datasourcetype; return datasourcetype;
} }
public void setDatasourcetype(Qualifier datasourcetype) { public void setDatasourcetype(Qualifier datasourcetype) {
this.datasourcetype = datasourcetype; this.datasourcetype = datasourcetype;
} }
public Qualifier getOpenairecompatibility() { public Qualifier getOpenairecompatibility() {
return openairecompatibility; return openairecompatibility;
} }
public void setOpenairecompatibility(Qualifier openairecompatibility) { public void setOpenairecompatibility(Qualifier openairecompatibility) {
this.openairecompatibility = openairecompatibility; this.openairecompatibility = openairecompatibility;
} }
public Field<String> getOfficialname() { public Field<String> getOfficialname() {
return officialname; return officialname;
} }
public void setOfficialname(Field<String> officialname) { public void setOfficialname(Field<String> officialname) {
this.officialname = officialname; this.officialname = officialname;
} }
public Field<String> getEnglishname() { public Field<String> getEnglishname() {
return englishname; return englishname;
} }
public void setEnglishname(Field<String> englishname) { public void setEnglishname(Field<String> englishname) {
this.englishname = englishname; this.englishname = englishname;
} }
public Field<String> getWebsiteurl() { public Field<String> getWebsiteurl() {
return websiteurl; return websiteurl;
} }
public void setWebsiteurl(Field<String> websiteurl) { public void setWebsiteurl(Field<String> websiteurl) {
this.websiteurl = websiteurl; this.websiteurl = websiteurl;
} }
public Field<String> getLogourl() { public Field<String> getLogourl() {
return logourl; return logourl;
} }
public void setLogourl(Field<String> logourl) { public void setLogourl(Field<String> logourl) {
this.logourl = logourl; this.logourl = logourl;
} }
public Field<String> getContactemail() { public Field<String> getContactemail() {
return contactemail; return contactemail;
} }
public void setContactemail(Field<String> contactemail) { public void setContactemail(Field<String> contactemail) {
this.contactemail = contactemail; this.contactemail = contactemail;
} }
public Field<String> getNamespaceprefix() { public Field<String> getNamespaceprefix() {
return namespaceprefix; return namespaceprefix;
} }
public void setNamespaceprefix(Field<String> namespaceprefix) { public void setNamespaceprefix(Field<String> namespaceprefix) {
this.namespaceprefix = namespaceprefix; this.namespaceprefix = namespaceprefix;
} }
public Field<String> getLatitude() { public Field<String> getLatitude() {
return latitude; return latitude;
} }
public void setLatitude(Field<String> latitude) { public void setLatitude(Field<String> latitude) {
this.latitude = latitude; this.latitude = latitude;
} }
public Field<String> getLongitude() { public Field<String> getLongitude() {
return longitude; return longitude;
} }
public void setLongitude(Field<String> longitude) { public void setLongitude(Field<String> longitude) {
this.longitude = longitude; this.longitude = longitude;
} }
public Field<String> getDateofvalidation() { public Field<String> getDateofvalidation() {
return dateofvalidation; return dateofvalidation;
} }
public void setDateofvalidation(Field<String> dateofvalidation) { public void setDateofvalidation(Field<String> dateofvalidation) {
this.dateofvalidation = dateofvalidation; this.dateofvalidation = dateofvalidation;
} }
public Field<String> getDescription() { public Field<String> getDescription() {
return description; return description;
} }
public void setDescription(Field<String> description) { public void setDescription(Field<String> description) {
this.description = description; this.description = description;
} }
public List<StructuredProperty> getSubjects() { public List<StructuredProperty> getSubjects() {
return subjects; return subjects;
} }
public void setSubjects(List<StructuredProperty> subjects) { public void setSubjects(List<StructuredProperty> subjects) {
this.subjects = subjects; this.subjects = subjects;
} }
public Field<String> getOdnumberofitems() { public Field<String> getOdnumberofitems() {
return odnumberofitems; return odnumberofitems;
} }
public void setOdnumberofitems(Field<String> odnumberofitems) { public void setOdnumberofitems(Field<String> odnumberofitems) {
this.odnumberofitems = odnumberofitems; this.odnumberofitems = odnumberofitems;
} }
public Field<String> getOdnumberofitemsdate() { public Field<String> getOdnumberofitemsdate() {
return odnumberofitemsdate; return odnumberofitemsdate;
} }
public void setOdnumberofitemsdate(Field<String> odnumberofitemsdate) { public void setOdnumberofitemsdate(Field<String> odnumberofitemsdate) {
this.odnumberofitemsdate = odnumberofitemsdate; this.odnumberofitemsdate = odnumberofitemsdate;
} }
public Field<String> getOdpolicies() { public Field<String> getOdpolicies() {
return odpolicies; return odpolicies;
} }
public void setOdpolicies(Field<String> odpolicies) { public void setOdpolicies(Field<String> odpolicies) {
this.odpolicies = odpolicies; this.odpolicies = odpolicies;
} }
public List<Field<String>> getOdlanguages() { public List<Field<String>> getOdlanguages() {
return odlanguages; return odlanguages;
} }
public void setOdlanguages(List<Field<String>> odlanguages) { public void setOdlanguages(List<Field<String>> odlanguages) {
this.odlanguages = odlanguages; this.odlanguages = odlanguages;
} }
public List<Field<String>> getOdcontenttypes() { public List<Field<String>> getOdcontenttypes() {
return odcontenttypes; return odcontenttypes;
} }
public void setOdcontenttypes(List<Field<String>> odcontenttypes) { public void setOdcontenttypes(List<Field<String>> odcontenttypes) {
this.odcontenttypes = odcontenttypes; this.odcontenttypes = odcontenttypes;
} }
public List<Field<String>> getAccessinfopackage() { public List<Field<String>> getAccessinfopackage() {
return accessinfopackage; return accessinfopackage;
} }
public void setAccessinfopackage(List<Field<String>> accessinfopackage) { public void setAccessinfopackage(List<Field<String>> accessinfopackage) {
this.accessinfopackage = accessinfopackage; this.accessinfopackage = accessinfopackage;
} }
public Field<String> getReleasestartdate() { public Field<String> getReleasestartdate() {
return releasestartdate; return releasestartdate;
} }
public void setReleasestartdate(Field<String> releasestartdate) { public void setReleasestartdate(Field<String> releasestartdate) {
this.releasestartdate = releasestartdate; this.releasestartdate = releasestartdate;
} }
public Field<String> getReleaseenddate() { public Field<String> getReleaseenddate() {
return releaseenddate; return releaseenddate;
} }
public void setReleaseenddate(Field<String> releaseenddate) { public void setReleaseenddate(Field<String> releaseenddate) {
this.releaseenddate = releaseenddate; this.releaseenddate = releaseenddate;
} }
public Field<String> getMissionstatementurl() { public Field<String> getMissionstatementurl() {
return missionstatementurl; return missionstatementurl;
} }
public void setMissionstatementurl(Field<String> missionstatementurl) { public void setMissionstatementurl(Field<String> missionstatementurl) {
this.missionstatementurl = missionstatementurl; this.missionstatementurl = missionstatementurl;
} }
public Field<Boolean> getDataprovider() { public Field<Boolean> getDataprovider() {
return dataprovider; return dataprovider;
} }
public void setDataprovider(Field<Boolean> dataprovider) { public void setDataprovider(Field<Boolean> dataprovider) {
this.dataprovider = dataprovider; this.dataprovider = dataprovider;
} }
public Field<Boolean> getServiceprovider() { public Field<Boolean> getServiceprovider() {
return serviceprovider; return serviceprovider;
} }
public void setServiceprovider(Field<Boolean> serviceprovider) { public void setServiceprovider(Field<Boolean> serviceprovider) {
this.serviceprovider = serviceprovider; this.serviceprovider = serviceprovider;
} }
public Field<String> getDatabaseaccesstype() { public Field<String> getDatabaseaccesstype() {
return databaseaccesstype; return databaseaccesstype;
} }
public void setDatabaseaccesstype(Field<String> databaseaccesstype) { public void setDatabaseaccesstype(Field<String> databaseaccesstype) {
this.databaseaccesstype = databaseaccesstype; this.databaseaccesstype = databaseaccesstype;
} }
public Field<String> getDatauploadtype() { public Field<String> getDatauploadtype() {
return datauploadtype; return datauploadtype;
} }
public void setDatauploadtype(Field<String> datauploadtype) { public void setDatauploadtype(Field<String> datauploadtype) {
this.datauploadtype = datauploadtype; this.datauploadtype = datauploadtype;
} }
public Field<String> getDatabaseaccessrestriction() { public Field<String> getDatabaseaccessrestriction() {
return databaseaccessrestriction; return databaseaccessrestriction;
} }
public void setDatabaseaccessrestriction(Field<String> databaseaccessrestriction) { public void setDatabaseaccessrestriction(Field<String> databaseaccessrestriction) {
this.databaseaccessrestriction = databaseaccessrestriction; this.databaseaccessrestriction = databaseaccessrestriction;
} }
public Field<String> getDatauploadrestriction() { public Field<String> getDatauploadrestriction() {
return datauploadrestriction; return datauploadrestriction;
} }
public void setDatauploadrestriction(Field<String> datauploadrestriction) { public void setDatauploadrestriction(Field<String> datauploadrestriction) {
this.datauploadrestriction = datauploadrestriction; this.datauploadrestriction = datauploadrestriction;
} }
public Field<Boolean> getVersioning() { public Field<Boolean> getVersioning() {
return versioning; return versioning;
} }
public void setVersioning(Field<Boolean> versioning) { public void setVersioning(Field<Boolean> versioning) {
this.versioning = versioning; this.versioning = versioning;
} }
public Field<String> getCitationguidelineurl() { public Field<String> getCitationguidelineurl() {
return citationguidelineurl; return citationguidelineurl;
} }
public void setCitationguidelineurl(Field<String> citationguidelineurl) { public void setCitationguidelineurl(Field<String> citationguidelineurl) {
this.citationguidelineurl = citationguidelineurl; this.citationguidelineurl = citationguidelineurl;
} }
public Field<String> getQualitymanagementkind() { public Field<String> getQualitymanagementkind() {
return qualitymanagementkind; return qualitymanagementkind;
} }
public void setQualitymanagementkind(Field<String> qualitymanagementkind) { public void setQualitymanagementkind(Field<String> qualitymanagementkind) {
this.qualitymanagementkind = qualitymanagementkind; this.qualitymanagementkind = qualitymanagementkind;
} }
public Field<String> getPidsystems() { public Field<String> getPidsystems() {
return pidsystems; return pidsystems;
} }
public void setPidsystems(Field<String> pidsystems) { public void setPidsystems(Field<String> pidsystems) {
this.pidsystems = pidsystems; this.pidsystems = pidsystems;
} }
public Field<String> getCertificates() { public Field<String> getCertificates() {
return certificates; return certificates;
} }
public void setCertificates(Field<String> certificates) { public void setCertificates(Field<String> certificates) {
this.certificates = certificates; this.certificates = certificates;
} }
public List<KeyValue> getPolicies() { public List<KeyValue> getPolicies() {
return policies; return policies;
} }
public void setPolicies(List<KeyValue> policies) { public void setPolicies(List<KeyValue> policies) {
this.policies = policies; this.policies = policies;
} }
public Journal getJournal() { public Journal getJournal() {
return journal; return journal;
} }
public void setJournal(Journal journal) { public void setJournal(Journal journal) {
this.journal = journal; this.journal = journal;
} }
@Override @Override
public void mergeFrom(OafEntity e) { public void mergeFrom(OafEntity e) {
super.mergeFrom(e); super.mergeFrom(e);
if (!Datasource.class.isAssignableFrom(e.getClass())){ if (!Datasource.class.isAssignableFrom(e.getClass())) {
return; return;
} }
Datasource d = (Datasource)e; Datasource d = (Datasource) e;
datasourcetype = d.getDatasourcetype() != null && compareTrust(this, e)<0? d.getDatasourcetype() : datasourcetype; datasourcetype = d.getDatasourcetype() != null && compareTrust(this, e) < 0
openairecompatibility = d.getOpenairecompatibility() != null && compareTrust(this, e)<0? d.getOpenairecompatibility() : openairecompatibility; ? d.getDatasourcetype()
officialname = d.getOfficialname() != null && compareTrust(this, e)<0? d.getOfficialname() : officialname; : datasourcetype;
englishname = d.getEnglishname() != null && compareTrust(this, e)<0? d.getEnglishname() : officialname; openairecompatibility = d.getOpenairecompatibility() != null && compareTrust(this, e) < 0
websiteurl = d.getWebsiteurl() != null && compareTrust(this, e)<0? d.getWebsiteurl() : websiteurl; ? d.getOpenairecompatibility()
logourl = d.getLogourl() != null && compareTrust(this, e)<0? d.getLogourl() : getLogourl(); : openairecompatibility;
contactemail = d.getContactemail() != null && compareTrust(this, e)<0? d.getContactemail() : contactemail; officialname = d.getOfficialname() != null && compareTrust(this, e) < 0
namespaceprefix = d.getNamespaceprefix() != null && compareTrust(this, e)<0? d.getNamespaceprefix() : namespaceprefix; ? d.getOfficialname()
latitude = d.getLatitude() != null && compareTrust(this, e)<0? d.getLatitude() : latitude; : officialname;
longitude = d.getLongitude() != null && compareTrust(this, e)<0? d.getLongitude() : longitude; englishname = d.getEnglishname() != null && compareTrust(this, e) < 0 ? d.getEnglishname() : officialname;
dateofvalidation = d.getDateofvalidation() != null && compareTrust(this, e)<0? d.getDateofvalidation() : dateofvalidation; websiteurl = d.getWebsiteurl() != null && compareTrust(this, e) < 0 ? d.getWebsiteurl() : websiteurl;
description = d.getDescription() != null && compareTrust(this, e)<0? d.getDescription() : description; logourl = d.getLogourl() != null && compareTrust(this, e) < 0 ? d.getLogourl() : getLogourl();
subjects = mergeLists(subjects, d.getSubjects()); contactemail = d.getContactemail() != null && compareTrust(this, e) < 0
? d.getContactemail()
// opendoar specific fields (od*) : contactemail;
odnumberofitems = d.getOdnumberofitems() != null && compareTrust(this, e)<0? d.getOdnumberofitems() : odnumberofitems; namespaceprefix = d.getNamespaceprefix() != null && compareTrust(this, e) < 0
odnumberofitemsdate = d.getOdnumberofitemsdate() != null && compareTrust(this, e)<0? d.getOdnumberofitemsdate() : odnumberofitemsdate; ? d.getNamespaceprefix()
odpolicies = d.getOdpolicies() != null && compareTrust(this, e)<0? d.getOdpolicies() : odpolicies; : namespaceprefix;
odlanguages = mergeLists(odlanguages, d.getOdlanguages()); latitude = d.getLatitude() != null && compareTrust(this, e) < 0 ? d.getLatitude() : latitude;
odcontenttypes = mergeLists(odcontenttypes, d.getOdcontenttypes()); longitude = d.getLongitude() != null && compareTrust(this, e) < 0 ? d.getLongitude() : longitude;
accessinfopackage = mergeLists(accessinfopackage, d.getAccessinfopackage()); dateofvalidation = d.getDateofvalidation() != null && compareTrust(this, e) < 0
? d.getDateofvalidation()
// re3data fields : dateofvalidation;
releasestartdate = d.getReleasestartdate() != null && compareTrust(this, e)<0? d.getReleasestartdate() : releasestartdate; description = d.getDescription() != null && compareTrust(this, e) < 0 ? d.getDescription() : description;
releaseenddate = d.getReleaseenddate() != null && compareTrust(this, e)<0? d.getReleaseenddate() : releaseenddate; subjects = mergeLists(subjects, d.getSubjects());
missionstatementurl = d.getMissionstatementurl() != null && compareTrust(this, e)<0? d.getMissionstatementurl() : missionstatementurl;
dataprovider = d.getDataprovider() != null && compareTrust(this, e)<0? d.getDataprovider() : dataprovider; // opendoar specific fields (od*)
serviceprovider = d.getServiceprovider() != null && compareTrust(this, e)<0? d.getServiceprovider() : serviceprovider; odnumberofitems = d.getOdnumberofitems() != null && compareTrust(this, e) < 0
? d.getOdnumberofitems()
// {open, restricted or closed} : odnumberofitems;
databaseaccesstype = d.getDatabaseaccesstype() != null && compareTrust(this, e)<0? d.getDatabaseaccesstype() : databaseaccesstype; odnumberofitemsdate = d.getOdnumberofitemsdate() != null && compareTrust(this, e) < 0
? d.getOdnumberofitemsdate()
// {open, restricted or closed} : odnumberofitemsdate;
datauploadtype = d.getDatauploadtype() != null && compareTrust(this, e)<0? d.getDatauploadtype() : datauploadtype; odpolicies = d.getOdpolicies() != null && compareTrust(this, e) < 0 ? d.getOdpolicies() : odpolicies;
odlanguages = mergeLists(odlanguages, d.getOdlanguages());
// {feeRequired, registration, other} odcontenttypes = mergeLists(odcontenttypes, d.getOdcontenttypes());
databaseaccessrestriction = d.getDatabaseaccessrestriction() != null && compareTrust(this, e)<0? d.getDatabaseaccessrestriction() : databaseaccessrestriction; accessinfopackage = mergeLists(accessinfopackage, d.getAccessinfopackage());
// {feeRequired, registration, other} // re3data fields
datauploadrestriction = d.getDatauploadrestriction() != null && compareTrust(this, e)<0? d.getDatauploadrestriction() : datauploadrestriction; releasestartdate = d.getReleasestartdate() != null && compareTrust(this, e) < 0
? d.getReleasestartdate()
versioning = d.getVersioning() != null && compareTrust(this, e)<0? d.getVersioning() : versioning; : releasestartdate;
citationguidelineurl = d.getCitationguidelineurl() != null && compareTrust(this, e)<0? d.getCitationguidelineurl() : citationguidelineurl; releaseenddate = d.getReleaseenddate() != null && compareTrust(this, e) < 0
? d.getReleaseenddate()
//{yes, no, unknown} : releaseenddate;
qualitymanagementkind = d.getQualitymanagementkind() != null && compareTrust(this, e)<0? d.getQualitymanagementkind() : qualitymanagementkind; missionstatementurl = d.getMissionstatementurl() != null && compareTrust(this, e) < 0
pidsystems = d.getPidsystems() != null && compareTrust(this, e)<0? d.getPidsystems() : pidsystems; ? d.getMissionstatementurl()
: missionstatementurl;
certificates = d.getCertificates() != null && compareTrust(this, e)<0? d.getCertificates() : certificates; dataprovider = d.getDataprovider() != null && compareTrust(this, e) < 0
? d.getDataprovider()
policies = mergeLists(policies, d.getPolicies()); : dataprovider;
serviceprovider = d.getServiceprovider() != null && compareTrust(this, e) < 0
journal = d.getJournal() != null && compareTrust(this, e)<0? d.getJournal() : journal; ? d.getServiceprovider()
: serviceprovider;
mergeOAFDataInfo(e);
} // {open, restricted or closed}
databaseaccesstype = d.getDatabaseaccesstype() != null && compareTrust(this, e) < 0
@Override ? d.getDatabaseaccesstype()
public boolean equals(Object o) { : databaseaccesstype;
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false; // {open, restricted or closed}
if (!super.equals(o)) return false; datauploadtype = d.getDatauploadtype() != null && compareTrust(this, e) < 0
Datasource that = (Datasource) o; ? d.getDatauploadtype()
return Objects.equals(datasourcetype, that.datasourcetype) && : datauploadtype;
Objects.equals(openairecompatibility, that.openairecompatibility) &&
Objects.equals(officialname, that.officialname) && // {feeRequired, registration, other}
Objects.equals(englishname, that.englishname) && databaseaccessrestriction = d.getDatabaseaccessrestriction() != null && compareTrust(this, e) < 0
Objects.equals(websiteurl, that.websiteurl) && ? d.getDatabaseaccessrestriction()
Objects.equals(logourl, that.logourl) && : databaseaccessrestriction;
Objects.equals(contactemail, that.contactemail) &&
Objects.equals(namespaceprefix, that.namespaceprefix) && // {feeRequired, registration, other}
Objects.equals(latitude, that.latitude) && datauploadrestriction = d.getDatauploadrestriction() != null && compareTrust(this, e) < 0
Objects.equals(longitude, that.longitude) && ? d.getDatauploadrestriction()
Objects.equals(dateofvalidation, that.dateofvalidation) && : datauploadrestriction;
Objects.equals(description, that.description) &&
Objects.equals(subjects, that.subjects) && versioning = d.getVersioning() != null && compareTrust(this, e) < 0 ? d.getVersioning() : versioning;
Objects.equals(odnumberofitems, that.odnumberofitems) && citationguidelineurl = d.getCitationguidelineurl() != null && compareTrust(this, e) < 0
Objects.equals(odnumberofitemsdate, that.odnumberofitemsdate) && ? d.getCitationguidelineurl()
Objects.equals(odpolicies, that.odpolicies) && : citationguidelineurl;
Objects.equals(odlanguages, that.odlanguages) &&
Objects.equals(odcontenttypes, that.odcontenttypes) && // {yes, no, unknown}
Objects.equals(accessinfopackage, that.accessinfopackage) && qualitymanagementkind = d.getQualitymanagementkind() != null && compareTrust(this, e) < 0
Objects.equals(releasestartdate, that.releasestartdate) && ? d.getQualitymanagementkind()
Objects.equals(releaseenddate, that.releaseenddate) && : qualitymanagementkind;
Objects.equals(missionstatementurl, that.missionstatementurl) && pidsystems = d.getPidsystems() != null && compareTrust(this, e) < 0 ? d.getPidsystems() : pidsystems;
Objects.equals(dataprovider, that.dataprovider) &&
Objects.equals(serviceprovider, that.serviceprovider) && certificates = d.getCertificates() != null && compareTrust(this, e) < 0
Objects.equals(databaseaccesstype, that.databaseaccesstype) && ? d.getCertificates()
Objects.equals(datauploadtype, that.datauploadtype) && : certificates;
Objects.equals(databaseaccessrestriction, that.databaseaccessrestriction) &&
Objects.equals(datauploadrestriction, that.datauploadrestriction) && policies = mergeLists(policies, d.getPolicies());
Objects.equals(versioning, that.versioning) &&
Objects.equals(citationguidelineurl, that.citationguidelineurl) && journal = d.getJournal() != null && compareTrust(this, e) < 0 ? d.getJournal() : journal;
Objects.equals(qualitymanagementkind, that.qualitymanagementkind) &&
Objects.equals(pidsystems, that.pidsystems) && mergeOAFDataInfo(e);
Objects.equals(certificates, that.certificates) && }
Objects.equals(policies, that.policies) &&
Objects.equals(journal, that.journal);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), datasourcetype, openairecompatibility, officialname, englishname, websiteurl, logourl, contactemail, namespaceprefix, latitude, longitude, dateofvalidation, description, subjects, odnumberofitems, odnumberofitemsdate, odpolicies, odlanguages, odcontenttypes, accessinfopackage, releasestartdate, releaseenddate, missionstatementurl, dataprovider, serviceprovider, databaseaccesstype, datauploadtype, databaseaccessrestriction, datauploadrestriction, versioning, citationguidelineurl, qualitymanagementkind, pidsystems, certificates, policies, journal);
}
} }

169
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/ExternalReference.java
View File

@ -1,114 +1,119 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.Objects; import java.util.Objects;
public class ExternalReference implements Serializable { public class ExternalReference implements Serializable {
// source // source
private String sitename; private String sitename;
// title // title
private String label; private String label;
// text() // text()
private String url; private String url;
// ?? not mapped yet ?? // ?? not mapped yet ??
private String description; private String description;
// type // type
private Qualifier qualifier; private Qualifier qualifier;
// site internal identifier // site internal identifier
private String refidentifier; private String refidentifier;
// maps the oaf:reference/@query attribute // maps the oaf:reference/@query attribute
private String query; private String query;
// ExternalReferences might be also inferred // ExternalReferences might be also inferred
private DataInfo dataInfo; private DataInfo dataInfo;
public String getSitename() { public String getSitename() {
return sitename; return sitename;
} }
public void setSitename(String sitename) { public void setSitename(String sitename) {
this.sitename = sitename; this.sitename = sitename;
} }
public String getLabel() { public String getLabel() {
return label; return label;
} }
public void setLabel(String label) { public void setLabel(String label) {
this.label = label; this.label = label;
} }
public String getUrl() { public String getUrl() {
return url; return url;
} }
public void setUrl(String url) { public void setUrl(String url) {
this.url = url; this.url = url;
} }
public String getDescription() { public String getDescription() {
return description; return description;
} }
public void setDescription(String description) { public void setDescription(String description) {
this.description = description; this.description = description;
} }
public Qualifier getQualifier() { public Qualifier getQualifier() {
return qualifier; return qualifier;
} }
public void setQualifier(Qualifier qualifier) { public void setQualifier(Qualifier qualifier) {
this.qualifier = qualifier; this.qualifier = qualifier;
} }
public String getRefidentifier() { public String getRefidentifier() {
return refidentifier; return refidentifier;
} }
public void setRefidentifier(String refidentifier) { public void setRefidentifier(String refidentifier) {
this.refidentifier = refidentifier; this.refidentifier = refidentifier;
} }
public String getQuery() { public String getQuery() {
return query; return query;
} }
public void setQuery(String query) { public void setQuery(String query) {
this.query = query; this.query = query;
} }
public DataInfo getDataInfo() { public DataInfo getDataInfo() {
return dataInfo; return dataInfo;
} }
public void setDataInfo(DataInfo dataInfo) { public void setDataInfo(DataInfo dataInfo) {
this.dataInfo = dataInfo; this.dataInfo = dataInfo;
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
ExternalReference that = (ExternalReference) o; if (o == null || getClass() != o.getClass())
return Objects.equals(sitename, that.sitename) && return false;
Objects.equals(label, that.label) && ExternalReference that = (ExternalReference) o;
Objects.equals(url, that.url) && return Objects.equals(sitename, that.sitename)
Objects.equals(description, that.description) && && Objects.equals(label, that.label)
Objects.equals(qualifier, that.qualifier) && && Objects.equals(url, that.url)
Objects.equals(refidentifier, that.refidentifier) && && Objects.equals(description, that.description)
Objects.equals(query, that.query) && && Objects.equals(qualifier, that.qualifier)
Objects.equals(dataInfo, that.dataInfo); && Objects.equals(refidentifier, that.refidentifier)
} && Objects.equals(query, that.query)
&& Objects.equals(dataInfo, that.dataInfo);
}
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(sitename, label, url, description, qualifier, refidentifier, query, dataInfo); return Objects
} .hash(
sitename, label, url, description, qualifier, refidentifier, query, dataInfo);
}
} }

105
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/ExtraInfo.java
View File

@ -1,74 +1,77 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.Objects; import java.util.Objects;
public class ExtraInfo implements Serializable { public class ExtraInfo implements Serializable {
private String name; private String name;
private String typology; private String typology;
private String provenance; private String provenance;
private String trust; private String trust;
// json containing a Citation or Statistics // json containing a Citation or Statistics
private String value; private String value;
public String getName() { public String getName() {
return name; return name;
} }
public void setName(String name) { public void setName(String name) {
this.name = name; this.name = name;
} }
public String getTypology() { public String getTypology() {
return typology; return typology;
} }
public void setTypology(String typology) { public void setTypology(String typology) {
this.typology = typology; this.typology = typology;
} }
public String getProvenance() { public String getProvenance() {
return provenance; return provenance;
} }
public void setProvenance(String provenance) { public void setProvenance(String provenance) {
this.provenance = provenance; this.provenance = provenance;
} }
public String getTrust() { public String getTrust() {
return trust; return trust;
} }
public void setTrust(String trust) { public void setTrust(String trust) {
this.trust = trust; this.trust = trust;
} }
public String getValue() { public String getValue() {
return value; return value;
} }
public void setValue(String value) { public void setValue(String value) {
this.value = value; this.value = value;
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
ExtraInfo extraInfo = (ExtraInfo) o; if (o == null || getClass() != o.getClass())
return Objects.equals(name, extraInfo.name) && return false;
Objects.equals(typology, extraInfo.typology) && ExtraInfo extraInfo = (ExtraInfo) o;
Objects.equals(provenance, extraInfo.provenance) && return Objects.equals(name, extraInfo.name)
Objects.equals(trust, extraInfo.trust) && && Objects.equals(typology, extraInfo.typology)
Objects.equals(value, extraInfo.value); && Objects.equals(provenance, extraInfo.provenance)
} && Objects.equals(trust, extraInfo.trust)
&& Objects.equals(value, extraInfo.value);
}
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(name, typology, provenance, trust, value); return Objects.hash(name, typology, provenance, trust, value);
} }
} }

59
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Field.java
View File

@ -1,43 +1,44 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
public class Field<T> implements Serializable { public class Field<T> implements Serializable {
private T value; private T value;
private DataInfo dataInfo; private DataInfo dataInfo;
public T getValue() { public T getValue() {
return value; return value;
} }
public void setValue(T value) { public void setValue(T value) {
this.value = value; this.value = value;
} }
public DataInfo getDataInfo() { public DataInfo getDataInfo() {
return dataInfo; return dataInfo;
} }
public void setDataInfo(DataInfo dataInfo) { public void setDataInfo(DataInfo dataInfo) {
this.dataInfo = dataInfo; this.dataInfo = dataInfo;
} }
@Override @Override
public int hashCode() { public int hashCode() {
return getValue() == null ? 0 : getValue().hashCode(); return getValue() == null ? 0 : getValue().hashCode();
} }
@Override @Override
public boolean equals(Object obj) { public boolean equals(Object obj) {
if (this == obj) if (this == obj)
return true; return true;
if (obj == null) if (obj == null)
return false; return false;
if (getClass() != obj.getClass()) if (getClass() != obj.getClass())
return false; return false;
Field<T> other = (Field<T>) obj; Field<T> other = (Field<T>) obj;
return getValue().equals(other.getValue()); return getValue().equals(other.getValue());
} }
} }

104
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/GeoLocation.java
View File

@ -1,70 +1,76 @@
package eu.dnetlib.dhp.schema.oaf;
import com.fasterxml.jackson.annotation.JsonIgnore; package eu.dnetlib.dhp.schema.oaf;
import org.apache.commons.lang3.StringUtils;
import java.io.Serializable; import java.io.Serializable;
import org.apache.commons.lang3.StringUtils;
import com.fasterxml.jackson.annotation.JsonIgnore;
public class GeoLocation implements Serializable { public class GeoLocation implements Serializable {
private String point; private String point;
private String box; private String box;
private String place; private String place;
public String getPoint() { public String getPoint() {
return point; return point;
} }
public void setPoint(String point) { public void setPoint(String point) {
this.point = point; this.point = point;
} }
public String getBox() { public String getBox() {
return box; return box;
} }
public void setBox(String box) { public void setBox(String box) {
this.box = box; this.box = box;
} }
public String getPlace() { public String getPlace() {
return place; return place;
} }
public void setPlace(String place) { public void setPlace(String place) {
this.place = place; this.place = place;
} }
@JsonIgnore @JsonIgnore
public boolean isBlank() { public boolean isBlank() {
return StringUtils.isBlank(point) && return StringUtils.isBlank(point) && StringUtils.isBlank(box) && StringUtils.isBlank(place);
StringUtils.isBlank(box) && }
StringUtils.isBlank(place);
}
public String toComparableString() { public String toComparableString() {
return isBlank()?"":String.format("%s::%s%s", point != null ? point.toLowerCase() : "", box != null ? box.toLowerCase() : "", place != null ? place.toLowerCase() : ""); return isBlank()
} ? ""
: String
.format(
"%s::%s%s",
point != null ? point.toLowerCase() : "",
box != null ? box.toLowerCase() : "",
place != null ? place.toLowerCase() : "");
}
@Override @Override
public int hashCode() { public int hashCode() {
return toComparableString().hashCode(); return toComparableString().hashCode();
} }
@Override @Override
public boolean equals(Object obj) { public boolean equals(Object obj) {
if (this == obj) if (this == obj)
return true; return true;
if (obj == null) if (obj == null)
return false; return false;
if (getClass() != obj.getClass()) if (getClass() != obj.getClass())
return false; return false;
GeoLocation other = (GeoLocation) obj; GeoLocation other = (GeoLocation) obj;
return toComparableString() return toComparableString().equals(other.toComparableString());
.equals(other.toComparableString()); }
}
} }

210
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Instance.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
@ -5,144 +6,147 @@ import java.util.List;
public class Instance implements Serializable { public class Instance implements Serializable {
private Field<String> license; private Field<String> license;
private Qualifier accessright; private Qualifier accessright;
private Qualifier instancetype; private Qualifier instancetype;
private KeyValue hostedby; private KeyValue hostedby;
private List<String> url; private List<String> url;
// other research products specifc // other research products specifc
private String distributionlocation; private String distributionlocation;
private KeyValue collectedfrom; private KeyValue collectedfrom;
private Field<String> dateofacceptance; private Field<String> dateofacceptance;
// ( article | book ) processing charges. Defined here to cope with possible wrongly typed results // ( article | book ) processing charges. Defined here to cope with possible wrongly typed
private Field<String> processingchargeamount; // results
private Field<String> processingchargeamount;
// currency - alphabetic code describe in ISO-4217. Defined here to cope with possible wrongly typed results // currency - alphabetic code describe in ISO-4217. Defined here to cope with possible wrongly
private Field<String> processingchargecurrency; // typed results
private Field<String> processingchargecurrency;
private Field<String> refereed; //peer-review status private Field<String> refereed; // peer-review status
public Field<String> getLicense() { public Field<String> getLicense() {
return license; return license;
} }
public void setLicense(Field<String> license) { public void setLicense(Field<String> license) {
this.license = license; this.license = license;
} }
public Qualifier getAccessright() { public Qualifier getAccessright() {
return accessright; return accessright;
} }
public void setAccessright(Qualifier accessright) { public void setAccessright(Qualifier accessright) {
this.accessright = accessright; this.accessright = accessright;
} }
public Qualifier getInstancetype() { public Qualifier getInstancetype() {
return instancetype; return instancetype;
} }
public void setInstancetype(Qualifier instancetype) { public void setInstancetype(Qualifier instancetype) {
this.instancetype = instancetype; this.instancetype = instancetype;
} }
public KeyValue getHostedby() { public KeyValue getHostedby() {
return hostedby; return hostedby;
} }
public void setHostedby(KeyValue hostedby) { public void setHostedby(KeyValue hostedby) {
this.hostedby = hostedby; this.hostedby = hostedby;
} }
public List<String> getUrl() { public List<String> getUrl() {
return url; return url;
} }
public void setUrl(List<String> url) { public void setUrl(List<String> url) {
this.url = url; this.url = url;
} }
public String getDistributionlocation() { public String getDistributionlocation() {
return distributionlocation; return distributionlocation;
} }
public void setDistributionlocation(String distributionlocation) { public void setDistributionlocation(String distributionlocation) {
this.distributionlocation = distributionlocation; this.distributionlocation = distributionlocation;
} }
public KeyValue getCollectedfrom() { public KeyValue getCollectedfrom() {
return collectedfrom; return collectedfrom;
} }
public void setCollectedfrom(KeyValue collectedfrom) { public void setCollectedfrom(KeyValue collectedfrom) {
this.collectedfrom = collectedfrom; this.collectedfrom = collectedfrom;
} }
public Field<String> getDateofacceptance() { public Field<String> getDateofacceptance() {
return dateofacceptance; return dateofacceptance;
} }
public void setDateofacceptance(Field<String> dateofacceptance) { public void setDateofacceptance(Field<String> dateofacceptance) {
this.dateofacceptance = dateofacceptance; this.dateofacceptance = dateofacceptance;
} }
public Field<String> getProcessingchargeamount() { public Field<String> getProcessingchargeamount() {
return processingchargeamount; return processingchargeamount;
} }
public void setProcessingchargeamount(Field<String> processingchargeamount) { public void setProcessingchargeamount(Field<String> processingchargeamount) {
this.processingchargeamount = processingchargeamount; this.processingchargeamount = processingchargeamount;
} }
public Field<String> getProcessingchargecurrency() { public Field<String> getProcessingchargecurrency() {
return processingchargecurrency; return processingchargecurrency;
} }
public void setProcessingchargecurrency(Field<String> processingchargecurrency) { public void setProcessingchargecurrency(Field<String> processingchargecurrency) {
this.processingchargecurrency = processingchargecurrency; this.processingchargecurrency = processingchargecurrency;
} }
public Field<String> getRefereed() { public Field<String> getRefereed() {
return refereed; return refereed;
} }
public void setRefereed(Field<String> refereed) { public void setRefereed(Field<String> refereed) {
this.refereed = refereed; this.refereed = refereed;
} }
public String toComparableString(){ public String toComparableString() {
return String.format("%s::%s::%s::%s", return String
hostedby != null && hostedby.getKey()!= null ? hostedby.getKey().toLowerCase() : "", .format(
accessright!= null && accessright.getClassid()!= null ? accessright.getClassid() : "", "%s::%s::%s::%s",
instancetype!= null && instancetype.getClassid()!= null ? instancetype.getClassid() : "", hostedby != null && hostedby.getKey() != null ? hostedby.getKey().toLowerCase() : "",
url != null ? url:""); accessright != null && accessright.getClassid() != null ? accessright.getClassid() : "",
} instancetype != null && instancetype.getClassid() != null ? instancetype.getClassid() : "",
url != null ? url : "");
}
@Override @Override
public int hashCode() { public int hashCode() {
return toComparableString().hashCode(); return toComparableString().hashCode();
} }
@Override @Override
public boolean equals(Object obj) { public boolean equals(Object obj) {
if (this == obj) if (this == obj)
return true; return true;
if (obj == null) if (obj == null)
return false; return false;
if (getClass() != obj.getClass()) if (getClass() != obj.getClass())
return false; return false;
Instance other = (Instance) obj; Instance other = (Instance) obj;
return toComparableString() return toComparableString().equals(other.toComparableString());
.equals(other.toComparableString()); }
}
} }

228
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Journal.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
@ -5,147 +6,162 @@ import java.util.Objects;
public class Journal implements Serializable { public class Journal implements Serializable {
private String name; private String name;
private String issnPrinted; private String issnPrinted;
private String issnOnline; private String issnOnline;
private String issnLinking; private String issnLinking;
private String ep; private String ep;
private String iss; private String iss;
private String sp; private String sp;
private String vol; private String vol;
private String edition; private String edition;
private String conferenceplace; private String conferenceplace;
private String conferencedate; private String conferencedate;
private DataInfo dataInfo; private DataInfo dataInfo;
public String getName() { public String getName() {
return name; return name;
} }
public void setName(String name) { public void setName(String name) {
this.name = name; this.name = name;
} }
public String getIssnPrinted() { public String getIssnPrinted() {
return issnPrinted; return issnPrinted;
} }
public void setIssnPrinted(String issnPrinted) { public void setIssnPrinted(String issnPrinted) {
this.issnPrinted = issnPrinted; this.issnPrinted = issnPrinted;
} }
public String getIssnOnline() { public String getIssnOnline() {
return issnOnline; return issnOnline;
} }
public void setIssnOnline(String issnOnline) { public void setIssnOnline(String issnOnline) {
this.issnOnline = issnOnline; this.issnOnline = issnOnline;
} }
public String getIssnLinking() { public String getIssnLinking() {
return issnLinking; return issnLinking;
} }
public void setIssnLinking(String issnLinking) { public void setIssnLinking(String issnLinking) {
this.issnLinking = issnLinking; this.issnLinking = issnLinking;
} }
public String getEp() { public String getEp() {
return ep; return ep;
} }
public void setEp(String ep) { public void setEp(String ep) {
this.ep = ep; this.ep = ep;
} }
public String getIss() { public String getIss() {
return iss; return iss;
} }
public void setIss(String iss) { public void setIss(String iss) {
this.iss = iss; this.iss = iss;
} }
public String getSp() { public String getSp() {
return sp; return sp;
} }
public void setSp(String sp) { public void setSp(String sp) {
this.sp = sp; this.sp = sp;
} }
public String getVol() { public String getVol() {
return vol; return vol;
} }
public void setVol(String vol) { public void setVol(String vol) {
this.vol = vol; this.vol = vol;
} }
public String getEdition() { public String getEdition() {
return edition; return edition;
} }
public void setEdition(String edition) { public void setEdition(String edition) {
this.edition = edition; this.edition = edition;
} }
public String getConferenceplace() { public String getConferenceplace() {
return conferenceplace; return conferenceplace;
} }
public void setConferenceplace(String conferenceplace) { public void setConferenceplace(String conferenceplace) {
this.conferenceplace = conferenceplace; this.conferenceplace = conferenceplace;
} }
public String getConferencedate() { public String getConferencedate() {
return conferencedate; return conferencedate;
} }
public void setConferencedate(String conferencedate) { public void setConferencedate(String conferencedate) {
this.conferencedate = conferencedate; this.conferencedate = conferencedate;
} }
public DataInfo getDataInfo() { public DataInfo getDataInfo() {
return dataInfo; return dataInfo;
} }
public void setDataInfo(DataInfo dataInfo) { public void setDataInfo(DataInfo dataInfo) {
this.dataInfo = dataInfo; this.dataInfo = dataInfo;
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
Journal journal = (Journal) o; if (o == null || getClass() != o.getClass())
return Objects.equals(name, journal.name) && return false;
Objects.equals(issnPrinted, journal.issnPrinted) && Journal journal = (Journal) o;
Objects.equals(issnOnline, journal.issnOnline) && return Objects.equals(name, journal.name)
Objects.equals(issnLinking, journal.issnLinking) && && Objects.equals(issnPrinted, journal.issnPrinted)
Objects.equals(ep, journal.ep) && && Objects.equals(issnOnline, journal.issnOnline)
Objects.equals(iss, journal.iss) && && Objects.equals(issnLinking, journal.issnLinking)
Objects.equals(sp, journal.sp) && && Objects.equals(ep, journal.ep)
Objects.equals(vol, journal.vol) && && Objects.equals(iss, journal.iss)
Objects.equals(edition, journal.edition) && && Objects.equals(sp, journal.sp)
Objects.equals(conferenceplace, journal.conferenceplace) && && Objects.equals(vol, journal.vol)
Objects.equals(conferencedate, journal.conferencedate) && && Objects.equals(edition, journal.edition)
Objects.equals(dataInfo, journal.dataInfo); && Objects.equals(conferenceplace, journal.conferenceplace)
} && Objects.equals(conferencedate, journal.conferencedate)
&& Objects.equals(dataInfo, journal.dataInfo);
}
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(name, issnPrinted, issnOnline, issnLinking, ep, iss, sp, vol, edition, conferenceplace, conferencedate, dataInfo); return Objects
} .hash(
name,
issnPrinted,
issnOnline,
issnLinking,
ep,
iss,
sp,
vol,
edition,
conferenceplace,
conferencedate,
dataInfo);
}
} }

99
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/KeyValue.java
View File

@ -1,67 +1,74 @@
package eu.dnetlib.dhp.schema.oaf;
import com.fasterxml.jackson.annotation.JsonIgnore; package eu.dnetlib.dhp.schema.oaf;
import org.apache.commons.lang3.StringUtils;
import java.io.Serializable; import java.io.Serializable;
import org.apache.commons.lang3.StringUtils;
import com.fasterxml.jackson.annotation.JsonIgnore;
public class KeyValue implements Serializable { public class KeyValue implements Serializable {
private String key; private String key;
private String value; private String value;
private DataInfo dataInfo; private DataInfo dataInfo;
public String getKey() { public String getKey() {
return key; return key;
} }
public void setKey(String key) { public void setKey(String key) {
this.key = key; this.key = key;
} }
public String getValue() { public String getValue() {
return value; return value;
} }
public void setValue(String value) { public void setValue(String value) {
this.value = value; this.value = value;
} }
public DataInfo getDataInfo() { public DataInfo getDataInfo() {
return dataInfo; return dataInfo;
} }
public void setDataInfo(DataInfo dataInfo) { public void setDataInfo(DataInfo dataInfo) {
this.dataInfo = dataInfo; this.dataInfo = dataInfo;
} }
public String toComparableString() { public String toComparableString() {
return isBlank()?"":String.format("%s::%s", key != null ? key.toLowerCase() : "", value != null ? value.toLowerCase() : ""); return isBlank()
} ? ""
: String
.format(
"%s::%s",
key != null ? key.toLowerCase() : "", value != null ? value.toLowerCase() : "");
}
@JsonIgnore @JsonIgnore
public boolean isBlank() { public boolean isBlank() {
return StringUtils.isBlank(key) && StringUtils.isBlank(value); return StringUtils.isBlank(key) && StringUtils.isBlank(value);
} }
@Override @Override
public int hashCode() { public int hashCode() {
return toComparableString().hashCode(); return toComparableString().hashCode();
} }
@Override @Override
public boolean equals(Object obj) { public boolean equals(Object obj) {
if (this == obj) if (this == obj)
return true; return true;
if (obj == null) if (obj == null)
return false; return false;
if (getClass() != obj.getClass()) if (getClass() != obj.getClass())
return false; return false;
KeyValue other = (KeyValue) obj; KeyValue other = (KeyValue) obj;
return toComparableString().equals(other.toComparableString()); return toComparableString().equals(other.toComparableString());
} }
} }

4
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Measure.java Normal file
View File

@ -0,0 +1,4 @@
package eu.dnetlib.dhp.schema.oaf;
public class Measure {
}

41
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/OAIProvenance.java
View File

@ -1,30 +1,33 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.Objects; import java.util.Objects;
public class OAIProvenance implements Serializable { public class OAIProvenance implements Serializable {
private OriginDescription originDescription; private OriginDescription originDescription;
public OriginDescription getOriginDescription() { public OriginDescription getOriginDescription() {
return originDescription; return originDescription;
} }
public void setOriginDescription(OriginDescription originDescription) { public void setOriginDescription(OriginDescription originDescription) {
this.originDescription = originDescription; this.originDescription = originDescription;
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
OAIProvenance that = (OAIProvenance) o; if (o == null || getClass() != o.getClass())
return Objects.equals(originDescription, that.originDescription); return false;
} OAIProvenance that = (OAIProvenance) o;
return Objects.equals(originDescription, that.originDescription);
}
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(originDescription); return Objects.hash(originDescription);
} }
} }

92
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Oaf.java
View File

@ -1,61 +1,73 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List;
import java.util.Objects; import java.util.Objects;
public abstract class Oaf implements Serializable { public abstract class Oaf implements Serializable {
private DataInfo dataInfo;
private Long lastupdatetimestamp; /**
* The list of datasource id/name pairs providing this relationship.
*/
protected List<KeyValue> collectedfrom;
public DataInfo getDataInfo() { private DataInfo dataInfo;
return dataInfo;
}
public void setDataInfo(DataInfo dataInfo) { private Long lastupdatetimestamp;
this.dataInfo = dataInfo;
}
public Long getLastupdatetimestamp() { public List<KeyValue> getCollectedfrom() {
return lastupdatetimestamp; return collectedfrom;
} }
public void setLastupdatetimestamp(Long lastupdatetimestamp) { public void setCollectedfrom(List<KeyValue> collectedfrom) {
this.lastupdatetimestamp = lastupdatetimestamp; this.collectedfrom = collectedfrom;
} }
public DataInfo getDataInfo() {
return dataInfo;
}
public void mergeOAFDataInfo(Oaf e) { public void setDataInfo(DataInfo dataInfo) {
if (e.getDataInfo()!= null && compareTrust(this,e)<0) this.dataInfo = dataInfo;
dataInfo = e.getDataInfo(); }
}
protected String extractTrust(Oaf e) { public Long getLastupdatetimestamp() {
if (e == null || e.getDataInfo()== null || e.getDataInfo().getTrust()== null) return lastupdatetimestamp;
return "0.0"; }
return e.getDataInfo().getTrust();
public void setLastupdatetimestamp(Long lastupdatetimestamp) {
this.lastupdatetimestamp = lastupdatetimestamp;
}
public void mergeOAFDataInfo(Oaf e) {
if (e.getDataInfo() != null && compareTrust(this, e) < 0)
dataInfo = e.getDataInfo();
}
} protected String extractTrust(Oaf e) {
if (e == null || e.getDataInfo() == null || e.getDataInfo().getTrust() == null)
return "0.0";
return e.getDataInfo().getTrust();
}
protected int compareTrust(Oaf a, Oaf b) { protected int compareTrust(Oaf a, Oaf b) {
return extractTrust(a).compareTo(extractTrust(b)); return extractTrust(a).compareTo(extractTrust(b));
}
} @Override
public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
Oaf oaf = (Oaf) o;
return Objects.equals(dataInfo, oaf.dataInfo)
&& Objects.equals(lastupdatetimestamp, oaf.lastupdatetimestamp);
}
@Override @Override
public boolean equals(Object o) { public int hashCode() {
if (this == o) return true; return Objects.hash(dataInfo, lastupdatetimestamp);
if (o == null || getClass() != o.getClass()) return false; }
Oaf oaf = (Oaf) o;
return Objects.equals(dataInfo, oaf.dataInfo) &&
Objects.equals(lastupdatetimestamp, oaf.lastupdatetimestamp);
}
@Override
public int hashCode() {
return Objects.hash(dataInfo, lastupdatetimestamp);
}
} }

184
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/OafEntity.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
@ -6,134 +7,123 @@ import java.util.stream.Collectors;
public abstract class OafEntity extends Oaf implements Serializable { public abstract class OafEntity extends Oaf implements Serializable {
private String id; private String id;
private List<String> originalId; private List<String> originalId;
private List<KeyValue> collectedfrom; private List<StructuredProperty> pid;
private List<StructuredProperty> pid; private String dateofcollection;
private String dateofcollection; private String dateoftransformation;
private String dateoftransformation; private List<ExtraInfo> extraInfo;
private List<ExtraInfo> extraInfo; private OAIProvenance oaiprovenance;
private OAIProvenance oaiprovenance; public String getId() {
return id;
}
public String getId() { public void setId(String id) {
return id; this.id = id;
} }
public void setId(String id) { public List<String> getOriginalId() {
this.id = id; return originalId;
} }
public List<String> getOriginalId() { public void setOriginalId(List<String> originalId) {
return originalId; this.originalId = originalId;
} }
public void setOriginalId(List<String> originalId) { public List<StructuredProperty> getPid() {
this.originalId = originalId; return pid;
} }
public List<KeyValue> getCollectedfrom() { public void setPid(List<StructuredProperty> pid) {
return collectedfrom; this.pid = pid;
} }
public void setCollectedfrom(List<KeyValue> collectedfrom) { public String getDateofcollection() {
this.collectedfrom = collectedfrom; return dateofcollection;
} }
public List<StructuredProperty> getPid() { public void setDateofcollection(String dateofcollection) {
return pid; this.dateofcollection = dateofcollection;
} }
public void setPid(List<StructuredProperty> pid) { public String getDateoftransformation() {
this.pid = pid; return dateoftransformation;
} }
public String getDateofcollection() { public void setDateoftransformation(String dateoftransformation) {
return dateofcollection; this.dateoftransformation = dateoftransformation;
} }
public void setDateofcollection(String dateofcollection) { public List<ExtraInfo> getExtraInfo() {
this.dateofcollection = dateofcollection; return extraInfo;
} }
public String getDateoftransformation() { public void setExtraInfo(List<ExtraInfo> extraInfo) {
return dateoftransformation; this.extraInfo = extraInfo;
} }
public void setDateoftransformation(String dateoftransformation) { public OAIProvenance getOaiprovenance() {
this.dateoftransformation = dateoftransformation; return oaiprovenance;
} }
public List<ExtraInfo> getExtraInfo() { public void setOaiprovenance(OAIProvenance oaiprovenance) {
return extraInfo; this.oaiprovenance = oaiprovenance;
} }
public void setExtraInfo(List<ExtraInfo> extraInfo) { public void mergeFrom(OafEntity e) {
this.extraInfo = extraInfo;
}
public OAIProvenance getOaiprovenance() { if (e == null)
return oaiprovenance; return;
}
public void setOaiprovenance(OAIProvenance oaiprovenance) { originalId = mergeLists(originalId, e.getOriginalId());
this.oaiprovenance = oaiprovenance;
}
collectedfrom = mergeLists(collectedfrom, e.getCollectedfrom());
public void mergeFrom(OafEntity e) { pid = mergeLists(pid, e.getPid());
if (e == null) if (e.getDateofcollection() != null && compareTrust(this, e) < 0)
return; dateofcollection = e.getDateofcollection();
originalId = mergeLists(originalId, e.getOriginalId()); if (e.getDateoftransformation() != null && compareTrust(this, e) < 0)
dateoftransformation = e.getDateoftransformation();
collectedfrom = mergeLists(collectedfrom, e.getCollectedfrom()); extraInfo = mergeLists(extraInfo, e.getExtraInfo());
pid = mergeLists(pid, e.getPid()); if (e.getOaiprovenance() != null && compareTrust(this, e) < 0)
oaiprovenance = e.getOaiprovenance();
}
if (e.getDateofcollection() != null && compareTrust(this, e) < 0) protected <T> List<T> mergeLists(final List<T>... lists) {
dateofcollection = e.getDateofcollection();
if (e.getDateoftransformation() != null && compareTrust(this, e) < 0) return Arrays
dateoftransformation = e.getDateoftransformation(); .stream(lists)
.filter(Objects::nonNull)
.flatMap(List::stream)
.distinct()
.collect(Collectors.toList());
}
extraInfo = mergeLists(extraInfo, e.getExtraInfo()); @Override
public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
if (!super.equals(o))
return false;
OafEntity oafEntity = (OafEntity) o;
return Objects.equals(id, oafEntity.id);
}
if (e.getOaiprovenance() != null && compareTrust(this, e) < 0) @Override
oaiprovenance = e.getOaiprovenance(); public int hashCode() {
return Objects.hash(super.hashCode(), id);
} }
protected <T> List<T> mergeLists(final List<T>... lists) {
return Arrays.stream(lists).filter(Objects::nonNull).flatMap(List::stream).distinct().collect(Collectors.toList());
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
if (!super.equals(o)) return false;
OafEntity oafEntity = (OafEntity) o;
return Objects.equals(id, oafEntity.id) &&
Objects.equals(originalId, oafEntity.originalId) &&
Objects.equals(collectedfrom, oafEntity.collectedfrom) &&
Objects.equals(pid, oafEntity.pid) &&
Objects.equals(dateofcollection, oafEntity.dateofcollection) &&
Objects.equals(dateoftransformation, oafEntity.dateoftransformation) &&
Objects.equals(extraInfo, oafEntity.extraInfo) &&
Objects.equals(oaiprovenance, oafEntity.oaiprovenance);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), id, originalId, collectedfrom, pid, dateofcollection, dateoftransformation, extraInfo, oaiprovenance);
}
} }

324
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Organization.java
View File

@ -1,226 +1,214 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List; import java.util.List;
import java.util.Objects;
public class Organization extends OafEntity implements Serializable { public class Organization extends OafEntity implements Serializable {
private Field<String> legalshortname; private Field<String> legalshortname;
private Field<String> legalname; private Field<String> legalname;
private List<Field<String>> alternativeNames; private List<Field<String>> alternativeNames;
private Field<String> websiteurl; private Field<String> websiteurl;
private Field<String> logourl; private Field<String> logourl;
private Field<String> eclegalbody; private Field<String> eclegalbody;
private Field<String> eclegalperson; private Field<String> eclegalperson;
private Field<String> ecnonprofit; private Field<String> ecnonprofit;
private Field<String> ecresearchorganization; private Field<String> ecresearchorganization;
private Field<String> echighereducation; private Field<String> echighereducation;
private Field<String> ecinternationalorganizationeurinterests; private Field<String> ecinternationalorganizationeurinterests;
private Field<String> ecinternationalorganization; private Field<String> ecinternationalorganization;
private Field<String> ecenterprise; private Field<String> ecenterprise;
private Field<String> ecsmevalidated; private Field<String> ecsmevalidated;
private Field<String> ecnutscode; private Field<String> ecnutscode;
private Qualifier country; private Qualifier country;
public Field<String> getLegalshortname() { public Field<String> getLegalshortname() {
return legalshortname; return legalshortname;
} }
public void setLegalshortname(Field<String> legalshortname) { public void setLegalshortname(Field<String> legalshortname) {
this.legalshortname = legalshortname; this.legalshortname = legalshortname;
} }
public Field<String> getLegalname() { public Field<String> getLegalname() {
return legalname; return legalname;
} }
public void setLegalname(Field<String> legalname) { public void setLegalname(Field<String> legalname) {
this.legalname = legalname; this.legalname = legalname;
} }
public List<Field<String>> getAlternativeNames() { public List<Field<String>> getAlternativeNames() {
return alternativeNames; return alternativeNames;
} }
public void setAlternativeNames(List<Field<String>> alternativeNames) { public void setAlternativeNames(List<Field<String>> alternativeNames) {
this.alternativeNames = alternativeNames; this.alternativeNames = alternativeNames;
} }
public Field<String> getWebsiteurl() { public Field<String> getWebsiteurl() {
return websiteurl; return websiteurl;
} }
public void setWebsiteurl(Field<String> websiteurl) { public void setWebsiteurl(Field<String> websiteurl) {
this.websiteurl = websiteurl; this.websiteurl = websiteurl;
} }
public Field<String> getLogourl() { public Field<String> getLogourl() {
return logourl; return logourl;
} }
public void setLogourl(Field<String> logourl) { public void setLogourl(Field<String> logourl) {
this.logourl = logourl; this.logourl = logourl;
} }
public Field<String> getEclegalbody() { public Field<String> getEclegalbody() {
return eclegalbody; return eclegalbody;
} }
public void setEclegalbody(Field<String> eclegalbody) { public void setEclegalbody(Field<String> eclegalbody) {
this.eclegalbody = eclegalbody; this.eclegalbody = eclegalbody;
} }
public Field<String> getEclegalperson() { public Field<String> getEclegalperson() {
return eclegalperson; return eclegalperson;
} }
public void setEclegalperson(Field<String> eclegalperson) { public void setEclegalperson(Field<String> eclegalperson) {
this.eclegalperson = eclegalperson; this.eclegalperson = eclegalperson;
} }
public Field<String> getEcnonprofit() { public Field<String> getEcnonprofit() {
return ecnonprofit; return ecnonprofit;
} }
public void setEcnonprofit(Field<String> ecnonprofit) { public void setEcnonprofit(Field<String> ecnonprofit) {
this.ecnonprofit = ecnonprofit; this.ecnonprofit = ecnonprofit;
} }
public Field<String> getEcresearchorganization() { public Field<String> getEcresearchorganization() {
return ecresearchorganization; return ecresearchorganization;
} }
public void setEcresearchorganization(Field<String> ecresearchorganization) { public void setEcresearchorganization(Field<String> ecresearchorganization) {
this.ecresearchorganization = ecresearchorganization; this.ecresearchorganization = ecresearchorganization;
} }
public Field<String> getEchighereducation() { public Field<String> getEchighereducation() {
return echighereducation; return echighereducation;
} }
public void setEchighereducation(Field<String> echighereducation) { public void setEchighereducation(Field<String> echighereducation) {
this.echighereducation = echighereducation; this.echighereducation = echighereducation;
} }
public Field<String> getEcinternationalorganizationeurinterests() { public Field<String> getEcinternationalorganizationeurinterests() {
return ecinternationalorganizationeurinterests; return ecinternationalorganizationeurinterests;
} }
public void setEcinternationalorganizationeurinterests(Field<String> ecinternationalorganizationeurinterests) { public void setEcinternationalorganizationeurinterests(
this.ecinternationalorganizationeurinterests = ecinternationalorganizationeurinterests; Field<String> ecinternationalorganizationeurinterests) {
} this.ecinternationalorganizationeurinterests = ecinternationalorganizationeurinterests;
}
public Field<String> getEcinternationalorganization() { public Field<String> getEcinternationalorganization() {
return ecinternationalorganization; return ecinternationalorganization;
} }
public void setEcinternationalorganization(Field<String> ecinternationalorganization) { public void setEcinternationalorganization(Field<String> ecinternationalorganization) {
this.ecinternationalorganization = ecinternationalorganization; this.ecinternationalorganization = ecinternationalorganization;
} }
public Field<String> getEcenterprise() { public Field<String> getEcenterprise() {
return ecenterprise; return ecenterprise;
} }
public void setEcenterprise(Field<String> ecenterprise) { public void setEcenterprise(Field<String> ecenterprise) {
this.ecenterprise = ecenterprise; this.ecenterprise = ecenterprise;
} }
public Field<String> getEcsmevalidated() { public Field<String> getEcsmevalidated() {
return ecsmevalidated; return ecsmevalidated;
} }
public void setEcsmevalidated(Field<String> ecsmevalidated) { public void setEcsmevalidated(Field<String> ecsmevalidated) {
this.ecsmevalidated = ecsmevalidated; this.ecsmevalidated = ecsmevalidated;
} }
public Field<String> getEcnutscode() { public Field<String> getEcnutscode() {
return ecnutscode; return ecnutscode;
} }
public void setEcnutscode(Field<String> ecnutscode) { public void setEcnutscode(Field<String> ecnutscode) {
this.ecnutscode = ecnutscode; this.ecnutscode = ecnutscode;
} }
public Qualifier getCountry() { public Qualifier getCountry() {
return country; return country;
} }
public void setCountry(Qualifier country) { public void setCountry(Qualifier country) {
this.country = country; this.country = country;
} }
@Override
public void mergeFrom(OafEntity e) {
super.mergeFrom(e);
@Override if (!Organization.class.isAssignableFrom(e.getClass())) {
public void mergeFrom(OafEntity e) { return;
super.mergeFrom(e); }
if (!Organization.class.isAssignableFrom(e.getClass())){ final Organization o = (Organization) e;
return; legalshortname = o.getLegalshortname() != null && compareTrust(this, e) < 0
} ? o.getLegalshortname()
: legalshortname;
final Organization o = (Organization) e; legalname = o.getLegalname() != null && compareTrust(this, e) < 0 ? o.getLegalname() : legalname;
legalshortname = o.getLegalshortname() != null && compareTrust(this, e)<0? o.getLegalshortname() : legalshortname; alternativeNames = mergeLists(o.getAlternativeNames(), alternativeNames);
legalname = o.getLegalname() != null && compareTrust(this, e)<0 ? o.getLegalname() : legalname; websiteurl = o.getWebsiteurl() != null && compareTrust(this, e) < 0 ? o.getWebsiteurl() : websiteurl;
alternativeNames = mergeLists(o.getAlternativeNames(), alternativeNames); logourl = o.getLogourl() != null && compareTrust(this, e) < 0 ? o.getLogourl() : logourl;
websiteurl = o.getWebsiteurl() != null && compareTrust(this, e)<0? o.getWebsiteurl() : websiteurl; eclegalbody = o.getEclegalbody() != null && compareTrust(this, e) < 0 ? o.getEclegalbody() : eclegalbody;
logourl = o.getLogourl() != null && compareTrust(this, e)<0? o.getLogourl() : logourl; eclegalperson = o.getEclegalperson() != null && compareTrust(this, e) < 0
eclegalbody = o.getEclegalbody() != null && compareTrust(this, e)<0? o.getEclegalbody() : eclegalbody; ? o.getEclegalperson()
eclegalperson = o.getEclegalperson() != null && compareTrust(this, e)<0? o.getEclegalperson() : eclegalperson; : eclegalperson;
ecnonprofit = o.getEcnonprofit() != null && compareTrust(this, e)<0? o.getEcnonprofit() : ecnonprofit; ecnonprofit = o.getEcnonprofit() != null && compareTrust(this, e) < 0 ? o.getEcnonprofit() : ecnonprofit;
ecresearchorganization = o.getEcresearchorganization() != null && compareTrust(this, e)<0? o.getEcresearchorganization() : ecresearchorganization; ecresearchorganization = o.getEcresearchorganization() != null && compareTrust(this, e) < 0
echighereducation = o.getEchighereducation() != null && compareTrust(this, e)<0? o.getEchighereducation() : echighereducation; ? o.getEcresearchorganization()
ecinternationalorganizationeurinterests = o.getEcinternationalorganizationeurinterests() != null && compareTrust(this, e)<0? o.getEcinternationalorganizationeurinterests() : ecinternationalorganizationeurinterests; : ecresearchorganization;
ecinternationalorganization = o.getEcinternationalorganization() != null && compareTrust(this, e)<0? o.getEcinternationalorganization() : ecinternationalorganization; echighereducation = o.getEchighereducation() != null && compareTrust(this, e) < 0
ecenterprise = o.getEcenterprise() != null && compareTrust(this, e)<0? o.getEcenterprise() :ecenterprise; ? o.getEchighereducation()
ecsmevalidated = o.getEcsmevalidated() != null && compareTrust(this, e)<0? o.getEcsmevalidated() :ecsmevalidated; : echighereducation;
ecnutscode = o.getEcnutscode() != null && compareTrust(this, e)<0? o.getEcnutscode() :ecnutscode; ecinternationalorganizationeurinterests = o.getEcinternationalorganizationeurinterests() != null
country = o.getCountry() != null && compareTrust(this, e)<0 ? o.getCountry() :country; && compareTrust(this, e) < 0
mergeOAFDataInfo(o); ? o.getEcinternationalorganizationeurinterests()
} : ecinternationalorganizationeurinterests;
ecinternationalorganization = o.getEcinternationalorganization() != null && compareTrust(this, e) < 0
@Override ? o.getEcinternationalorganization()
public boolean equals(Object o) { : ecinternationalorganization;
if (this == o) return true; ecenterprise = o.getEcenterprise() != null && compareTrust(this, e) < 0
if (o == null || getClass() != o.getClass()) return false; ? o.getEcenterprise()
if (!super.equals(o)) return false; : ecenterprise;
Organization that = (Organization) o; ecsmevalidated = o.getEcsmevalidated() != null && compareTrust(this, e) < 0
return Objects.equals(legalshortname, that.legalshortname) && ? o.getEcsmevalidated()
Objects.equals(legalname, that.legalname) && : ecsmevalidated;
Objects.equals(alternativeNames, that.alternativeNames) && ecnutscode = o.getEcnutscode() != null && compareTrust(this, e) < 0 ? o.getEcnutscode() : ecnutscode;
Objects.equals(websiteurl, that.websiteurl) && country = o.getCountry() != null && compareTrust(this, e) < 0 ? o.getCountry() : country;
Objects.equals(logourl, that.logourl) && mergeOAFDataInfo(o);
Objects.equals(eclegalbody, that.eclegalbody) && }
Objects.equals(eclegalperson, that.eclegalperson) &&
Objects.equals(ecnonprofit, that.ecnonprofit) &&
Objects.equals(ecresearchorganization, that.ecresearchorganization) &&
Objects.equals(echighereducation, that.echighereducation) &&
Objects.equals(ecinternationalorganizationeurinterests, that.ecinternationalorganizationeurinterests) &&
Objects.equals(ecinternationalorganization, that.ecinternationalorganization) &&
Objects.equals(ecenterprise, that.ecenterprise) &&
Objects.equals(ecsmevalidated, that.ecsmevalidated) &&
Objects.equals(ecnutscode, that.ecnutscode) &&
Objects.equals(country, that.country);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), legalshortname, legalname, alternativeNames, websiteurl, logourl, eclegalbody, eclegalperson, ecnonprofit, ecresearchorganization, echighereducation, ecinternationalorganizationeurinterests, ecinternationalorganization, ecenterprise, ecsmevalidated, ecnutscode, country);
}
} }

119
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/OriginDescription.java
View File

@ -1,3 +1,4 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
@ -5,81 +6,83 @@ import java.util.Objects;
public class OriginDescription implements Serializable { public class OriginDescription implements Serializable {
private String harvestDate; private String harvestDate;
private Boolean altered = true; private Boolean altered = true;
private String baseURL; private String baseURL;
private String identifier; private String identifier;
private String datestamp; private String datestamp;
private String metadataNamespace; private String metadataNamespace;
public String getHarvestDate() { public String getHarvestDate() {
return harvestDate; return harvestDate;
} }
public void setHarvestDate(String harvestDate) { public void setHarvestDate(String harvestDate) {
this.harvestDate = harvestDate; this.harvestDate = harvestDate;
} }
public Boolean getAltered() { public Boolean getAltered() {
return altered; return altered;
} }
public void setAltered(Boolean altered) { public void setAltered(Boolean altered) {
this.altered = altered; this.altered = altered;
} }
public String getBaseURL() { public String getBaseURL() {
return baseURL; return baseURL;
} }
public void setBaseURL(String baseURL) { public void setBaseURL(String baseURL) {
this.baseURL = baseURL; this.baseURL = baseURL;
} }
public String getIdentifier() { public String getIdentifier() {
return identifier; return identifier;
} }
public void setIdentifier(String identifier) { public void setIdentifier(String identifier) {
this.identifier = identifier; this.identifier = identifier;
} }
public String getDatestamp() { public String getDatestamp() {
return datestamp; return datestamp;
} }
public void setDatestamp(String datestamp) { public void setDatestamp(String datestamp) {
this.datestamp = datestamp; this.datestamp = datestamp;
} }
public String getMetadataNamespace() { public String getMetadataNamespace() {
return metadataNamespace; return metadataNamespace;
} }
public void setMetadataNamespace(String metadataNamespace) { public void setMetadataNamespace(String metadataNamespace) {
this.metadataNamespace = metadataNamespace; this.metadataNamespace = metadataNamespace;
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
OriginDescription that = (OriginDescription) o; if (o == null || getClass() != o.getClass())
return Objects.equals(harvestDate, that.harvestDate) && return false;
Objects.equals(altered, that.altered) && OriginDescription that = (OriginDescription) o;
Objects.equals(baseURL, that.baseURL) && return Objects.equals(harvestDate, that.harvestDate)
Objects.equals(identifier, that.identifier) && && Objects.equals(altered, that.altered)
Objects.equals(datestamp, that.datestamp) && && Objects.equals(baseURL, that.baseURL)
Objects.equals(metadataNamespace, that.metadataNamespace); && Objects.equals(identifier, that.identifier)
} && Objects.equals(datestamp, that.datestamp)
&& Objects.equals(metadataNamespace, that.metadataNamespace);
}
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(harvestDate, altered, baseURL, identifier, datestamp, metadataNamespace); return Objects.hash(harvestDate, altered, baseURL, identifier, datestamp, metadataNamespace);
} }
} }

88
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/OtherResearchProduct.java
View File

@ -1,70 +1,60 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List; import java.util.List;
import java.util.Objects;
import eu.dnetlib.dhp.schema.common.ModelConstants;
public class OtherResearchProduct extends Result implements Serializable { public class OtherResearchProduct extends Result implements Serializable {
private List<Field<String>> contactperson; private List<Field<String>> contactperson;
private List<Field<String>> contactgroup; private List<Field<String>> contactgroup;
private List<Field<String>> tool; private List<Field<String>> tool;
public List<Field<String>> getContactperson() { public OtherResearchProduct() {
return contactperson; setResulttype(ModelConstants.ORP_DEFAULT_RESULTTYPE);
} }
public void setContactperson(List<Field<String>> contactperson) { public List<Field<String>> getContactperson() {
this.contactperson = contactperson; return contactperson;
} }
public List<Field<String>> getContactgroup() { public void setContactperson(List<Field<String>> contactperson) {
return contactgroup; this.contactperson = contactperson;
} }
public void setContactgroup(List<Field<String>> contactgroup) { public List<Field<String>> getContactgroup() {
this.contactgroup = contactgroup; return contactgroup;
} }
public List<Field<String>> getTool() { public void setContactgroup(List<Field<String>> contactgroup) {
return tool; this.contactgroup = contactgroup;
} }
public void setTool(List<Field<String>> tool) { public List<Field<String>> getTool() {
this.tool = tool; return tool;
} }
@Override public void setTool(List<Field<String>> tool) {
public void mergeFrom(OafEntity e) { this.tool = tool;
super.mergeFrom(e); }
if (!OtherResearchProduct.class.isAssignableFrom(e.getClass())){ @Override
return; public void mergeFrom(OafEntity e) {
} super.mergeFrom(e);
OtherResearchProduct o = (OtherResearchProduct)e; if (!OtherResearchProduct.class.isAssignableFrom(e.getClass())) {
return;
}
contactperson = mergeLists(contactperson, o.getContactperson()); OtherResearchProduct o = (OtherResearchProduct) e;
contactgroup = mergeLists(contactgroup, o.getContactgroup());
tool = mergeLists(tool, o.getTool());
mergeOAFDataInfo(e);
}
@Override contactperson = mergeLists(contactperson, o.getContactperson());
public boolean equals(Object o) { contactgroup = mergeLists(contactgroup, o.getContactgroup());
if (this == o) return true; tool = mergeLists(tool, o.getTool());
if (o == null || getClass() != o.getClass()) return false; mergeOAFDataInfo(e);
if (!super.equals(o)) return false; }
OtherResearchProduct that = (OtherResearchProduct) o;
return Objects.equals(contactperson, that.contactperson) &&
Objects.equals(contactgroup, that.contactgroup) &&
Objects.equals(tool, that.tool);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), contactperson, contactgroup, tool);
}
} }

542
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Project.java
View File

@ -1,347 +1,325 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List; import java.util.List;
import java.util.Objects;
public class Project extends OafEntity implements Serializable { public class Project extends OafEntity implements Serializable {
private Field<String> websiteurl; private Field<String> websiteurl;
private Field<String> code; private Field<String> code;
private Field<String> acronym; private Field<String> acronym;
private Field<String> title; private Field<String> title;
private Field<String> startdate; private Field<String> startdate;
private Field<String> enddate; private Field<String> enddate;
private Field<String> callidentifier; private Field<String> callidentifier;
private Field<String> keywords; private Field<String> keywords;
private Field<String> duration; private Field<String> duration;
private Field<String> ecsc39; private Field<String> ecsc39;
private Field<String> oamandatepublications; private Field<String> oamandatepublications;
private Field<String> ecarticle29_3; private Field<String> ecarticle29_3;
private List<StructuredProperty> subjects; private List<StructuredProperty> subjects;
private List<Field<String>> fundingtree; private List<Field<String>> fundingtree;
private Qualifier contracttype; private Qualifier contracttype;
private Field<String> optional1; private Field<String> optional1;
private Field<String> optional2; private Field<String> optional2;
private Field<String> jsonextrainfo; private Field<String> jsonextrainfo;
private Field<String> contactfullname; private Field<String> contactfullname;
private Field<String> contactfax; private Field<String> contactfax;
private Field<String> contactphone; private Field<String> contactphone;
private Field<String> contactemail; private Field<String> contactemail;
private Field<String> summary; private Field<String> summary;
private Field<String> currency; private Field<String> currency;
private Float totalcost; private Float totalcost;
private Float fundedamount; private Float fundedamount;
public Field<String> getWebsiteurl() { public Field<String> getWebsiteurl() {
return websiteurl; return websiteurl;
} }
public void setWebsiteurl(Field<String> websiteurl) { public void setWebsiteurl(Field<String> websiteurl) {
this.websiteurl = websiteurl; this.websiteurl = websiteurl;
} }
public Field<String> getCode() { public Field<String> getCode() {
return code; return code;
} }
public void setCode(Field<String> code) { public void setCode(Field<String> code) {
this.code = code; this.code = code;
} }
public Field<String> getAcronym() { public Field<String> getAcronym() {
return acronym; return acronym;
} }
public void setAcronym(Field<String> acronym) { public void setAcronym(Field<String> acronym) {
this.acronym = acronym; this.acronym = acronym;
} }
public Field<String> getTitle() { public Field<String> getTitle() {
return title; return title;
} }
public void setTitle(Field<String> title) { public void setTitle(Field<String> title) {
this.title = title; this.title = title;
} }
public Field<String> getStartdate() { public Field<String> getStartdate() {
return startdate; return startdate;
} }
public void setStartdate(Field<String> startdate) { public void setStartdate(Field<String> startdate) {
this.startdate = startdate; this.startdate = startdate;
} }
public Field<String> getEnddate() { public Field<String> getEnddate() {
return enddate; return enddate;
} }
public void setEnddate(Field<String> enddate) { public void setEnddate(Field<String> enddate) {
this.enddate = enddate; this.enddate = enddate;
} }
public Field<String> getCallidentifier() { public Field<String> getCallidentifier() {
return callidentifier; return callidentifier;
} }
public void setCallidentifier(Field<String> callidentifier) { public void setCallidentifier(Field<String> callidentifier) {
this.callidentifier = callidentifier; this.callidentifier = callidentifier;
} }
public Field<String> getKeywords() { public Field<String> getKeywords() {
return keywords; return keywords;
} }
public void setKeywords(Field<String> keywords) { public void setKeywords(Field<String> keywords) {
this.keywords = keywords; this.keywords = keywords;
} }
public Field<String> getDuration() { public Field<String> getDuration() {
return duration; return duration;
} }
public void setDuration(Field<String> duration) { public void setDuration(Field<String> duration) {
this.duration = duration; this.duration = duration;
} }
public Field<String> getEcsc39() { public Field<String> getEcsc39() {
return ecsc39; return ecsc39;
} }
public void setEcsc39(Field<String> ecsc39) { public void setEcsc39(Field<String> ecsc39) {
this.ecsc39 = ecsc39; this.ecsc39 = ecsc39;
} }
public Field<String> getOamandatepublications() { public Field<String> getOamandatepublications() {
return oamandatepublications; return oamandatepublications;
} }
public void setOamandatepublications(Field<String> oamandatepublications) { public void setOamandatepublications(Field<String> oamandatepublications) {
this.oamandatepublications = oamandatepublications; this.oamandatepublications = oamandatepublications;
} }
public Field<String> getEcarticle29_3() { public Field<String> getEcarticle29_3() {
return ecarticle29_3; return ecarticle29_3;
} }
public void setEcarticle29_3(Field<String> ecarticle29_3) { public void setEcarticle29_3(Field<String> ecarticle29_3) {
this.ecarticle29_3 = ecarticle29_3; this.ecarticle29_3 = ecarticle29_3;
} }
public List<StructuredProperty> getSubjects() { public List<StructuredProperty> getSubjects() {
return subjects; return subjects;
} }
public void setSubjects(List<StructuredProperty> subjects) { public void setSubjects(List<StructuredProperty> subjects) {
this.subjects = subjects; this.subjects = subjects;
} }
public List<Field<String>> getFundingtree() { public List<Field<String>> getFundingtree() {
return fundingtree; return fundingtree;
} }
public void setFundingtree(List<Field<String>> fundingtree) { public void setFundingtree(List<Field<String>> fundingtree) {
this.fundingtree = fundingtree; this.fundingtree = fundingtree;
} }
public Qualifier getContracttype() { public Qualifier getContracttype() {
return contracttype; return contracttype;
} }
public void setContracttype(Qualifier contracttype) { public void setContracttype(Qualifier contracttype) {
this.contracttype = contracttype; this.contracttype = contracttype;
} }
public Field<String> getOptional1() { public Field<String> getOptional1() {
return optional1; return optional1;
} }
public void setOptional1(Field<String> optional1) { public void setOptional1(Field<String> optional1) {
this.optional1 = optional1; this.optional1 = optional1;
} }
public Field<String> getOptional2() { public Field<String> getOptional2() {
return optional2; return optional2;
} }
public void setOptional2(Field<String> optional2) { public void setOptional2(Field<String> optional2) {
this.optional2 = optional2; this.optional2 = optional2;
} }
public Field<String> getJsonextrainfo() { public Field<String> getJsonextrainfo() {
return jsonextrainfo; return jsonextrainfo;
} }
public void setJsonextrainfo(Field<String> jsonextrainfo) { public void setJsonextrainfo(Field<String> jsonextrainfo) {
this.jsonextrainfo = jsonextrainfo; this.jsonextrainfo = jsonextrainfo;
} }
public Field<String> getContactfullname() { public Field<String> getContactfullname() {
return contactfullname; return contactfullname;
} }
public void setContactfullname(Field<String> contactfullname) { public void setContactfullname(Field<String> contactfullname) {
this.contactfullname = contactfullname; this.contactfullname = contactfullname;
} }
public Field<String> getContactfax() { public Field<String> getContactfax() {
return contactfax; return contactfax;
} }
public void setContactfax(Field<String> contactfax) { public void setContactfax(Field<String> contactfax) {
this.contactfax = contactfax; this.contactfax = contactfax;
} }
public Field<String> getContactphone() { public Field<String> getContactphone() {
return contactphone; return contactphone;
} }
public void setContactphone(Field<String> contactphone) { public void setContactphone(Field<String> contactphone) {
this.contactphone = contactphone; this.contactphone = contactphone;
} }
public Field<String> getContactemail() { public Field<String> getContactemail() {
return contactemail; return contactemail;
} }
public void setContactemail(Field<String> contactemail) { public void setContactemail(Field<String> contactemail) {
this.contactemail = contactemail; this.contactemail = contactemail;
} }
public Field<String> getSummary() { public Field<String> getSummary() {
return summary; return summary;
} }
public void setSummary(Field<String> summary) { public void setSummary(Field<String> summary) {
this.summary = summary; this.summary = summary;
} }
public Field<String> getCurrency() { public Field<String> getCurrency() {
return currency; return currency;
} }
public void setCurrency(Field<String> currency) { public void setCurrency(Field<String> currency) {
this.currency = currency; this.currency = currency;
} }
public Float getTotalcost() { public Float getTotalcost() {
return totalcost; return totalcost;
} }
public void setTotalcost(Float totalcost) { public void setTotalcost(Float totalcost) {
this.totalcost = totalcost; this.totalcost = totalcost;
} }
public Float getFundedamount() { public Float getFundedamount() {
return fundedamount; return fundedamount;
} }
public void setFundedamount(Float fundedamount) { public void setFundedamount(Float fundedamount) {
this.fundedamount = fundedamount; this.fundedamount = fundedamount;
} }
@Override
@Override public void mergeFrom(OafEntity e) {
public void mergeFrom(OafEntity e) { super.mergeFrom(e);
super.mergeFrom(e);
if (!Project.class.isAssignableFrom(e.getClass())) {
if (!Project.class.isAssignableFrom(e.getClass())){ return;
return; }
}
Project p = (Project) e;
Project p = (Project)e;
websiteurl = p.getWebsiteurl() != null && compareTrust(this, e) < 0 ? p.getWebsiteurl() : websiteurl;
websiteurl= p.getWebsiteurl()!= null && compareTrust(this,e)<0?p.getWebsiteurl():websiteurl; code = p.getCode() != null && compareTrust(this, e) < 0 ? p.getCode() : code;
code= p.getCode()!=null && compareTrust(this,e)<0?p.getCode():code; acronym = p.getAcronym() != null && compareTrust(this, e) < 0 ? p.getAcronym() : acronym;
acronym= p.getAcronym()!= null && compareTrust(this,e)<0?p.getAcronym():acronym; title = p.getTitle() != null && compareTrust(this, e) < 0 ? p.getTitle() : title;
title= p.getTitle()!= null && compareTrust(this,e)<0?p.getTitle():title; startdate = p.getStartdate() != null && compareTrust(this, e) < 0 ? p.getStartdate() : startdate;
startdate= p.getStartdate()!=null && compareTrust(this,e)<0?p.getStartdate():startdate; enddate = p.getEnddate() != null && compareTrust(this, e) < 0 ? p.getEnddate() : enddate;
enddate= p.getEnddate()!=null && compareTrust(this,e)<0?p.getEnddate():enddate; callidentifier = p.getCallidentifier() != null && compareTrust(this, e) < 0
callidentifier= p.getCallidentifier()!=null && compareTrust(this,e)<0?p.getCallidentifier():callidentifier; ? p.getCallidentifier()
keywords= p.getKeywords()!=null && compareTrust(this,e)<0?p.getKeywords():keywords; : callidentifier;
duration= p.getDuration()!=null && compareTrust(this,e)<0?p.getDuration():duration; keywords = p.getKeywords() != null && compareTrust(this, e) < 0 ? p.getKeywords() : keywords;
ecsc39= p.getEcsc39()!=null && compareTrust(this,e)<0?p.getEcsc39():ecsc39; duration = p.getDuration() != null && compareTrust(this, e) < 0 ? p.getDuration() : duration;
oamandatepublications= p.getOamandatepublications()!=null && compareTrust(this,e)<0?p.getOamandatepublications():oamandatepublications; ecsc39 = p.getEcsc39() != null && compareTrust(this, e) < 0 ? p.getEcsc39() : ecsc39;
ecarticle29_3= p.getEcarticle29_3()!=null && compareTrust(this,e)<0?p.getEcarticle29_3():ecarticle29_3; oamandatepublications = p.getOamandatepublications() != null && compareTrust(this, e) < 0
subjects= mergeLists(subjects, p.getSubjects()); ? p.getOamandatepublications()
fundingtree= mergeLists(fundingtree, p.getFundingtree()); : oamandatepublications;
contracttype= p.getContracttype()!=null && compareTrust(this,e)<0?p.getContracttype():contracttype; ecarticle29_3 = p.getEcarticle29_3() != null && compareTrust(this, e) < 0
optional1= p.getOptional1()!=null && compareTrust(this,e)<0?p.getOptional1():optional1; ? p.getEcarticle29_3()
optional2= p.getOptional2()!=null && compareTrust(this,e)<0?p.getOptional2():optional2; : ecarticle29_3;
jsonextrainfo= p.getJsonextrainfo()!=null && compareTrust(this,e)<0?p.getJsonextrainfo():jsonextrainfo; subjects = mergeLists(subjects, p.getSubjects());
contactfullname= p.getContactfullname()!=null && compareTrust(this,e)<0?p.getContactfullname():contactfullname; fundingtree = mergeLists(fundingtree, p.getFundingtree());
contactfax= p.getContactfax()!=null && compareTrust(this,e)<0?p.getContactfax():contactfax; contracttype = p.getContracttype() != null && compareTrust(this, e) < 0
contactphone= p.getContactphone()!=null && compareTrust(this,e)<0?p.getContactphone():contactphone; ? p.getContracttype()
contactemail= p.getContactemail()!=null && compareTrust(this,e)<0?p.getContactemail():contactemail; : contracttype;
summary= p.getSummary()!=null && compareTrust(this,e)<0?p.getSummary():summary; optional1 = p.getOptional1() != null && compareTrust(this, e) < 0 ? p.getOptional1() : optional1;
currency= p.getCurrency()!=null && compareTrust(this,e)<0?p.getCurrency():currency; optional2 = p.getOptional2() != null && compareTrust(this, e) < 0 ? p.getOptional2() : optional2;
totalcost= p.getTotalcost()!=null && compareTrust(this,e)<0?p.getTotalcost():totalcost; jsonextrainfo = p.getJsonextrainfo() != null && compareTrust(this, e) < 0
fundedamount= p.getFundedamount()!= null && compareTrust(this,e)<0?p.getFundedamount():fundedamount; ? p.getJsonextrainfo()
mergeOAFDataInfo(e); : jsonextrainfo;
} contactfullname = p.getContactfullname() != null && compareTrust(this, e) < 0
? p.getContactfullname()
@Override : contactfullname;
public boolean equals(Object o) { contactfax = p.getContactfax() != null && compareTrust(this, e) < 0 ? p.getContactfax() : contactfax;
if (this == o) return true; contactphone = p.getContactphone() != null && compareTrust(this, e) < 0
if (o == null || getClass() != o.getClass()) return false; ? p.getContactphone()
if (!super.equals(o)) return false; : contactphone;
Project project = (Project) o; contactemail = p.getContactemail() != null && compareTrust(this, e) < 0
return Objects.equals(websiteurl, project.websiteurl) && ? p.getContactemail()
Objects.equals(code, project.code) && : contactemail;
Objects.equals(acronym, project.acronym) && summary = p.getSummary() != null && compareTrust(this, e) < 0 ? p.getSummary() : summary;
Objects.equals(title, project.title) && currency = p.getCurrency() != null && compareTrust(this, e) < 0 ? p.getCurrency() : currency;
Objects.equals(startdate, project.startdate) && totalcost = p.getTotalcost() != null && compareTrust(this, e) < 0 ? p.getTotalcost() : totalcost;
Objects.equals(enddate, project.enddate) && fundedamount = p.getFundedamount() != null && compareTrust(this, e) < 0
Objects.equals(callidentifier, project.callidentifier) && ? p.getFundedamount()
Objects.equals(keywords, project.keywords) && : fundedamount;
Objects.equals(duration, project.duration) && mergeOAFDataInfo(e);
Objects.equals(ecsc39, project.ecsc39) && }
Objects.equals(oamandatepublications, project.oamandatepublications) &&
Objects.equals(ecarticle29_3, project.ecarticle29_3) &&
Objects.equals(subjects, project.subjects) &&
Objects.equals(fundingtree, project.fundingtree) &&
Objects.equals(contracttype, project.contracttype) &&
Objects.equals(optional1, project.optional1) &&
Objects.equals(optional2, project.optional2) &&
Objects.equals(jsonextrainfo, project.jsonextrainfo) &&
Objects.equals(contactfullname, project.contactfullname) &&
Objects.equals(contactfax, project.contactfax) &&
Objects.equals(contactphone, project.contactphone) &&
Objects.equals(contactemail, project.contactemail) &&
Objects.equals(summary, project.summary) &&
Objects.equals(currency, project.currency) &&
Objects.equals(totalcost, project.totalcost) &&
Objects.equals(fundedamount, project.fundedamount);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), websiteurl, code, acronym, title, startdate, enddate, callidentifier, keywords, duration, ecsc39, oamandatepublications, ecarticle29_3, subjects, fundingtree, contracttype, optional1, optional2, jsonextrainfo, contactfullname, contactfax, contactphone, contactemail, summary, currency, totalcost, fundedamount);
}
} }

58
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Publication.java
View File

@ -1,47 +1,39 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.Objects;
import eu.dnetlib.dhp.schema.common.ModelConstants;
public class Publication extends Result implements Serializable { public class Publication extends Result implements Serializable {
// publication specific // publication specific
private Journal journal; private Journal journal;
public Journal getJournal() { public Publication() {
return journal; setResulttype(ModelConstants.PUBLICATION_DEFAULT_RESULTTYPE);
} }
public void setJournal(Journal journal) { public Journal getJournal() {
this.journal = journal; return journal;
} }
@Override public void setJournal(Journal journal) {
public void mergeFrom(OafEntity e) { this.journal = journal;
super.mergeFrom(e); }
if (!Publication.class.isAssignableFrom(e.getClass())){ @Override
return; public void mergeFrom(OafEntity e) {
} super.mergeFrom(e);
Publication p = (Publication) e; if (!Publication.class.isAssignableFrom(e.getClass())) {
return;
}
if (p.getJournal() != null && compareTrust(this, e)<0) Publication p = (Publication) e;
journal = p.getJournal();
mergeOAFDataInfo(e);
}
@Override if (p.getJournal() != null && compareTrust(this, e) < 0)
public boolean equals(Object o) { journal = p.getJournal();
if (this == o) return true; mergeOAFDataInfo(e);
if (o == null || getClass() != o.getClass()) return false; }
if (!super.equals(o)) return false;
Publication that = (Publication) o;
return Objects.equals(journal, that.journal);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), journal);
}
} }

128
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Qualifier.java
View File

@ -1,81 +1,87 @@
package eu.dnetlib.dhp.schema.oaf;
import com.fasterxml.jackson.annotation.JsonIgnore; package eu.dnetlib.dhp.schema.oaf;
import org.apache.commons.lang3.StringUtils;
import java.io.Serializable; import java.io.Serializable;
import org.apache.commons.lang3.StringUtils;
import com.fasterxml.jackson.annotation.JsonIgnore;
public class Qualifier implements Serializable { public class Qualifier implements Serializable {
private String classid; private String classid;
private String classname; private String classname;
private String schemeid; private String schemeid;
private String schemename; private String schemename;
public String getClassid() { public String getClassid() {
return classid; return classid;
} }
public void setClassid(String classid) { public void setClassid(String classid) {
this.classid = classid; this.classid = classid;
} }
public String getClassname() { public String getClassname() {
return classname; return classname;
} }
public void setClassname(String classname) { public void setClassname(String classname) {
this.classname = classname; this.classname = classname;
} }
public String getSchemeid() { public String getSchemeid() {
return schemeid; return schemeid;
} }
public void setSchemeid(String schemeid) { public void setSchemeid(String schemeid) {
this.schemeid = schemeid; this.schemeid = schemeid;
} }
public String getSchemename() { public String getSchemename() {
return schemename; return schemename;
} }
public void setSchemename(String schemename) { public void setSchemename(String schemename) {
this.schemename = schemename; this.schemename = schemename;
} }
public String toComparableString() { public String toComparableString() {
return isBlank()?"": String.format("%s::%s::%s::%s", return isBlank()
classid != null ? classid : "", ? ""
classname != null ? classname : "", : String
schemeid != null ? schemeid : "", .format(
schemename != null ? schemename : ""); "%s::%s::%s::%s",
} classid != null ? classid : "",
classname != null ? classname : "",
schemeid != null ? schemeid : "",
schemename != null ? schemename : "");
}
@JsonIgnore @JsonIgnore
public boolean isBlank() { public boolean isBlank() {
return StringUtils.isBlank(classid) && return StringUtils.isBlank(classid)
StringUtils.isBlank(classname) && && StringUtils.isBlank(classname)
StringUtils.isBlank(schemeid) && && StringUtils.isBlank(schemeid)
StringUtils.isBlank(schemename); && StringUtils.isBlank(schemename);
} }
@Override
public int hashCode() {
return toComparableString().hashCode();
}
@Override @Override
public boolean equals(Object obj) { public int hashCode() {
if (this == obj) return toComparableString().hashCode();
return true; }
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Qualifier other = (Qualifier) obj; @Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
return toComparableString() Qualifier other = (Qualifier) obj;
.equals(other.toComparableString());
} return toComparableString().equals(other.toComparableString());
}
} }

84
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Relation.java
View File

@ -1,14 +1,15 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import static com.google.common.base.Preconditions.checkArgument;
import java.util.*; import java.util.*;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import java.util.stream.Stream; import java.util.stream.Stream;
import static com.google.common.base.Preconditions.checkArgument;
/** /**
* Relation aims to model any edge between two nodes in the OpenAIRE graph. It has a source id and a target id * Relation models any edge between two nodes in the OpenAIRE graph. It has a source id and a target id
* pointing to graph node identifiers and it is further characterised by the semantic of the link throught the fields * pointing to graph node identifiers and it is further characterised by the semantic of the link through the fields
* relType, subRelType and relClass. Provenance information is modeled according to the dataInfo element and collectedFrom, * relType, subRelType and relClass. Provenance information is modeled according to the dataInfo element and collectedFrom,
* while individual relationship types can provide extra information via the properties field. * while individual relationship types can provide extra information via the properties field.
*/ */
@ -39,16 +40,6 @@ public class Relation extends Oaf {
*/ */
private String target; private String target;
/**
* The list of datasource id/name pairs providing this relationship.
*/
private List<KeyValue> collectedFrom = new ArrayList<>();
/**
* List of relation specific properties. Values include 'similarityLevel', indicating the similarity score between a pair of publications.
*/
private List<KeyValue> properties = new ArrayList<>();
public String getRelType() { public String getRelType() {
return relType; return relType;
} }
@ -89,53 +80,46 @@ public class Relation extends Oaf {
this.target = target; this.target = target;
} }
public List<KeyValue> getCollectedFrom() {
return collectedFrom;
}
public void setCollectedFrom(final List<KeyValue> collectedFrom) {
this.collectedFrom = collectedFrom;
}
public List<KeyValue> getProperties() {
return properties;
}
public void setProperties(List<KeyValue> properties) {
this.properties = properties;
}
public void mergeFrom(final Relation r) { public void mergeFrom(final Relation r) {
checkArgument(Objects.equals(getSource(), r.getSource()),"source ids must be equal"); checkArgument(Objects.equals(getSource(), r.getSource()), "source ids must be equal");
checkArgument(Objects.equals(getTarget(), r.getTarget()),"target ids must be equal"); checkArgument(Objects.equals(getTarget(), r.getTarget()), "target ids must be equal");
checkArgument(Objects.equals(getRelType(), r.getRelType()),"relType(s) must be equal"); checkArgument(Objects.equals(getRelType(), r.getRelType()), "relType(s) must be equal");
checkArgument(Objects.equals(getSubRelType(), r.getSubRelType()),"subRelType(s) must be equal"); checkArgument(
checkArgument(Objects.equals(getRelClass(), r.getRelClass()),"relClass(es) must be equal"); Objects.equals(getSubRelType(), r.getSubRelType()), "subRelType(s) must be equal");
checkArgument(Objects.equals(getRelClass(), r.getRelClass()), "relClass(es) must be equal");
setCollectedFrom( setCollectedfrom(
Stream Stream
.concat(Optional.ofNullable(getCollectedFrom()).map(Collection::stream).orElse(Stream.empty()), .concat(
Optional.ofNullable(r.getCollectedFrom()).map(Collection::stream).orElse(Stream.empty())) Optional
.distinct() // relies on KeyValue.equals .ofNullable(getCollectedfrom())
.collect(Collectors.toList())); .map(Collection::stream)
.orElse(Stream.empty()),
Optional
.ofNullable(r.getCollectedfrom())
.map(Collection::stream)
.orElse(Stream.empty()))
.distinct() // relies on KeyValue.equals
.collect(Collectors.toList()));
} }
@Override @Override
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o)
if (o == null || getClass() != o.getClass()) return false; return true;
if (o == null || getClass() != o.getClass())
return false;
Relation relation = (Relation) o; Relation relation = (Relation) o;
return relType.equals(relation.relType) && return relType.equals(relation.relType)
subRelType.equals(relation.subRelType) && && subRelType.equals(relation.subRelType)
relClass.equals(relation.relClass) && && relClass.equals(relation.relClass)
source.equals(relation.source) && && source.equals(relation.source)
target.equals(relation.target); && target.equals(relation.target);
} }
@Override @Override
public int hashCode() { public int hashCode() {
return Objects.hash(relType, subRelType, relClass, source, target, collectedFrom); return Objects.hash(relType, subRelType, relClass, source, target, collectedfrom);
} }
} }

428
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Result.java
View File

@ -1,319 +1,297 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.Comparator; import java.util.Comparator;
import java.util.List; import java.util.List;
import java.util.Objects;
public class Result extends OafEntity implements Serializable { public class Result extends OafEntity implements Serializable {
private List<Author> author; private List<Author> author;
// resulttype allows subclassing results into publications | datasets | software // resulttype allows subclassing results into publications | datasets | software
private Qualifier resulttype; private Qualifier resulttype;
// common fields // common fields
private Qualifier language; private Qualifier language;
private List<Country> country; private List<Country> country;
private List<StructuredProperty> subject; private List<StructuredProperty> subject;
private List<StructuredProperty> title; private List<StructuredProperty> title;
private List<StructuredProperty> relevantdate; private List<StructuredProperty> relevantdate;
private List<Field<String>> description; private List<Field<String>> description;
private Field<String> dateofacceptance; private Field<String> dateofacceptance;
private Field<String> publisher; private Field<String> publisher;
private Field<String> embargoenddate; private Field<String> embargoenddate;
private List<Field<String>> source; private List<Field<String>> source;
private List<Field<String>> fulltext; // remove candidate private List<Field<String>> fulltext; // remove candidate
private List<Field<String>> format; private List<Field<String>> format;
private List<Field<String>> contributor; private List<Field<String>> contributor;
private Qualifier resourcetype; private Qualifier resourcetype;
private List<Field<String>> coverage; private List<Field<String>> coverage;
private Qualifier bestaccessright; private Qualifier bestaccessright;
private List<Context> context; private List<Context> context;
private List<ExternalReference> externalReference; private List<ExternalReference> externalReference;
private List<Instance> instance; private List<Instance> instance;
public List<Author> getAuthor() { public List<Author> getAuthor() {
return author; return author;
} }
public void setAuthor(List<Author> author) { public void setAuthor(List<Author> author) {
this.author = author; this.author = author;
} }
public Qualifier getResulttype() { public Qualifier getResulttype() {
return resulttype; return resulttype;
} }
public void setResulttype(Qualifier resulttype) { public void setResulttype(Qualifier resulttype) {
this.resulttype = resulttype; this.resulttype = resulttype;
} }
public Qualifier getLanguage() { public Qualifier getLanguage() {
return language; return language;
} }
public void setLanguage(Qualifier language) { public void setLanguage(Qualifier language) {
this.language = language; this.language = language;
} }
public List<Country> getCountry() { public List<Country> getCountry() {
return country; return country;
} }
public void setCountry(List<Country> country) { public void setCountry(List<Country> country) {
this.country = country; this.country = country;
} }
public List<StructuredProperty> getSubject() { public List<StructuredProperty> getSubject() {
return subject; return subject;
} }
public void setSubject(List<StructuredProperty> subject) { public void setSubject(List<StructuredProperty> subject) {
this.subject = subject; this.subject = subject;
} }
public List<StructuredProperty> getTitle() { public List<StructuredProperty> getTitle() {
return title; return title;
} }
public void setTitle(List<StructuredProperty> title) { public void setTitle(List<StructuredProperty> title) {
this.title = title; this.title = title;
} }
public List<StructuredProperty> getRelevantdate() { public List<StructuredProperty> getRelevantdate() {
return relevantdate; return relevantdate;
} }
public void setRelevantdate(List<StructuredProperty> relevantdate) { public void setRelevantdate(List<StructuredProperty> relevantdate) {
this.relevantdate = relevantdate; this.relevantdate = relevantdate;
} }
public List<Field<String>> getDescription() { public List<Field<String>> getDescription() {
return description; return description;
} }
public void setDescription(List<Field<String>> description) { public void setDescription(List<Field<String>> description) {
this.description = description; this.description = description;
} }
public Field<String> getDateofacceptance() { public Field<String> getDateofacceptance() {
return dateofacceptance; return dateofacceptance;
} }
public void setDateofacceptance(Field<String> dateofacceptance) { public void setDateofacceptance(Field<String> dateofacceptance) {
this.dateofacceptance = dateofacceptance; this.dateofacceptance = dateofacceptance;
} }
public Field<String> getPublisher() { public Field<String> getPublisher() {
return publisher; return publisher;
} }
public void setPublisher(Field<String> publisher) { public void setPublisher(Field<String> publisher) {
this.publisher = publisher; this.publisher = publisher;
} }
public Field<String> getEmbargoenddate() { public Field<String> getEmbargoenddate() {
return embargoenddate; return embargoenddate;
} }
public void setEmbargoenddate(Field<String> embargoenddate) { public void setEmbargoenddate(Field<String> embargoenddate) {
this.embargoenddate = embargoenddate; this.embargoenddate = embargoenddate;
} }
public List<Field<String>> getSource() { public List<Field<String>> getSource() {
return source; return source;
} }
public void setSource(List<Field<String>> source) { public void setSource(List<Field<String>> source) {
this.source = source; this.source = source;
} }
public List<Field<String>> getFulltext() { public List<Field<String>> getFulltext() {
return fulltext; return fulltext;
} }
public void setFulltext(List<Field<String>> fulltext) { public void setFulltext(List<Field<String>> fulltext) {
this.fulltext = fulltext; this.fulltext = fulltext;
} }
public List<Field<String>> getFormat() { public List<Field<String>> getFormat() {
return format; return format;
} }
public void setFormat(List<Field<String>> format) { public void setFormat(List<Field<String>> format) {
this.format = format; this.format = format;
} }
public List<Field<String>> getContributor() { public List<Field<String>> getContributor() {
return contributor; return contributor;
} }
public void setContributor(List<Field<String>> contributor) { public void setContributor(List<Field<String>> contributor) {
this.contributor = contributor; this.contributor = contributor;
} }
public Qualifier getResourcetype() { public Qualifier getResourcetype() {
return resourcetype; return resourcetype;
} }
public void setResourcetype(Qualifier resourcetype) { public void setResourcetype(Qualifier resourcetype) {
this.resourcetype = resourcetype; this.resourcetype = resourcetype;
} }
public List<Field<String>> getCoverage() { public List<Field<String>> getCoverage() {
return coverage; return coverage;
} }
public void setCoverage(List<Field<String>> coverage) { public void setCoverage(List<Field<String>> coverage) {
this.coverage = coverage; this.coverage = coverage;
} }
public Qualifier getBestaccessright() { public Qualifier getBestaccessright() {
return bestaccessright; return bestaccessright;
} }
public void setBestaccessright(Qualifier bestaccessright) { public void setBestaccessright(Qualifier bestaccessright) {
this.bestaccessright = bestaccessright; this.bestaccessright = bestaccessright;
} }
public List<Context> getContext() { public List<Context> getContext() {
return context; return context;
} }
public void setContext(List<Context> context) { public void setContext(List<Context> context) {
this.context = context; this.context = context;
} }
public List<ExternalReference> getExternalReference() { public List<ExternalReference> getExternalReference() {
return externalReference; return externalReference;
} }
public void setExternalReference(List<ExternalReference> externalReference) { public void setExternalReference(List<ExternalReference> externalReference) {
this.externalReference = externalReference; this.externalReference = externalReference;
} }
public List<Instance> getInstance() { public List<Instance> getInstance() {
return instance; return instance;
} }
public void setInstance(List<Instance> instance) { public void setInstance(List<Instance> instance) {
this.instance = instance; this.instance = instance;
} }
@Override @Override
public void mergeFrom(OafEntity e) { public void mergeFrom(OafEntity e) {
super.mergeFrom(e); super.mergeFrom(e);
if (!Result.class.isAssignableFrom(e.getClass())){ if (!Result.class.isAssignableFrom(e.getClass())) {
return; return;
} }
Result r = (Result) e; Result r = (Result) e;
instance = mergeLists(instance, r.getInstance()); instance = mergeLists(instance, r.getInstance());
if (r.getResulttype() != null && compareTrust(this, r) < 0) if (r.getBestaccessright() != null && compareTrust(this, r) < 0)
resulttype = r.getResulttype(); bestaccessright = r.getBestaccessright();
if (r.getLanguage() != null && compareTrust(this, r) < 0) if (r.getResulttype() != null && compareTrust(this, r) < 0)
language = r.getLanguage(); resulttype = r.getResulttype();
country = mergeLists(country, r.getCountry()); if (r.getLanguage() != null && compareTrust(this, r) < 0)
language = r.getLanguage();
subject = mergeLists(subject, r.getSubject()); country = mergeLists(country, r.getCountry());
title = mergeLists(title, r.getTitle()); subject = mergeLists(subject, r.getSubject());
relevantdate = mergeLists(relevantdate, r.getRelevantdate()); title = mergeLists(title, r.getTitle());
description = longestLists(description, r.getDescription()); relevantdate = mergeLists(relevantdate, r.getRelevantdate());
if (r.getPublisher() != null && compareTrust(this, r) < 0) description = longestLists(description, r.getDescription());
publisher = r.getPublisher();
if (r.getEmbargoenddate() != null && compareTrust(this, r) < 0) if (r.getPublisher() != null && compareTrust(this, r) < 0)
embargoenddate = r.getEmbargoenddate(); publisher = r.getPublisher();
source = mergeLists(source, r.getSource()); if (r.getEmbargoenddate() != null && compareTrust(this, r) < 0)
embargoenddate = r.getEmbargoenddate();
fulltext = mergeLists(fulltext, r.getFulltext()); source = mergeLists(source, r.getSource());
format = mergeLists(format, r.getFormat()); fulltext = mergeLists(fulltext, r.getFulltext());
contributor = mergeLists(contributor, r.getContributor()); format = mergeLists(format, r.getFormat());
if (r.getResourcetype() != null) contributor = mergeLists(contributor, r.getContributor());
resourcetype = r.getResourcetype();
coverage = mergeLists(coverage, r.getCoverage()); if (r.getResourcetype() != null)
resourcetype = r.getResourcetype();
context = mergeLists(context, r.getContext()); coverage = mergeLists(coverage, r.getCoverage());
externalReference = mergeLists(externalReference, r.getExternalReference()); context = mergeLists(context, r.getContext());
}
externalReference = mergeLists(externalReference, r.getExternalReference());
}
private List<Field<String>> longestLists(List<Field<String>> a, List<Field<String>> b) { private List<Field<String>> longestLists(List<Field<String>> a, List<Field<String>> b) {
if (a == null || b == null) if (a == null || b == null)
return a == null ? b : a; return a == null ? b : a;
if (a.size() == b.size()) { if (a.size() == b.size()) {
int msa = a.stream().filter(i -> i.getValue() != null).map(i -> i.getValue().length()).max(Comparator.naturalOrder()).orElse(0); int msa = a
int msb = b.stream().filter(i -> i.getValue() != null).map(i -> i.getValue().length()).max(Comparator.naturalOrder()).orElse(0); .stream()
return msa > msb ? a : b; .filter(i -> i.getValue() != null)
} .map(i -> i.getValue().length())
return a.size() > b.size() ? a : b; .max(Comparator.naturalOrder())
} .orElse(0);
int msb = b
@Override .stream()
public boolean equals(Object o) { .filter(i -> i.getValue() != null)
if (this == o) return true; .map(i -> i.getValue().length())
if (o == null || getClass() != o.getClass()) return false; .max(Comparator.naturalOrder())
if (!super.equals(o)) return false; .orElse(0);
Result result = (Result) o; return msa > msb ? a : b;
return Objects.equals(author, result.author) && }
Objects.equals(resulttype, result.resulttype) && return a.size() > b.size() ? a : b;
Objects.equals(language, result.language) && }
Objects.equals(country, result.country) &&
Objects.equals(subject, result.subject) &&
Objects.equals(title, result.title) &&
Objects.equals(relevantdate, result.relevantdate) &&
Objects.equals(description, result.description) &&
Objects.equals(dateofacceptance, result.dateofacceptance) &&
Objects.equals(publisher, result.publisher) &&
Objects.equals(embargoenddate, result.embargoenddate) &&
Objects.equals(source, result.source) &&
Objects.equals(fulltext, result.fulltext) &&
Objects.equals(format, result.format) &&
Objects.equals(contributor, result.contributor) &&
Objects.equals(resourcetype, result.resourcetype) &&
Objects.equals(coverage, result.coverage) &&
Objects.equals(bestaccessright, result.bestaccessright) &&
Objects.equals(context, result.context) &&
Objects.equals(externalReference, result.externalReference) &&
Objects.equals(instance, result.instance);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), author, resulttype, language, country, subject, title, relevantdate, description, dateofacceptance, publisher, embargoenddate, source, fulltext, format, contributor, resourcetype, coverage, bestaccessright, context, externalReference, instance);
}
} }

109
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/Software.java
View File

@ -1,85 +1,78 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
import java.util.List; import java.util.List;
import java.util.Objects;
import eu.dnetlib.dhp.schema.common.ModelConstants;
public class Software extends Result implements Serializable { public class Software extends Result implements Serializable {
private List<Field<String>> documentationUrl; private List<Field<String>> documentationUrl;
private List<StructuredProperty> license; private List<StructuredProperty> license;
private Field<String> codeRepositoryUrl; private Field<String> codeRepositoryUrl;
private Qualifier programmingLanguage; private Qualifier programmingLanguage;
public List<Field<String>> getDocumentationUrl() { public Software() {
return documentationUrl; setResulttype(ModelConstants.SOFTWARE_DEFAULT_RESULTTYPE);
} }
public void setDocumentationUrl(List<Field<String>> documentationUrl) { public List<Field<String>> getDocumentationUrl() {
this.documentationUrl = documentationUrl; return documentationUrl;
} }
public List<StructuredProperty> getLicense() { public void setDocumentationUrl(List<Field<String>> documentationUrl) {
return license; this.documentationUrl = documentationUrl;
} }
public void setLicense(List<StructuredProperty> license) { public List<StructuredProperty> getLicense() {
this.license = license; return license;
} }
public Field<String> getCodeRepositoryUrl() { public void setLicense(List<StructuredProperty> license) {
return codeRepositoryUrl; this.license = license;
} }
public void setCodeRepositoryUrl(Field<String> codeRepositoryUrl) { public Field<String> getCodeRepositoryUrl() {
this.codeRepositoryUrl = codeRepositoryUrl; return codeRepositoryUrl;
} }
public Qualifier getProgrammingLanguage() { public void setCodeRepositoryUrl(Field<String> codeRepositoryUrl) {
return programmingLanguage; this.codeRepositoryUrl = codeRepositoryUrl;
} }
public void setProgrammingLanguage(Qualifier programmingLanguage) { public Qualifier getProgrammingLanguage() {
this.programmingLanguage = programmingLanguage; return programmingLanguage;
} }
@Override public void setProgrammingLanguage(Qualifier programmingLanguage) {
public void mergeFrom(OafEntity e) { this.programmingLanguage = programmingLanguage;
super.mergeFrom(e); }
if (!Software.class.isAssignableFrom(e.getClass())){ @Override
return; public void mergeFrom(OafEntity e) {
} super.mergeFrom(e);
final Software s = (Software) e; if (!Software.class.isAssignableFrom(e.getClass())) {
documentationUrl = mergeLists(documentationUrl, s.getDocumentationUrl()); return;
}
license = mergeLists(license, s.getLicense()); final Software s = (Software) e;
documentationUrl = mergeLists(documentationUrl, s.getDocumentationUrl());
codeRepositoryUrl = s.getCodeRepositoryUrl()!= null && compareTrust(this, s)<0?s.getCodeRepositoryUrl():codeRepositoryUrl; license = mergeLists(license, s.getLicense());
programmingLanguage= s.getProgrammingLanguage()!= null && compareTrust(this, s)<0?s.getProgrammingLanguage():programmingLanguage; codeRepositoryUrl = s.getCodeRepositoryUrl() != null && compareTrust(this, s) < 0
? s.getCodeRepositoryUrl()
: codeRepositoryUrl;
mergeOAFDataInfo(e); programmingLanguage = s.getProgrammingLanguage() != null && compareTrust(this, s) < 0
} ? s.getProgrammingLanguage()
: programmingLanguage;
@Override mergeOAFDataInfo(e);
public boolean equals(Object o) { }
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
if (!super.equals(o)) return false;
Software software = (Software) o;
return Objects.equals(documentationUrl, software.documentationUrl) &&
Objects.equals(license, software.license) &&
Objects.equals(codeRepositoryUrl, software.codeRepositoryUrl) &&
Objects.equals(programmingLanguage, software.programmingLanguage);
}
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), documentationUrl, license, codeRepositoryUrl, programmingLanguage);
}
} }

80
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/oaf/StructuredProperty.java
View File

@ -1,60 +1,60 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import java.io.Serializable; import java.io.Serializable;
public class StructuredProperty implements Serializable { public class StructuredProperty implements Serializable {
private String value; private String value;
private Qualifier qualifier; private Qualifier qualifier;
private DataInfo dataInfo; private DataInfo dataInfo;
public String getValue() { public String getValue() {
return value; return value;
} }
public void setValue(String value) { public void setValue(String value) {
this.value = value; this.value = value;
} }
public Qualifier getQualifier() { public Qualifier getQualifier() {
return qualifier; return qualifier;
} }
public void setQualifier(Qualifier qualifier) { public void setQualifier(Qualifier qualifier) {
this.qualifier = qualifier; this.qualifier = qualifier;
} }
public DataInfo getDataInfo() { public DataInfo getDataInfo() {
return dataInfo; return dataInfo;
} }
public void setDataInfo(DataInfo dataInfo) { public void setDataInfo(DataInfo dataInfo) {
this.dataInfo = dataInfo; this.dataInfo = dataInfo;
} }
public String toComparableString(){ public String toComparableString() {
return value != null ? value.toLowerCase() : ""; return value != null ? value.toLowerCase() : "";
} }
@Override @Override
public int hashCode() { public int hashCode() {
return toComparableString().hashCode(); return toComparableString().hashCode();
} }
@Override @Override
public boolean equals(Object obj) { public boolean equals(Object obj) {
if (this == obj) if (this == obj)
return true; return true;
if (obj == null) if (obj == null)
return false; return false;
if (getClass() != obj.getClass()) if (getClass() != obj.getClass())
return false; return false;
StructuredProperty other = (StructuredProperty) obj; StructuredProperty other = (StructuredProperty) obj;
return toComparableString() return toComparableString().equals(other.toComparableString());
.equals(other.toComparableString()); }
}
} }

123
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/scholexplorer/DLIDataset.java
View File

@ -1,80 +1,89 @@
package eu.dnetlib.dhp.schema.scholexplorer;
import eu.dnetlib.dhp.schema.oaf.Dataset; package eu.dnetlib.dhp.schema.scholexplorer;
import eu.dnetlib.dhp.schema.oaf.OafEntity;
import org.apache.commons.lang3.StringUtils;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.HashMap; import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import eu.dnetlib.dhp.schema.oaf.Dataset;
import eu.dnetlib.dhp.schema.oaf.OafEntity;
public class DLIDataset extends Dataset { public class DLIDataset extends Dataset {
private String originalObjIdentifier; private String originalObjIdentifier;
private List<ProvenaceInfo> dlicollectedfrom; private List<ProvenaceInfo> dlicollectedfrom;
private String completionStatus; private String completionStatus;
public String getCompletionStatus() { public String getCompletionStatus() {
return completionStatus; return completionStatus;
} }
public void setCompletionStatus(String completionStatus) { public void setCompletionStatus(String completionStatus) {
this.completionStatus = completionStatus; this.completionStatus = completionStatus;
} }
public List<ProvenaceInfo> getDlicollectedfrom() { public List<ProvenaceInfo> getDlicollectedfrom() {
return dlicollectedfrom; return dlicollectedfrom;
} }
public void setDlicollectedfrom(List<ProvenaceInfo> dlicollectedfrom) { public void setDlicollectedfrom(List<ProvenaceInfo> dlicollectedfrom) {
this.dlicollectedfrom = dlicollectedfrom; this.dlicollectedfrom = dlicollectedfrom;
} }
public String getOriginalObjIdentifier() { public String getOriginalObjIdentifier() {
return originalObjIdentifier; return originalObjIdentifier;
} }
public void setOriginalObjIdentifier(String originalObjIdentifier) { public void setOriginalObjIdentifier(String originalObjIdentifier) {
this.originalObjIdentifier = originalObjIdentifier; this.originalObjIdentifier = originalObjIdentifier;
} }
@Override @Override
public void mergeFrom(OafEntity e) { public void mergeFrom(OafEntity e) {
super.mergeFrom(e); super.mergeFrom(e);
DLIDataset p = (DLIDataset) e; DLIDataset p = (DLIDataset) e;
if (StringUtils.isBlank(completionStatus) && StringUtils.isNotBlank(p.completionStatus)) if (StringUtils.isBlank(completionStatus) && StringUtils.isNotBlank(p.completionStatus))
completionStatus = p.completionStatus; completionStatus = p.completionStatus;
if ("complete".equalsIgnoreCase(p.completionStatus)) if ("complete".equalsIgnoreCase(p.completionStatus))
completionStatus = "complete"; completionStatus = "complete";
dlicollectedfrom = mergeProvenance(dlicollectedfrom, p.getDlicollectedfrom()); dlicollectedfrom = mergeProvenance(dlicollectedfrom, p.getDlicollectedfrom());
} }
private List<ProvenaceInfo> mergeProvenance(final List<ProvenaceInfo> a, final List<ProvenaceInfo> b) { private List<ProvenaceInfo> mergeProvenance(
Map<String, ProvenaceInfo> result = new HashMap<>(); final List<ProvenaceInfo> a, final List<ProvenaceInfo> b) {
if (a != null) Map<String, ProvenaceInfo> result = new HashMap<>();
a.forEach(p -> { if (a != null)
if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) { a
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus()) && StringUtils.isNotBlank(p.getCompletionStatus())) { .forEach(
result.put(p.getId(), p); p -> {
} if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) {
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus())
&& StringUtils.isNotBlank(p.getCompletionStatus())) {
result.put(p.getId(), p);
}
} else if (p != null && p.getId() != null && !result.containsKey(p.getId())) } else if (p != null && p.getId() != null && !result.containsKey(p.getId()))
result.put(p.getId(), p); result.put(p.getId(), p);
}); });
if (b != null) if (b != null)
b.forEach(p -> { b
if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) { .forEach(
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus()) && StringUtils.isNotBlank(p.getCompletionStatus())) { p -> {
result.put(p.getId(), p); if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) {
} if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus())
&& StringUtils.isNotBlank(p.getCompletionStatus())) {
result.put(p.getId(), p);
}
} else if (p != null && p.getId() != null && !result.containsKey(p.getId())) } else if (p != null && p.getId() != null && !result.containsKey(p.getId()))
result.put(p.getId(), p); result.put(p.getId(), p);
}); });
return new ArrayList<>(result.values()); return new ArrayList<>(result.values());
} }
} }

122
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/scholexplorer/DLIPublication.java
View File

@ -1,77 +1,87 @@
package eu.dnetlib.dhp.schema.scholexplorer; package eu.dnetlib.dhp.schema.scholexplorer;
import eu.dnetlib.dhp.schema.oaf.OafEntity;
import eu.dnetlib.dhp.schema.oaf.Publication;
import org.apache.commons.lang3.StringUtils;
import java.io.Serializable; import java.io.Serializable;
import java.util.*; import java.util.*;
import org.apache.commons.lang3.StringUtils;
import eu.dnetlib.dhp.schema.oaf.OafEntity;
import eu.dnetlib.dhp.schema.oaf.Publication;
public class DLIPublication extends Publication implements Serializable { public class DLIPublication extends Publication implements Serializable {
private String originalObjIdentifier; private String originalObjIdentifier;
private List<ProvenaceInfo> dlicollectedfrom; private List<ProvenaceInfo> dlicollectedfrom;
private String completionStatus; private String completionStatus;
public String getCompletionStatus() { public String getCompletionStatus() {
return completionStatus; return completionStatus;
} }
public void setCompletionStatus(String completionStatus) { public void setCompletionStatus(String completionStatus) {
this.completionStatus = completionStatus; this.completionStatus = completionStatus;
} }
public List<ProvenaceInfo> getDlicollectedfrom() { public List<ProvenaceInfo> getDlicollectedfrom() {
return dlicollectedfrom; return dlicollectedfrom;
} }
public void setDlicollectedfrom(List<ProvenaceInfo> dlicollectedfrom) { public void setDlicollectedfrom(List<ProvenaceInfo> dlicollectedfrom) {
this.dlicollectedfrom = dlicollectedfrom; this.dlicollectedfrom = dlicollectedfrom;
} }
public String getOriginalObjIdentifier() { public String getOriginalObjIdentifier() {
return originalObjIdentifier; return originalObjIdentifier;
} }
public void setOriginalObjIdentifier(String originalObjIdentifier) { public void setOriginalObjIdentifier(String originalObjIdentifier) {
this.originalObjIdentifier = originalObjIdentifier; this.originalObjIdentifier = originalObjIdentifier;
} }
@Override @Override
public void mergeFrom(OafEntity e) { public void mergeFrom(OafEntity e) {
super.mergeFrom(e); super.mergeFrom(e);
DLIPublication p = (DLIPublication) e; DLIPublication p = (DLIPublication) e;
if (StringUtils.isBlank(completionStatus) && StringUtils.isNotBlank(p.completionStatus)) if (StringUtils.isBlank(completionStatus) && StringUtils.isNotBlank(p.completionStatus))
completionStatus = p.completionStatus; completionStatus = p.completionStatus;
if ("complete".equalsIgnoreCase(p.completionStatus)) if ("complete".equalsIgnoreCase(p.completionStatus))
completionStatus = "complete"; completionStatus = "complete";
dlicollectedfrom = mergeProvenance(dlicollectedfrom, p.getDlicollectedfrom()); dlicollectedfrom = mergeProvenance(dlicollectedfrom, p.getDlicollectedfrom());
} }
private List<ProvenaceInfo> mergeProvenance(final List<ProvenaceInfo> a, final List<ProvenaceInfo> b) { private List<ProvenaceInfo> mergeProvenance(
Map<String, ProvenaceInfo> result = new HashMap<>(); final List<ProvenaceInfo> a, final List<ProvenaceInfo> b) {
if (a != null) Map<String, ProvenaceInfo> result = new HashMap<>();
a.forEach(p -> { if (a != null)
if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) { a
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus()) && StringUtils.isNotBlank(p.getCompletionStatus())) { .forEach(
result.put(p.getId(), p); p -> {
} if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) {
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus())
&& StringUtils.isNotBlank(p.getCompletionStatus())) {
result.put(p.getId(), p);
}
} else if (p != null && p.getId() != null && !result.containsKey(p.getId())) } else if (p != null && p.getId() != null && !result.containsKey(p.getId()))
result.put(p.getId(), p); result.put(p.getId(), p);
}); });
if (b != null) if (b != null)
b.forEach(p -> { b
if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) { .forEach(
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus()) && StringUtils.isNotBlank(p.getCompletionStatus())) { p -> {
result.put(p.getId(), p); if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) {
} if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus())
&& StringUtils.isNotBlank(p.getCompletionStatus())) {
result.put(p.getId(), p);
}
} else if (p != null && p.getId() != null && !result.containsKey(p.getId())) } else if (p != null && p.getId() != null && !result.containsKey(p.getId()))
result.put(p.getId(), p); result.put(p.getId(), p);
}); });
return new ArrayList<>(result.values()); return new ArrayList<>(result.values());
} }
} }

16
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/scholexplorer/DLIRelation.java Normal file
View File

@ -0,0 +1,16 @@
package eu.dnetlib.dhp.schema.scholexplorer;
import eu.dnetlib.dhp.schema.oaf.Relation;
public class DLIRelation extends Relation {
private String dateOfCollection;
public String getDateOfCollection() {
return dateOfCollection;
}
public void setDateOfCollection(String dateOfCollection) {
this.dateOfCollection = dateOfCollection;
}
}

157
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/scholexplorer/DLIUnknown.java
View File

@ -1,9 +1,5 @@
package eu.dnetlib.dhp.schema.scholexplorer;
import eu.dnetlib.dhp.schema.oaf.Oaf; package eu.dnetlib.dhp.schema.scholexplorer;
import eu.dnetlib.dhp.schema.oaf.OafEntity;
import eu.dnetlib.dhp.schema.oaf.StructuredProperty;
import org.apache.commons.lang3.StringUtils;
import java.io.Serializable; import java.io.Serializable;
import java.util.ArrayList; import java.util.ArrayList;
@ -11,98 +7,109 @@ import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import eu.dnetlib.dhp.schema.oaf.Oaf;
import eu.dnetlib.dhp.schema.oaf.StructuredProperty;
public class DLIUnknown extends Oaf implements Serializable { public class DLIUnknown extends Oaf implements Serializable {
private String id; private String id;
private List<StructuredProperty> pid; private List<StructuredProperty> pid;
private String dateofcollection; private String dateofcollection;
private String dateoftransformation; private String dateoftransformation;
private List<ProvenaceInfo> dlicollectedfrom; private List<ProvenaceInfo> dlicollectedfrom;
private String completionStatus = "incomplete"; private String completionStatus = "incomplete";
public String getCompletionStatus() { public String getCompletionStatus() {
return completionStatus; return completionStatus;
} }
public void setCompletionStatus(String completionStatus) { public void setCompletionStatus(String completionStatus) {
this.completionStatus = completionStatus; this.completionStatus = completionStatus;
} }
public List<ProvenaceInfo> getDlicollectedfrom() { public List<ProvenaceInfo> getDlicollectedfrom() {
return dlicollectedfrom; return dlicollectedfrom;
} }
public void setDlicollectedfrom(List<ProvenaceInfo> dlicollectedfrom) { public void setDlicollectedfrom(List<ProvenaceInfo> dlicollectedfrom) {
this.dlicollectedfrom = dlicollectedfrom; this.dlicollectedfrom = dlicollectedfrom;
} }
public String getId() { public String getId() {
return id; return id;
} }
public void setId(String id) { public void setId(String id) {
this.id = id; this.id = id;
} }
public List<StructuredProperty> getPid() {
return pid;
}
public List<StructuredProperty> getPid() { public void setPid(List<StructuredProperty> pid) {
return pid; this.pid = pid;
} }
public void setPid(List<StructuredProperty> pid) { public String getDateofcollection() {
this.pid = pid; return dateofcollection;
} }
public String getDateofcollection() { public void setDateofcollection(String dateofcollection) {
return dateofcollection; this.dateofcollection = dateofcollection;
} }
public void setDateofcollection(String dateofcollection) { public String getDateoftransformation() {
this.dateofcollection = dateofcollection; return dateoftransformation;
} }
public String getDateoftransformation() { public void setDateoftransformation(String dateoftransformation) {
return dateoftransformation; this.dateoftransformation = dateoftransformation;
} }
public void setDateoftransformation(String dateoftransformation) { public void mergeFrom(DLIUnknown p) {
this.dateoftransformation = dateoftransformation; if ("complete".equalsIgnoreCase(p.completionStatus))
} completionStatus = "complete";
dlicollectedfrom = mergeProvenance(dlicollectedfrom, p.getDlicollectedfrom());
}
public void mergeFrom(DLIUnknown p) { private List<ProvenaceInfo> mergeProvenance(
if ("complete".equalsIgnoreCase(p.completionStatus)) final List<ProvenaceInfo> a, final List<ProvenaceInfo> b) {
completionStatus = "complete"; Map<String, ProvenaceInfo> result = new HashMap<>();
dlicollectedfrom = mergeProvenance(dlicollectedfrom, p.getDlicollectedfrom()); if (a != null)
} a
.forEach(
p -> {
if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) {
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus())
&& StringUtils.isNotBlank(p.getCompletionStatus())) {
result.put(p.getId(), p);
}
private List<ProvenaceInfo> mergeProvenance(final List<ProvenaceInfo> a, final List<ProvenaceInfo> b) { } else if (p != null && p.getId() != null && !result.containsKey(p.getId()))
Map<String, ProvenaceInfo> result = new HashMap<>(); result.put(p.getId(), p);
if (a != null) });
a.forEach(p -> { if (b != null)
if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) { b
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus()) && StringUtils.isNotBlank(p.getCompletionStatus())) { .forEach(
result.put(p.getId(), p); p -> {
} if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) {
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus())
&& StringUtils.isNotBlank(p.getCompletionStatus())) {
result.put(p.getId(), p);
}
} else if (p != null && p.getId() != null && !result.containsKey(p.getId())) } else if (p != null && p.getId() != null && !result.containsKey(p.getId()))
result.put(p.getId(), p); result.put(p.getId(), p);
}); });
if (b != null)
b.forEach(p -> {
if (p != null && StringUtils.isNotBlank(p.getId()) && result.containsKey(p.getId())) {
if ("incomplete".equalsIgnoreCase(result.get(p.getId()).getCompletionStatus()) && StringUtils.isNotBlank(p.getCompletionStatus())) {
result.put(p.getId(), p);
}
} else if (p != null && p.getId() != null && !result.containsKey(p.getId())) return new ArrayList<>(result.values());
result.put(p.getId(), p); }
});
return new ArrayList<>(result.values());
}
} }

57
dhp-schemas/src/main/java/eu/dnetlib/dhp/schema/scholexplorer/ProvenaceInfo.java
View File

@ -1,46 +1,47 @@
package eu.dnetlib.dhp.schema.scholexplorer; package eu.dnetlib.dhp.schema.scholexplorer;
import java.io.Serializable; import java.io.Serializable;
public class ProvenaceInfo implements Serializable { public class ProvenaceInfo implements Serializable {
private String id; private String id;
private String name; private String name;
private String completionStatus; private String completionStatus;
private String collectionMode ="collected"; private String collectionMode = "collected";
public String getId() { public String getId() {
return id; return id;
} }
public void setId(String id) { public void setId(String id) {
this.id = id; this.id = id;
} }
public String getName() { public String getName() {
return name; return name;
} }
public void setName(String name) { public void setName(String name) {
this.name = name; this.name = name;
} }
public String getCompletionStatus() { public String getCompletionStatus() {
return completionStatus; return completionStatus;
} }
public void setCompletionStatus(String completionStatus) { public void setCompletionStatus(String completionStatus) {
this.completionStatus = completionStatus; this.completionStatus = completionStatus;
} }
public String getCollectionMode() { public String getCollectionMode() {
return collectionMode; return collectionMode;
} }
public void setCollectionMode(String collectionMode) { public void setCollectionMode(String collectionMode) {
this.collectionMode = collectionMode; this.collectionMode = collectionMode;
} }
} }

56
dhp-schemas/src/test/java/eu/dnetlib/dhp/schema/action/AtomicActionTest.java
View File

@ -1,42 +1,40 @@
package eu.dnetlib.dhp.schema.action; package eu.dnetlib.dhp.schema.action;
import com.fasterxml.jackson.databind.ObjectMapper;
import eu.dnetlib.dhp.schema.oaf.Relation;
import org.apache.commons.lang3.StringUtils;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import static org.junit.jupiter.api.Assertions.*; import static org.junit.jupiter.api.Assertions.*;
/** import java.io.IOException;
* @author claudio.atzori
*/ import org.apache.commons.lang3.StringUtils;
import org.junit.jupiter.api.Test;
import com.fasterxml.jackson.databind.ObjectMapper;
import eu.dnetlib.dhp.schema.oaf.Relation;
/** @author claudio.atzori */
public class AtomicActionTest { public class AtomicActionTest {
@Test @Test
public void serializationTest() throws IOException { public void serializationTest() throws IOException {
Relation rel = new Relation(); Relation rel = new Relation();
rel.setSource("1"); rel.setSource("1");
rel.setTarget("2"); rel.setTarget("2");
rel.setRelType("resultResult"); rel.setRelType("resultResult");
rel.setSubRelType("dedup"); rel.setSubRelType("dedup");
rel.setRelClass("merges"); rel.setRelClass("merges");
AtomicAction aa1 = new AtomicAction(Relation.class, rel); AtomicAction aa1 = new AtomicAction(Relation.class, rel);
final ObjectMapper mapper = new ObjectMapper(); final ObjectMapper mapper = new ObjectMapper();
String json = mapper.writeValueAsString(aa1); String json = mapper.writeValueAsString(aa1);
assertTrue(StringUtils.isNotBlank(json)); assertTrue(StringUtils.isNotBlank(json));
AtomicAction aa2 = mapper.readValue(json, AtomicAction.class); AtomicAction aa2 = mapper.readValue(json, AtomicAction.class);
assertEquals(aa1.getClazz(), aa2.getClazz());
assertEquals(aa1.getPayload(), aa2.getPayload());
}
assertEquals(aa1.getClazz(), aa2.getClazz());
assertEquals(aa1.getPayload(), aa2.getPayload());
}
} }

50
dhp-schemas/src/test/java/eu/dnetlib/dhp/schema/common/ModelSupportTest.java
View File

@ -1,35 +1,37 @@
package eu.dnetlib.dhp.schema.common;
import eu.dnetlib.dhp.schema.oaf.OafEntity; package eu.dnetlib.dhp.schema.common;
import eu.dnetlib.dhp.schema.oaf.Relation;
import eu.dnetlib.dhp.schema.oaf.Result;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue; import static org.junit.jupiter.api.Assertions.assertTrue;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import eu.dnetlib.dhp.schema.oaf.OafEntity;
import eu.dnetlib.dhp.schema.oaf.Relation;
import eu.dnetlib.dhp.schema.oaf.Result;
public class ModelSupportTest { public class ModelSupportTest {
@Nested @Nested
class IsSubClass { class IsSubClass {
@Test @Test
public void shouldReturnFalseWhenSubClassDoesNotExtendSuperClass() { public void shouldReturnFalseWhenSubClassDoesNotExtendSuperClass() {
// when // when
Boolean result = ModelSupport.isSubClass(Relation.class, OafEntity.class); Boolean result = ModelSupport.isSubClass(Relation.class, OafEntity.class);
// then // then
assertFalse(result); assertFalse(result);
} }
@Test @Test
public void shouldReturnTrueWhenSubClassExtendsSuperClass() { public void shouldReturnTrueWhenSubClassExtendsSuperClass() {
// when // when
Boolean result = ModelSupport.isSubClass(Result.class, OafEntity.class); Boolean result = ModelSupport.isSubClass(Result.class, OafEntity.class);
// then // then
assertTrue(result); assertTrue(result);
} }
} }
} }

4
dhp-schemas/src/test/java/eu/dnetlib/dhp/schema/oaf/MeasureTest.java Normal file
View File

@ -0,0 +1,4 @@
package eu.dnetlib.dhp.schema.oaf;
public class MeasureTest {
}

113
dhp-schemas/src/test/java/eu/dnetlib/dhp/schema/oaf/MergeTest.java
View File

@ -1,87 +1,88 @@
package eu.dnetlib.dhp.schema.oaf; package eu.dnetlib.dhp.schema.oaf;
import static org.junit.jupiter.api.Assertions.*; import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import java.util.Arrays; import java.util.Arrays;
import java.util.List; import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
public class MergeTest { public class MergeTest {
OafEntity oaf; OafEntity oaf;
@BeforeEach @BeforeEach
public void setUp() { public void setUp() {
oaf = new Publication(); oaf = new Publication();
} }
@Test @Test
public void mergeListsTest() { public void mergeListsTest() {
//string list merge test // string list merge test
List<String> a = Arrays.asList("a", "b", "c", "e"); List<String> a = Arrays.asList("a", "b", "c", "e");
List<String> b = Arrays.asList("a", "b", "c", "d"); List<String> b = Arrays.asList("a", "b", "c", "d");
List<String> c = null; List<String> c = null;
System.out.println("merge result 1 = " + oaf.mergeLists(a, b)); System.out.println("merge result 1 = " + oaf.mergeLists(a, b));
System.out.println("merge result 2 = " + oaf.mergeLists(a, c)); System.out.println("merge result 2 = " + oaf.mergeLists(a, c));
System.out.println("merge result 3 = " + oaf.mergeLists(c, c)); System.out.println("merge result 3 = " + oaf.mergeLists(c, c));
} }
@Test @Test
public void mergePublicationCollectedFromTest() { public void mergePublicationCollectedFromTest() {
Publication a = new Publication(); Publication a = new Publication();
Publication b = new Publication(); Publication b = new Publication();
a.setCollectedfrom(Arrays.asList(setKV("a", "open"), setKV("b", "closed"))); a.setCollectedfrom(Arrays.asList(setKV("a", "open"), setKV("b", "closed")));
b.setCollectedfrom(Arrays.asList(setKV("A", "open"), setKV("b", "Open"))); b.setCollectedfrom(Arrays.asList(setKV("A", "open"), setKV("b", "Open")));
a.mergeFrom(b); a.mergeFrom(b);
assertNotNull(a.getCollectedfrom()); assertNotNull(a.getCollectedfrom());
assertEquals(3, a.getCollectedfrom().size()); assertEquals(3, a.getCollectedfrom().size());
}
} @Test
public void mergePublicationSubjectTest() {
@Test Publication a = new Publication();
public void mergePublicationSubjectTest() { Publication b = new Publication();
Publication a = new Publication(); a.setSubject(Arrays.asList(setSP("a", "open", "classe"), setSP("b", "open", "classe")));
Publication b = new Publication(); b.setSubject(Arrays.asList(setSP("A", "open", "classe"), setSP("c", "open", "classe")));
a.setSubject(Arrays.asList(setSP("a", "open", "classe"), setSP("b", "open", "classe"))); a.mergeFrom(b);
b.setSubject(Arrays.asList(setSP("A", "open", "classe"), setSP("c", "open", "classe")));
a.mergeFrom(b); assertNotNull(a.getSubject());
assertEquals(3, a.getSubject().size());
}
assertNotNull(a.getSubject()); private KeyValue setKV(final String key, final String value) {
assertEquals(3, a.getSubject().size());
} KeyValue k = new KeyValue();
private KeyValue setKV(final String key, final String value) { k.setKey(key);
k.setValue(value);
KeyValue k = new KeyValue(); return k;
}
k.setKey(key); private StructuredProperty setSP(
k.setValue(value); final String value, final String schema, final String classname) {
StructuredProperty s = new StructuredProperty();
return k; s.setValue(value);
} Qualifier q = new Qualifier();
q.setClassname(classname);
private StructuredProperty setSP(final String value, final String schema, final String classname) { q.setClassid(classname);
StructuredProperty s = new StructuredProperty(); q.setSchemename(schema);
s.setValue(value); q.setSchemeid(schema);
Qualifier q = new Qualifier(); s.setQualifier(q);
q.setClassname(classname); return s;
q.setClassid(classname); }
q.setSchemename(schema);
q.setSchemeid(schema);
s.setQualifier(q);
return s;
}
} }

130
dhp-schemas/src/test/java/eu/dnetlib/dhp/schema/scholexplorer/DLItest.java
View File

@ -1,81 +1,83 @@
package eu.dnetlib.dhp.schema.scholexplorer;
import com.fasterxml.jackson.core.JsonProcessingException; package eu.dnetlib.dhp.schema.scholexplorer;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import eu.dnetlib.dhp.schema.oaf.Qualifier;
import eu.dnetlib.dhp.schema.oaf.StructuredProperty;
import org.junit.jupiter.api.Test;
import java.io.IOException; import java.io.IOException;
import java.util.Arrays; import java.util.Arrays;
import java.util.Collections; import java.util.Collections;
import org.junit.jupiter.api.Test;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import eu.dnetlib.dhp.schema.oaf.Qualifier;
import eu.dnetlib.dhp.schema.oaf.StructuredProperty;
public class DLItest { public class DLItest {
@Test
public void testMergePublication() throws JsonProcessingException {
DLIPublication a1 = new DLIPublication();
a1.setPid(Arrays.asList(createSP("123456", "pdb", "dnet:pid_types")));
a1.setTitle(Collections.singletonList(createSP("Un Titolo", "title", "dnetTitle")));
a1.setDlicollectedfrom(Arrays.asList(createCollectedFrom("znd", "Zenodo", "complete")));
a1.setCompletionStatus("complete");
@Test DLIPublication a = new DLIPublication();
public void testMergePublication() throws JsonProcessingException { a
DLIPublication a1 = new DLIPublication(); .setPid(
a1.setPid(Arrays.asList( createSP("123456","pdb","dnet:pid_types"))); Arrays
a1.setTitle(Collections.singletonList(createSP("Un Titolo", "title", "dnetTitle"))); .asList(
a1.setDlicollectedfrom(Arrays.asList(createCollectedFrom("znd","Zenodo","complete"))); createSP("10.11", "doi", "dnet:pid_types"),
a1.setCompletionStatus("complete"); createSP("123456", "pdb", "dnet:pid_types")));
a.setTitle(Collections.singletonList(createSP("A Title", "title", "dnetTitle")));
a
.setDlicollectedfrom(
Arrays
.asList(
createCollectedFrom("dct", "datacite", "complete"),
createCollectedFrom("dct", "datacite", "incomplete")));
a.setCompletionStatus("incomplete");
DLIPublication a = new DLIPublication(); a.mergeFrom(a1);
a.setPid(Arrays.asList(createSP("10.11","doi","dnet:pid_types"), createSP("123456","pdb","dnet:pid_types")));
a.setTitle(Collections.singletonList(createSP("A Title", "title", "dnetTitle")));
a.setDlicollectedfrom(Arrays.asList(createCollectedFrom("dct","datacite","complete"),createCollectedFrom("dct","datacite","incomplete")));
a.setCompletionStatus("incomplete");
a.mergeFrom(a1); ObjectMapper mapper = new ObjectMapper();
System.out.println(mapper.writeValueAsString(a));
}
ObjectMapper mapper = new ObjectMapper(); @Test
System.out.println(mapper.writeValueAsString(a)); public void testDeserialization() throws IOException {
final String json = "{\"dataInfo\":{\"invisible\":false,\"inferred\":null,\"deletedbyinference\":false,\"trust\":\"0.9\",\"inferenceprovenance\":null,\"provenanceaction\":null},\"lastupdatetimestamp\":null,\"id\":\"60|bd9352547098929a394655ad1a44a479\",\"originalId\":[\"bd9352547098929a394655ad1a44a479\"],\"collectedfrom\":[{\"key\":\"dli_________::datacite\",\"value\":\"Datasets in Datacite\",\"dataInfo\":null,\"blank\":false}],\"pid\":[{\"value\":\"10.7925/DRS1.DUCHAS_5078760\",\"qualifier\":{\"classid\":\"doi\",\"classname\":\"doi\",\"schemeid\":\"dnet:pid_types\",\"schemename\":\"dnet:pid_types\",\"blank\":false},\"dataInfo\":null}],\"dateofcollection\":\"2020-01-09T08:29:31.885Z\",\"dateoftransformation\":null,\"extraInfo\":null,\"oaiprovenance\":null,\"author\":[{\"fullname\":\"Cathail, S. Ó\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Donnell, Breda Mc\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Ireland. Department of Arts, Culture, and the Gaeltacht\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"University College Dublin\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"National Folklore Foundation\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Cathail, S. Ó\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Donnell, Breda Mc\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null}],\"resulttype\":null,\"language\":null,\"country\":null,\"subject\":[{\"value\":\"Recreation\",\"qualifier\":{\"classid\":\"dnet:subject\",\"classname\":\"dnet:subject\",\"schemeid\":\"unknown\",\"schemename\":\"unknown\",\"blank\":false},\"dataInfo\":null},{\"value\":\"Entertainments and recreational activities\",\"qualifier\":{\"classid\":\"dnet:subject\",\"classname\":\"dnet:subject\",\"schemeid\":\"unknown\",\"schemename\":\"unknown\",\"blank\":false},\"dataInfo\":null},{\"value\":\"Siamsaíocht agus caitheamh aimsire\",\"qualifier\":{\"classid\":\"dnet:subject\",\"classname\":\"dnet:subject\",\"schemeid\":\"unknown\",\"schemename\":\"unknown\",\"blank\":false},\"dataInfo\":null}],\"title\":[{\"value\":\"Games We Play\",\"qualifier\":null,\"dataInfo\":null}],\"relevantdate\":[{\"value\":\"1938-09-28\",\"qualifier\":{\"classid\":\"date\",\"classname\":\"date\",\"schemeid\":\"dnet::date\",\"schemename\":\"dnet::date\",\"blank\":false},\"dataInfo\":null}],\"description\":[{\"value\":\"Story collected by Breda Mc Donnell, a student at Tenure school (Tinure, Co. Louth) (no informant identified).\",\"dataInfo\":null}],\"dateofacceptance\":null,\"publisher\":{\"value\":\"University College Dublin\",\"dataInfo\":null},\"embargoenddate\":null,\"source\":null,\"fulltext\":null,\"format\":null,\"contributor\":null,\"resourcetype\":null,\"coverage\":null,\"refereed\":null,\"context\":null,\"processingchargeamount\":null,\"processingchargecurrency\":null,\"externalReference\":null,\"instance\":[],\"storagedate\":null,\"device\":null,\"size\":null,\"version\":null,\"lastmetadataupdate\":null,\"metadataversionnumber\":null,\"geolocation\":null,\"dlicollectedfrom\":[{\"id\":\"dli_________::datacite\",\"name\":\"Datasets in Datacite\",\"completionStatus\":\"complete\",\"collectionMode\":\"resolved\"}],\"completionStatus\":\"complete\"}";
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
DLIDataset dliDataset = mapper.readValue(json, DLIDataset.class);
mapper.enable(SerializationFeature.INDENT_OUTPUT);
System.out.println(mapper.writeValueAsString(dliDataset));
}
private ProvenaceInfo createCollectedFrom(
final String id, final String name, final String completionStatus) {
ProvenaceInfo p = new ProvenaceInfo();
p.setId(id);
p.setName(name);
p.setCompletionStatus(completionStatus);
return p;
}
private StructuredProperty createSP(
final String value, final String className, final String schemeName) {
StructuredProperty p = new StructuredProperty();
} p.setValue(value);
Qualifier schema = new Qualifier();
schema.setClassname(className);
schema.setClassid(className);
@Test schema.setSchemename(schemeName);
public void testDeserialization() throws IOException { schema.setSchemeid(schemeName);
p.setQualifier(schema);
final String json ="{\"dataInfo\":{\"invisible\":false,\"inferred\":null,\"deletedbyinference\":false,\"trust\":\"0.9\",\"inferenceprovenance\":null,\"provenanceaction\":null},\"lastupdatetimestamp\":null,\"id\":\"60|bd9352547098929a394655ad1a44a479\",\"originalId\":[\"bd9352547098929a394655ad1a44a479\"],\"collectedfrom\":[{\"key\":\"dli_________::datacite\",\"value\":\"Datasets in Datacite\",\"dataInfo\":null,\"blank\":false}],\"pid\":[{\"value\":\"10.7925/DRS1.DUCHAS_5078760\",\"qualifier\":{\"classid\":\"doi\",\"classname\":\"doi\",\"schemeid\":\"dnet:pid_types\",\"schemename\":\"dnet:pid_types\",\"blank\":false},\"dataInfo\":null}],\"dateofcollection\":\"2020-01-09T08:29:31.885Z\",\"dateoftransformation\":null,\"extraInfo\":null,\"oaiprovenance\":null,\"author\":[{\"fullname\":\"Cathail, S. Ó\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Donnell, Breda Mc\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Ireland. Department of Arts, Culture, and the Gaeltacht\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"University College Dublin\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"National Folklore Foundation\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Cathail, S. Ó\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null},{\"fullname\":\"Donnell, Breda Mc\",\"name\":null,\"surname\":null,\"rank\":null,\"pid\":null,\"affiliation\":null}],\"resulttype\":null,\"language\":null,\"country\":null,\"subject\":[{\"value\":\"Recreation\",\"qualifier\":{\"classid\":\"dnet:subject\",\"classname\":\"dnet:subject\",\"schemeid\":\"unknown\",\"schemename\":\"unknown\",\"blank\":false},\"dataInfo\":null},{\"value\":\"Entertainments and recreational activities\",\"qualifier\":{\"classid\":\"dnet:subject\",\"classname\":\"dnet:subject\",\"schemeid\":\"unknown\",\"schemename\":\"unknown\",\"blank\":false},\"dataInfo\":null},{\"value\":\"Siamsaíocht agus caitheamh aimsire\",\"qualifier\":{\"classid\":\"dnet:subject\",\"classname\":\"dnet:subject\",\"schemeid\":\"unknown\",\"schemename\":\"unknown\",\"blank\":false},\"dataInfo\":null}],\"title\":[{\"value\":\"Games We Play\",\"qualifier\":null,\"dataInfo\":null}],\"relevantdate\":[{\"value\":\"1938-09-28\",\"qualifier\":{\"classid\":\"date\",\"classname\":\"date\",\"schemeid\":\"dnet::date\",\"schemename\":\"dnet::date\",\"blank\":false},\"dataInfo\":null}],\"description\":[{\"value\":\"Story collected by Breda Mc Donnell, a student at Tenure school (Tinure, Co. Louth) (no informant identified).\",\"dataInfo\":null}],\"dateofacceptance\":null,\"publisher\":{\"value\":\"University College Dublin\",\"dataInfo\":null},\"embargoenddate\":null,\"source\":null,\"fulltext\":null,\"format\":null,\"contributor\":null,\"resourcetype\":null,\"coverage\":null,\"refereed\":null,\"context\":null,\"processingchargeamount\":null,\"processingchargecurrency\":null,\"externalReference\":null,\"instance\":[],\"storagedate\":null,\"device\":null,\"size\":null,\"version\":null,\"lastmetadataupdate\":null,\"metadataversionnumber\":null,\"geolocation\":null,\"dlicollectedfrom\":[{\"id\":\"dli_________::datacite\",\"name\":\"Datasets in Datacite\",\"completionStatus\":\"complete\",\"collectionMode\":\"resolved\"}],\"completionStatus\":\"complete\"}"; return p;
}
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
DLIDataset dliDataset = mapper.readValue(json, DLIDataset.class);
mapper.enable(SerializationFeature.INDENT_OUTPUT);
System.out.println(mapper.writeValueAsString(dliDataset));
}
private ProvenaceInfo createCollectedFrom(final String id, final String name, final String completionStatus) {
ProvenaceInfo p = new ProvenaceInfo();
p.setId(id);
p.setName(name);
p.setCompletionStatus(completionStatus);
return p;
}
private StructuredProperty createSP(final String value, final String className, final String schemeName) {
StructuredProperty p = new StructuredProperty();
p.setValue(value);
Qualifier schema = new Qualifier();
schema.setClassname(className);
schema.setClassid(className);
schema.setSchemename(schemeName);
schema.setSchemeid(schemeName);
p.setQualifier(schema);
return p;
}
} }

48
dhp-workflows/dhp-actionmanager/pom.xml
View File

@ -47,6 +47,16 @@
<artifactId>jaxen</artifactId> <artifactId>jaxen</artifactId>
</dependency> </dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-distcp</artifactId>
</dependency>
<dependency>
<groupId>eu.dnetlib</groupId>
<artifactId>dnet-openaire-data-protos</artifactId>
</dependency>
<dependency> <dependency>
<groupId>eu.dnetlib.dhp</groupId> <groupId>eu.dnetlib.dhp</groupId>
<artifactId>dhp-schemas</artifactId> <artifactId>dhp-schemas</artifactId>
@ -57,6 +67,44 @@
<groupId>eu.dnetlib</groupId> <groupId>eu.dnetlib</groupId>
<artifactId>dnet-actionmanager-api</artifactId> <artifactId>dnet-actionmanager-api</artifactId>
</dependency> </dependency>
<dependency>
<groupId>eu.dnetlib</groupId>
<artifactId>dnet-actionmanager-common</artifactId>
<exclusions>
<exclusion>
<groupId>eu.dnetlib</groupId>
<artifactId>dnet-openaireplus-mapping-utils</artifactId>
</exclusion>
<exclusion>
<groupId>saxonica</groupId>
<artifactId>saxon</artifactId>
</exclusion>
<exclusion>
<groupId>saxonica</groupId>
<artifactId>saxon-dom</artifactId>
</exclusion>
<exclusion>
<groupId>jgrapht</groupId>
<artifactId>jgrapht</artifactId>
</exclusion>
<exclusion>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache</artifactId>
</exclusion>
<exclusion>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.*</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>apache</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies> </dependencies>
</project> </project>

92
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/ISClient.java
View File

@ -1,20 +1,5 @@
package eu.dnetlib.dhp.actionmanager;
import com.google.common.base.Splitter; package eu.dnetlib.dhp.actionmanager;
import com.google.common.collect.Iterables;
import com.google.common.collect.Lists;
import eu.dnetlib.actionmanager.rmi.ActionManagerException;
import eu.dnetlib.actionmanager.set.ActionManagerSet;
import eu.dnetlib.actionmanager.set.ActionManagerSet.ImpactTypes;
import eu.dnetlib.dhp.actionmanager.partition.PartitionActionSetsByPayloadTypeJob;
import eu.dnetlib.dhp.utils.ISLookupClientFactory;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpException;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpService;
import org.dom4j.Document;
import org.dom4j.Element;
import org.dom4j.io.SAXReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.Serializable; import java.io.Serializable;
import java.io.StringReader; import java.io.StringReader;
@ -23,31 +8,59 @@ import java.util.List;
import java.util.NoSuchElementException; import java.util.NoSuchElementException;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import org.dom4j.Document;
import org.dom4j.Element;
import org.dom4j.io.SAXReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.base.Splitter;
import com.google.common.collect.Iterables;
import com.google.common.collect.Lists;
import eu.dnetlib.actionmanager.rmi.ActionManagerException;
import eu.dnetlib.actionmanager.set.ActionManagerSet;
import eu.dnetlib.actionmanager.set.ActionManagerSet.ImpactTypes;
import eu.dnetlib.dhp.actionmanager.partition.PartitionActionSetsByPayloadTypeJob;
import eu.dnetlib.dhp.utils.ISLookupClientFactory;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpException;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpService;
public class ISClient implements Serializable { public class ISClient implements Serializable {
private static final Logger log = LoggerFactory.getLogger(PartitionActionSetsByPayloadTypeJob.class); private static final Logger log = LoggerFactory.getLogger(PartitionActionSetsByPayloadTypeJob.class);
private static final String INPUT_ACTION_SET_ID_SEPARATOR = ","; private static final String INPUT_ACTION_SET_ID_SEPARATOR = ",";
public static List<String> getLatestRawsetPaths(String isLookupUrl, String setIds) { private final ISLookUpService isLookup;
ISLookUpService isLookup = ISLookupClientFactory.getLookUpService(isLookupUrl); public ISClient(String isLookupUrl) {
ISClient isClient = new ISClient(); isLookup = ISLookupClientFactory.getLookUpService(isLookupUrl);
List<String> ids = Lists.newArrayList(Splitter.on(INPUT_ACTION_SET_ID_SEPARATOR) }
.omitEmptyStrings()
.trimResults()
.split(setIds));
return ids.stream() public List<String> getLatestRawsetPaths(String setIds) {
.map(id -> isClient.getSet(isLookup, id))
.map(as -> as.getPathToLatest()) List<String> ids = Lists
.collect(Collectors.toCollection(ArrayList::new)); .newArrayList(
Splitter
.on(INPUT_ACTION_SET_ID_SEPARATOR)
.omitEmptyStrings()
.trimResults()
.split(setIds));
return ids
.stream()
.map(id -> getSet(isLookup, id))
.map(as -> as.getPathToLatest())
.collect(Collectors.toCollection(ArrayList::new));
} }
private ActionManagerSet getSet(ISLookUpService isLookup, final String setId) { private ActionManagerSet getSet(ISLookUpService isLookup, final String setId) {
final String q = "for $x in collection('/db/DRIVER/ActionManagerSetDSResources/ActionManagerSetDSResourceType') " final String q = "for $x in collection('/db/DRIVER/ActionManagerSetDSResources/ActionManagerSetDSResourceType') "
+ "where $x//SET/@id = '" + setId + "' return $x"; + "where $x//SET/@id = '"
+ setId
+ "' return $x";
try { try {
final String basePath = getBasePathHDFS(isLookup); final String basePath = getBasePathHDFS(isLookup);
@ -58,7 +71,8 @@ public class ISClient implements Serializable {
} }
} }
private ActionManagerSet getActionManagerSet(final String basePath, final String profile) throws ActionManagerException { private ActionManagerSet getActionManagerSet(final String basePath, final String profile)
throws ActionManagerException {
final SAXReader reader = new SAXReader(); final SAXReader reader = new SAXReader();
final ActionManagerSet set = new ActionManagerSet(); final ActionManagerSet set = new ActionManagerSet();
@ -68,13 +82,21 @@ public class ISClient implements Serializable {
set.setId(doc.valueOf("//SET/@id").trim()); set.setId(doc.valueOf("//SET/@id").trim());
set.setName(doc.valueOf("//SET").trim()); set.setName(doc.valueOf("//SET").trim());
set.setImpact(ImpactTypes.valueOf(doc.valueOf("//IMPACT").trim())); set.setImpact(ImpactTypes.valueOf(doc.valueOf("//IMPACT").trim()));
set.setLatest(doc.valueOf("//RAW_SETS/LATEST/@id"), doc.valueOf("//RAW_SETS/LATEST/@creationDate"), doc.valueOf("//RAW_SETS/LATEST/@lastUpdate")); set
.setLatest(
doc.valueOf("//RAW_SETS/LATEST/@id"),
doc.valueOf("//RAW_SETS/LATEST/@creationDate"),
doc.valueOf("//RAW_SETS/LATEST/@lastUpdate"));
set.setDirectory(doc.valueOf("//SET/@directory")); set.setDirectory(doc.valueOf("//SET/@directory"));
final List expiredNodes = doc.selectNodes("//RAW_SETS/EXPIRED"); final List expiredNodes = doc.selectNodes("//RAW_SETS/EXPIRED");
if (expiredNodes != null) { if (expiredNodes != null) {
for (int i = 0; i < expiredNodes.size(); i++) { for (int i = 0; i < expiredNodes.size(); i++) {
Element ex = (Element) expiredNodes.get(i); Element ex = (Element) expiredNodes.get(i);
set.addExpired(ex.attributeValue("id"), ex.attributeValue("creationDate"), ex.attributeValue("lastUpdate")); set
.addExpired(
ex.attributeValue("id"),
ex.attributeValue("creationDate"),
ex.attributeValue("lastUpdate"));
} }
} }
@ -96,9 +118,11 @@ public class ISClient implements Serializable {
return queryServiceProperty(isLookup, "basePath"); return queryServiceProperty(isLookup, "basePath");
} }
private String queryServiceProperty(ISLookUpService isLookup, final String propertyName) throws ActionManagerException { private String queryServiceProperty(ISLookUpService isLookup, final String propertyName)
throws ActionManagerException {
final String q = "for $x in /RESOURCE_PROFILE[.//RESOURCE_TYPE/@value='ActionManagerServiceResourceType'] return $x//SERVICE_PROPERTIES/PROPERTY[./@ key='" final String q = "for $x in /RESOURCE_PROFILE[.//RESOURCE_TYPE/@value='ActionManagerServiceResourceType'] return $x//SERVICE_PROPERTIES/PROPERTY[./@ key='"
+ propertyName + "']/@value/string()"; + propertyName
+ "']/@value/string()";
log.debug("quering for service property: " + q); log.debug("quering for service property: " + q);
try { try {
final List<String> value = isLookup.quickSearchProfile(q); final List<String> value = isLookup.quickSearchProfile(q);
@ -117,6 +141,4 @@ public class ISClient implements Serializable {
throw new ActionManagerException(msg, e); throw new ActionManagerException(msg, e);
} }
} }
} }

69
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/migration/LicenseComparator.java Normal file
View File

@ -0,0 +1,69 @@
package eu.dnetlib.dhp.actionmanager.migration;
import java.util.Comparator;
import eu.dnetlib.data.proto.FieldTypeProtos.Qualifier;
public class LicenseComparator implements Comparator<Qualifier> {
@Override
public int compare(Qualifier left, Qualifier right) {
if (left == null && right == null)
return 0;
if (left == null)
return 1;
if (right == null)
return -1;
String lClass = left.getClassid();
String rClass = right.getClassid();
if (lClass.equals(rClass))
return 0;
if (lClass.equals("OPEN SOURCE"))
return -1;
if (rClass.equals("OPEN SOURCE"))
return 1;
if (lClass.equals("OPEN"))
return -1;
if (rClass.equals("OPEN"))
return 1;
if (lClass.equals("6MONTHS"))
return -1;
if (rClass.equals("6MONTHS"))
return 1;
if (lClass.equals("12MONTHS"))
return -1;
if (rClass.equals("12MONTHS"))
return 1;
if (lClass.equals("EMBARGO"))
return -1;
if (rClass.equals("EMBARGO"))
return 1;
if (lClass.equals("RESTRICTED"))
return -1;
if (rClass.equals("RESTRICTED"))
return 1;
if (lClass.equals("CLOSED"))
return -1;
if (rClass.equals("CLOSED"))
return 1;
if (lClass.equals("UNKNOWN"))
return -1;
if (rClass.equals("UNKNOWN"))
return 1;
// Else (but unlikely), lexicographical ordering will do.
return lClass.compareTo(rClass);
}
}

194
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/migration/MigrateActionSet.java Normal file
View File

@ -0,0 +1,194 @@
package eu.dnetlib.dhp.actionmanager.migration;
import java.io.File;
import java.io.FileOutputStream;
import java.io.OutputStream;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Properties;
import java.util.stream.Collectors;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.tools.DistCp;
import org.apache.hadoop.tools.DistCpOptions;
import org.apache.hadoop.util.ToolRunner;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.base.Splitter;
import com.google.common.collect.Lists;
import eu.dnetlib.dhp.application.ArgumentApplicationParser;
import eu.dnetlib.dhp.utils.ISLookupClientFactory;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpException;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpService;
public class MigrateActionSet {
private static final Logger log = LoggerFactory.getLogger(MigrateActionSet.class);
private static final String SEPARATOR = "/";
private static final String TARGET_PATHS = "target_paths";
private static final String RAWSET_PREFIX = "rawset_";
public static void main(String[] args) throws Exception {
final ArgumentApplicationParser parser = new ArgumentApplicationParser(
IOUtils
.toString(
MigrateActionSet.class
.getResourceAsStream(
"/eu/dnetlib/dhp/actionmanager/migration/migrate_actionsets_parameters.json")));
parser.parseArgument(args);
new MigrateActionSet().run(parser);
}
private void run(ArgumentApplicationParser parser) throws Exception {
final String isLookupUrl = parser.get("isLookupUrl");
final String sourceNN = parser.get("sourceNameNode");
final String targetNN = parser.get("targetNameNode");
final String workDir = parser.get("workingDirectory");
final Integer distcp_num_maps = Integer.parseInt(parser.get("distcp_num_maps"));
final String distcp_memory_mb = parser.get("distcp_memory_mb");
final String distcp_task_timeout = parser.get("distcp_task_timeout");
final String transform_only_s = parser.get("transform_only");
log.info("transform only param: {}", transform_only_s);
final Boolean transformOnly = Boolean.valueOf(parser.get("transform_only"));
log.info("transform only: {}", transformOnly);
ISLookUpService isLookUp = ISLookupClientFactory.getLookUpService(isLookupUrl);
Configuration conf = getConfiguration(distcp_task_timeout, distcp_memory_mb, distcp_num_maps);
FileSystem targetFS = FileSystem.get(conf);
Configuration sourceConf = getConfiguration(distcp_task_timeout, distcp_memory_mb, distcp_num_maps);
sourceConf.set(FileSystem.FS_DEFAULT_NAME_KEY, sourceNN);
FileSystem sourceFS = FileSystem.get(sourceConf);
Properties props = new Properties();
List<Path> targetPaths = new ArrayList<>();
final List<Path> sourcePaths = getSourcePaths(sourceNN, isLookUp);
log
.info(
"paths to process:\n{}",
sourcePaths.stream().map(p -> p.toString()).collect(Collectors.joining("\n")));
for (Path source : sourcePaths) {
if (!sourceFS.exists(source)) {
log.warn("skipping unexisting path: {}", source);
} else {
LinkedList<String> pathQ = Lists.newLinkedList(Splitter.on(SEPARATOR).split(source.toUri().getPath()));
final String rawSet = pathQ.pollLast();
log.info("got RAWSET: {}", rawSet);
if (StringUtils.isNotBlank(rawSet) && rawSet.startsWith(RAWSET_PREFIX)) {
final String actionSetDirectory = pathQ.pollLast();
final Path targetPath = new Path(
targetNN + workDir + SEPARATOR + actionSetDirectory + SEPARATOR + rawSet);
log.info("using TARGET PATH: {}", targetPath);
if (!transformOnly) {
if (targetFS.exists(targetPath)) {
targetFS.delete(targetPath, true);
}
runDistcp(
distcp_num_maps, distcp_memory_mb, distcp_task_timeout, conf, source, targetPath);
}
targetPaths.add(targetPath);
}
}
}
props
.setProperty(
TARGET_PATHS, targetPaths.stream().map(p -> p.toString()).collect(Collectors.joining(",")));
File file = new File(System.getProperty("oozie.action.output.properties"));
try (OutputStream os = new FileOutputStream(file)) {
props.store(os, "");
}
System.out.println(file.getAbsolutePath());
}
private void runDistcp(
Integer distcp_num_maps,
String distcp_memory_mb,
String distcp_task_timeout,
Configuration conf,
Path source,
Path targetPath)
throws Exception {
final DistCpOptions op = new DistCpOptions(source, targetPath);
op.setMaxMaps(distcp_num_maps);
op.preserve(DistCpOptions.FileAttribute.BLOCKSIZE);
op.preserve(DistCpOptions.FileAttribute.REPLICATION);
op.preserve(DistCpOptions.FileAttribute.CHECKSUMTYPE);
int res = ToolRunner
.run(
new DistCp(conf, op),
new String[] {
"-Dmapred.task.timeout=" + distcp_task_timeout,
"-Dmapreduce.map.memory.mb=" + distcp_memory_mb,
"-pb",
"-m " + distcp_num_maps,
source.toString(),
targetPath.toString()
});
if (res != 0) {
throw new RuntimeException(String.format("distcp exited with code %s", res));
}
}
private Configuration getConfiguration(
String distcp_task_timeout, String distcp_memory_mb, Integer distcp_num_maps) {
final Configuration conf = new Configuration();
conf.set("dfs.webhdfs.socket.connect-timeout", distcp_task_timeout);
conf.set("dfs.webhdfs.socket.read-timeout", distcp_task_timeout);
conf.set("dfs.http.client.retry.policy.enabled", "true");
conf.set("mapred.task.timeout", distcp_task_timeout);
conf.set("mapreduce.map.memory.mb", distcp_memory_mb);
conf.set("mapred.map.tasks", String.valueOf(distcp_num_maps));
return conf;
}
private List<Path> getSourcePaths(String sourceNN, ISLookUpService isLookUp)
throws ISLookUpException {
String XQUERY = "distinct-values(\n"
+ "let $basePath := collection('/db/DRIVER/ServiceResources/ActionManagerServiceResourceType')//SERVICE_PROPERTIES/PROPERTY[@key = 'basePath']/@value/string()\n"
+ "for $x in collection('/db/DRIVER/ActionManagerSetDSResources/ActionManagerSetDSResourceType') \n"
+ "let $setDir := $x//SET/@directory/string()\n"
+ "let $rawSet := $x//RAW_SETS/LATEST/@id/string()\n"
+ "return concat($basePath, '/', $setDir, '/', $rawSet))";
log.info(String.format("running xquery:\n%s", XQUERY));
return isLookUp
.quickSearchProfile(XQUERY)
.stream()
.map(p -> sourceNN + p)
.map(Path::new)
.collect(Collectors.toList());
}
}

664
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/migration/ProtoConverter.java Normal file
View File

@ -0,0 +1,664 @@
package eu.dnetlib.dhp.actionmanager.migration;
import static eu.dnetlib.data.proto.KindProtos.Kind.entity;
import static eu.dnetlib.data.proto.KindProtos.Kind.relation;
import static eu.dnetlib.data.proto.TypeProtos.*;
import static eu.dnetlib.data.proto.TypeProtos.Type.*;
import java.io.Serializable;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;
import org.apache.commons.lang3.StringUtils;
import com.google.common.collect.Lists;
import com.googlecode.protobuf.format.JsonFormat;
import eu.dnetlib.data.proto.*;
import eu.dnetlib.dhp.schema.oaf.*;
public class ProtoConverter implements Serializable {
public static final String UNKNOWN = "UNKNOWN";
public static final String NOT_AVAILABLE = "not available";
public static final String DNET_ACCESS_MODES = "dnet:access_modes";
public static Oaf convert(OafProtos.Oaf oaf) {
try {
switch (oaf.getKind()) {
case entity:
return convertEntity(oaf);
case relation:
return convertRelation(oaf);
default:
throw new IllegalArgumentException("invalid kind " + oaf.getKind());
}
} catch (Throwable e) {
throw new RuntimeException("error on getting " + JsonFormat.printToString(oaf), e);
}
}
private static Relation convertRelation(OafProtos.Oaf oaf) {
final OafProtos.OafRel r = oaf.getRel();
final Relation rel = new Relation();
rel.setDataInfo(mapDataInfo(oaf.getDataInfo()));
rel.setLastupdatetimestamp(oaf.getLastupdatetimestamp());
rel.setSource(r.getSource());
rel.setTarget(r.getTarget());
rel.setRelType(r.getRelType().toString());
rel.setSubRelType(r.getSubRelType().toString());
rel.setRelClass(r.getRelClass());
rel
.setCollectedfrom(
r.getCollectedfromCount() > 0
? r.getCollectedfromList().stream().map(kv -> mapKV(kv)).collect(Collectors.toList())
: null);
return rel;
}
private static OafEntity convertEntity(OafProtos.Oaf oaf) {
switch (oaf.getEntity().getType()) {
case result:
final Result r = convertResult(oaf);
r.setInstance(convertInstances(oaf));
return r;
case project:
return convertProject(oaf);
case datasource:
return convertDataSource(oaf);
case organization:
return convertOrganization(oaf);
default:
throw new RuntimeException("received unknown type");
}
}
private static List<Instance> convertInstances(OafProtos.Oaf oaf) {
final ResultProtos.Result r = oaf.getEntity().getResult();
if (r.getInstanceCount() > 0) {
return r.getInstanceList().stream().map(i -> convertInstance(i)).collect(Collectors.toList());
}
return Lists.newArrayList();
}
private static Instance convertInstance(ResultProtos.Result.Instance ri) {
final Instance i = new Instance();
i.setAccessright(mapQualifier(ri.getAccessright()));
i.setCollectedfrom(mapKV(ri.getCollectedfrom()));
i.setDateofacceptance(mapStringField(ri.getDateofacceptance()));
i.setDistributionlocation(ri.getDistributionlocation());
i.setHostedby(mapKV(ri.getHostedby()));
i.setInstancetype(mapQualifier(ri.getInstancetype()));
i.setLicense(mapStringField(ri.getLicense()));
i.setUrl(ri.getUrlList());
i.setRefereed(mapStringField(ri.getRefereed()));
i.setProcessingchargeamount(mapStringField(ri.getProcessingchargeamount()));
i.setProcessingchargecurrency(mapStringField(ri.getProcessingchargecurrency()));
return i;
}
private static Organization convertOrganization(OafProtos.Oaf oaf) {
final OrganizationProtos.Organization.Metadata m = oaf.getEntity().getOrganization().getMetadata();
final Organization org = setOaf(new Organization(), oaf);
setEntity(org, oaf);
org.setLegalshortname(mapStringField(m.getLegalshortname()));
org.setLegalname(mapStringField(m.getLegalname()));
org
.setAlternativeNames(
m
.getAlternativeNamesList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
org.setWebsiteurl(mapStringField(m.getWebsiteurl()));
org.setLogourl(mapStringField(m.getLogourl()));
org.setEclegalbody(mapStringField(m.getEclegalbody()));
org.setEclegalperson(mapStringField(m.getEclegalperson()));
org.setEcnonprofit(mapStringField(m.getEcnonprofit()));
org.setEcresearchorganization(mapStringField(m.getEcresearchorganization()));
org.setEchighereducation(mapStringField(m.getEchighereducation()));
org
.setEcinternationalorganizationeurinterests(
mapStringField(m.getEcinternationalorganizationeurinterests()));
org.setEcinternationalorganization(mapStringField(m.getEcinternationalorganization()));
org.setEcenterprise(mapStringField(m.getEcenterprise()));
org.setEcsmevalidated(mapStringField(m.getEcsmevalidated()));
org.setEcnutscode(mapStringField(m.getEcnutscode()));
org.setCountry(mapQualifier(m.getCountry()));
return org;
}
private static Datasource convertDataSource(OafProtos.Oaf oaf) {
final DatasourceProtos.Datasource.Metadata m = oaf.getEntity().getDatasource().getMetadata();
final Datasource datasource = setOaf(new Datasource(), oaf);
setEntity(datasource, oaf);
datasource
.setAccessinfopackage(
m
.getAccessinfopackageList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
datasource.setCertificates(mapStringField(m.getCertificates()));
datasource.setCitationguidelineurl(mapStringField(m.getCitationguidelineurl()));
datasource.setContactemail(mapStringField(m.getContactemail()));
datasource.setDatabaseaccessrestriction(mapStringField(m.getDatabaseaccessrestriction()));
datasource.setDatabaseaccesstype(mapStringField(m.getDatabaseaccesstype()));
datasource.setDataprovider(mapBoolField(m.getDataprovider()));
datasource.setDatasourcetype(mapQualifier(m.getDatasourcetype()));
datasource.setDatauploadrestriction(mapStringField(m.getDatauploadrestriction()));
datasource.setCitationguidelineurl(mapStringField(m.getCitationguidelineurl()));
datasource.setDatauploadtype(mapStringField(m.getDatauploadtype()));
datasource.setDateofvalidation(mapStringField(m.getDateofvalidation()));
datasource.setDescription(mapStringField(m.getDescription()));
datasource.setEnglishname(mapStringField(m.getEnglishname()));
datasource.setLatitude(mapStringField(m.getLatitude()));
datasource.setLongitude(mapStringField(m.getLongitude()));
datasource.setLogourl(mapStringField(m.getLogourl()));
datasource.setMissionstatementurl(mapStringField(m.getMissionstatementurl()));
datasource.setNamespaceprefix(mapStringField(m.getNamespaceprefix()));
datasource
.setOdcontenttypes(
m
.getOdcontenttypesList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
datasource
.setOdlanguages(
m
.getOdlanguagesList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
datasource.setOdnumberofitems(mapStringField(m.getOdnumberofitems()));
datasource.setOdnumberofitemsdate(mapStringField(m.getOdnumberofitemsdate()));
datasource.setOdpolicies(mapStringField(m.getOdpolicies()));
datasource.setOfficialname(mapStringField(m.getOfficialname()));
datasource.setOpenairecompatibility(mapQualifier(m.getOpenairecompatibility()));
datasource.setPidsystems(mapStringField(m.getPidsystems()));
datasource
.setPolicies(
m.getPoliciesList().stream().map(ProtoConverter::mapKV).collect(Collectors.toList()));
datasource.setQualitymanagementkind(mapStringField(m.getQualitymanagementkind()));
datasource.setReleaseenddate(mapStringField(m.getReleaseenddate()));
datasource.setServiceprovider(mapBoolField(m.getServiceprovider()));
datasource.setReleasestartdate(mapStringField(m.getReleasestartdate()));
datasource
.setSubjects(
m
.getSubjectsList()
.stream()
.map(ProtoConverter::mapStructuredProperty)
.collect(Collectors.toList()));
datasource.setVersioning(mapBoolField(m.getVersioning()));
datasource.setWebsiteurl(mapStringField(m.getWebsiteurl()));
datasource.setJournal(mapJournal(m.getJournal()));
return datasource;
}
private static Project convertProject(OafProtos.Oaf oaf) {
final ProjectProtos.Project.Metadata m = oaf.getEntity().getProject().getMetadata();
final Project project = setOaf(new Project(), oaf);
setEntity(project, oaf);
project.setAcronym(mapStringField(m.getAcronym()));
project.setCallidentifier(mapStringField(m.getCallidentifier()));
project.setCode(mapStringField(m.getCode()));
project.setContactemail(mapStringField(m.getContactemail()));
project.setContactfax(mapStringField(m.getContactfax()));
project.setContactfullname(mapStringField(m.getContactfullname()));
project.setContactphone(mapStringField(m.getContactphone()));
project.setContracttype(mapQualifier(m.getContracttype()));
project.setCurrency(mapStringField(m.getCurrency()));
project.setDuration(mapStringField(m.getDuration()));
project.setEcarticle29_3(mapStringField(m.getEcarticle293()));
project.setEcsc39(mapStringField(m.getEcsc39()));
project.setOamandatepublications(mapStringField(m.getOamandatepublications()));
project.setStartdate(mapStringField(m.getStartdate()));
project.setEnddate(mapStringField(m.getEnddate()));
project.setFundedamount(m.getFundedamount());
project.setTotalcost(m.getTotalcost());
project.setKeywords(mapStringField(m.getKeywords()));
project
.setSubjects(
m
.getSubjectsList()
.stream()
.map(sp -> mapStructuredProperty(sp))
.collect(Collectors.toList()));
project.setTitle(mapStringField(m.getTitle()));
project.setWebsiteurl(mapStringField(m.getWebsiteurl()));
project
.setFundingtree(
m.getFundingtreeList().stream().map(f -> mapStringField(f)).collect(Collectors.toList()));
project.setJsonextrainfo(mapStringField(m.getJsonextrainfo()));
project.setSummary(mapStringField(m.getSummary()));
project.setOptional1(mapStringField(m.getOptional1()));
project.setOptional2(mapStringField(m.getOptional2()));
return project;
}
private static Result convertResult(OafProtos.Oaf oaf) {
switch (oaf.getEntity().getResult().getMetadata().getResulttype().getClassid()) {
case "dataset":
return createDataset(oaf);
case "publication":
return createPublication(oaf);
case "software":
return createSoftware(oaf);
case "other":
return createORP(oaf);
default:
Result result = setOaf(new Result(), oaf);
setEntity(result, oaf);
return setResult(result, oaf);
}
}
private static Software createSoftware(OafProtos.Oaf oaf) {
ResultProtos.Result.Metadata m = oaf.getEntity().getResult().getMetadata();
Software software = setOaf(new Software(), oaf);
setEntity(software, oaf);
setResult(software, oaf);
software
.setDocumentationUrl(
m
.getDocumentationUrlList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
software
.setLicense(
m
.getLicenseList()
.stream()
.map(ProtoConverter::mapStructuredProperty)
.collect(Collectors.toList()));
software.setCodeRepositoryUrl(mapStringField(m.getCodeRepositoryUrl()));
software.setProgrammingLanguage(mapQualifier(m.getProgrammingLanguage()));
return software;
}
private static OtherResearchProduct createORP(OafProtos.Oaf oaf) {
ResultProtos.Result.Metadata m = oaf.getEntity().getResult().getMetadata();
OtherResearchProduct otherResearchProducts = setOaf(new OtherResearchProduct(), oaf);
setEntity(otherResearchProducts, oaf);
setResult(otherResearchProducts, oaf);
otherResearchProducts
.setContactperson(
m
.getContactpersonList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
otherResearchProducts
.setContactgroup(
m
.getContactgroupList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
otherResearchProducts
.setTool(
m.getToolList().stream().map(ProtoConverter::mapStringField).collect(Collectors.toList()));
return otherResearchProducts;
}
private static Publication createPublication(OafProtos.Oaf oaf) {
ResultProtos.Result.Metadata m = oaf.getEntity().getResult().getMetadata();
Publication publication = setOaf(new Publication(), oaf);
setEntity(publication, oaf);
setResult(publication, oaf);
publication.setJournal(mapJournal(m.getJournal()));
return publication;
}
private static Dataset createDataset(OafProtos.Oaf oaf) {
ResultProtos.Result.Metadata m = oaf.getEntity().getResult().getMetadata();
Dataset dataset = setOaf(new Dataset(), oaf);
setEntity(dataset, oaf);
setResult(dataset, oaf);
dataset.setStoragedate(mapStringField(m.getStoragedate()));
dataset.setDevice(mapStringField(m.getDevice()));
dataset.setSize(mapStringField(m.getSize()));
dataset.setVersion(mapStringField(m.getVersion()));
dataset.setLastmetadataupdate(mapStringField(m.getLastmetadataupdate()));
dataset.setMetadataversionnumber(mapStringField(m.getMetadataversionnumber()));
dataset
.setGeolocation(
m
.getGeolocationList()
.stream()
.map(ProtoConverter::mapGeolocation)
.collect(Collectors.toList()));
return dataset;
}
public static <T extends Oaf> T setOaf(T oaf, OafProtos.Oaf o) {
oaf.setDataInfo(mapDataInfo(o.getDataInfo()));
oaf.setLastupdatetimestamp(o.getLastupdatetimestamp());
return oaf;
}
public static <T extends OafEntity> T setEntity(T entity, OafProtos.Oaf oaf) {
// setting Entity fields
final OafProtos.OafEntity e = oaf.getEntity();
entity.setId(e.getId());
entity.setOriginalId(e.getOriginalIdList());
entity
.setCollectedfrom(
e.getCollectedfromList().stream().map(ProtoConverter::mapKV).collect(Collectors.toList()));
entity
.setPid(
e
.getPidList()
.stream()
.map(ProtoConverter::mapStructuredProperty)
.collect(Collectors.toList()));
entity.setDateofcollection(e.getDateofcollection());
entity.setDateoftransformation(e.getDateoftransformation());
entity
.setExtraInfo(
e
.getExtraInfoList()
.stream()
.map(ProtoConverter::mapExtraInfo)
.collect(Collectors.toList()));
return entity;
}
public static <T extends Result> T setResult(T entity, OafProtos.Oaf oaf) {
// setting Entity fields
final ResultProtos.Result.Metadata m = oaf.getEntity().getResult().getMetadata();
entity
.setAuthor(
m.getAuthorList().stream().map(ProtoConverter::mapAuthor).collect(Collectors.toList()));
entity.setResulttype(mapQualifier(m.getResulttype()));
entity.setLanguage(mapQualifier(m.getLanguage()));
entity
.setCountry(
m
.getCountryList()
.stream()
.map(ProtoConverter::mapQualifierAsCountry)
.collect(Collectors.toList()));
entity
.setSubject(
m
.getSubjectList()
.stream()
.map(ProtoConverter::mapStructuredProperty)
.collect(Collectors.toList()));
entity
.setTitle(
m
.getTitleList()
.stream()
.map(ProtoConverter::mapStructuredProperty)
.collect(Collectors.toList()));
entity
.setRelevantdate(
m
.getRelevantdateList()
.stream()
.map(ProtoConverter::mapStructuredProperty)
.collect(Collectors.toList()));
entity
.setDescription(
m
.getDescriptionList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
entity.setDateofacceptance(mapStringField(m.getDateofacceptance()));
entity.setPublisher(mapStringField(m.getPublisher()));
entity.setEmbargoenddate(mapStringField(m.getEmbargoenddate()));
entity
.setSource(
m
.getSourceList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
entity
.setFulltext(
m
.getFulltextList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
entity
.setFormat(
m
.getFormatList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
entity
.setContributor(
m
.getContributorList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
entity.setResourcetype(mapQualifier(m.getResourcetype()));
entity
.setCoverage(
m
.getCoverageList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
entity
.setContext(
m.getContextList().stream().map(ProtoConverter::mapContext).collect(Collectors.toList()));
entity.setBestaccessright(getBestAccessRights(oaf.getEntity().getResult().getInstanceList()));
return entity;
}
private static Qualifier getBestAccessRights(List<ResultProtos.Result.Instance> instanceList) {
if (instanceList != null) {
final Optional<FieldTypeProtos.Qualifier> min = instanceList
.stream()
.map(i -> i.getAccessright())
.min(new LicenseComparator());
final Qualifier rights = min.isPresent() ? mapQualifier(min.get()) : new Qualifier();
if (StringUtils.isBlank(rights.getClassid())) {
rights.setClassid(UNKNOWN);
}
if (StringUtils.isBlank(rights.getClassname())
|| UNKNOWN.equalsIgnoreCase(rights.getClassname())) {
rights.setClassname(NOT_AVAILABLE);
}
if (StringUtils.isBlank(rights.getSchemeid())) {
rights.setSchemeid(DNET_ACCESS_MODES);
}
if (StringUtils.isBlank(rights.getSchemename())) {
rights.setSchemename(DNET_ACCESS_MODES);
}
return rights;
}
return null;
}
private static Context mapContext(ResultProtos.Result.Context context) {
final Context entity = new Context();
entity.setId(context.getId());
entity
.setDataInfo(
context
.getDataInfoList()
.stream()
.map(ProtoConverter::mapDataInfo)
.collect(Collectors.toList()));
return entity;
}
public static KeyValue mapKV(FieldTypeProtos.KeyValue kv) {
final KeyValue keyValue = new KeyValue();
keyValue.setKey(kv.getKey());
keyValue.setValue(kv.getValue());
keyValue.setDataInfo(mapDataInfo(kv.getDataInfo()));
return keyValue;
}
public static DataInfo mapDataInfo(FieldTypeProtos.DataInfo d) {
final DataInfo dataInfo = new DataInfo();
dataInfo.setDeletedbyinference(d.getDeletedbyinference());
dataInfo.setInferenceprovenance(d.getInferenceprovenance());
dataInfo.setInferred(d.getInferred());
dataInfo.setInvisible(d.getInvisible());
dataInfo.setProvenanceaction(mapQualifier(d.getProvenanceaction()));
dataInfo.setTrust(d.getTrust());
return dataInfo;
}
public static Qualifier mapQualifier(FieldTypeProtos.Qualifier q) {
final Qualifier qualifier = new Qualifier();
qualifier.setClassid(q.getClassid());
qualifier.setClassname(q.getClassname());
qualifier.setSchemeid(q.getSchemeid());
qualifier.setSchemename(q.getSchemename());
return qualifier;
}
public static Country mapQualifierAsCountry(FieldTypeProtos.Qualifier q) {
final Country c = new Country();
c.setClassid(q.getClassid());
c.setClassname(q.getClassname());
c.setSchemeid(q.getSchemeid());
c.setSchemename(q.getSchemename());
c.setDataInfo(mapDataInfo(q.getDataInfo()));
return c;
}
public static StructuredProperty mapStructuredProperty(FieldTypeProtos.StructuredProperty sp) {
final StructuredProperty structuredProperty = new StructuredProperty();
structuredProperty.setValue(sp.getValue());
structuredProperty.setQualifier(mapQualifier(sp.getQualifier()));
structuredProperty.setDataInfo(mapDataInfo(sp.getDataInfo()));
return structuredProperty;
}
public static ExtraInfo mapExtraInfo(FieldTypeProtos.ExtraInfo extraInfo) {
final ExtraInfo entity = new ExtraInfo();
entity.setName(extraInfo.getName());
entity.setTypology(extraInfo.getTypology());
entity.setProvenance(extraInfo.getProvenance());
entity.setTrust(extraInfo.getTrust());
entity.setValue(extraInfo.getValue());
return entity;
}
public static OAIProvenance mapOAIProvenance(FieldTypeProtos.OAIProvenance oaiProvenance) {
final OAIProvenance entity = new OAIProvenance();
entity.setOriginDescription(mapOriginalDescription(oaiProvenance.getOriginDescription()));
return entity;
}
public static OriginDescription mapOriginalDescription(
FieldTypeProtos.OAIProvenance.OriginDescription originDescription) {
final OriginDescription originDescriptionResult = new OriginDescription();
originDescriptionResult.setHarvestDate(originDescription.getHarvestDate());
originDescriptionResult.setAltered(originDescription.getAltered());
originDescriptionResult.setBaseURL(originDescription.getBaseURL());
originDescriptionResult.setIdentifier(originDescription.getIdentifier());
originDescriptionResult.setDatestamp(originDescription.getDatestamp());
originDescriptionResult.setMetadataNamespace(originDescription.getMetadataNamespace());
return originDescriptionResult;
}
public static Field<String> mapStringField(FieldTypeProtos.StringField s) {
final Field<String> stringField = new Field<>();
stringField.setValue(s.getValue());
stringField.setDataInfo(mapDataInfo(s.getDataInfo()));
return stringField;
}
public static Field<Boolean> mapBoolField(FieldTypeProtos.BoolField b) {
final Field<Boolean> booleanField = new Field<>();
booleanField.setValue(b.getValue());
booleanField.setDataInfo(mapDataInfo(b.getDataInfo()));
return booleanField;
}
public static Field<Integer> mapIntField(FieldTypeProtos.IntField b) {
final Field<Integer> entity = new Field<>();
entity.setValue(b.getValue());
entity.setDataInfo(mapDataInfo(b.getDataInfo()));
return entity;
}
public static Journal mapJournal(FieldTypeProtos.Journal j) {
final Journal journal = new Journal();
journal.setConferencedate(j.getConferencedate());
journal.setConferenceplace(j.getConferenceplace());
journal.setEdition(j.getEdition());
journal.setEp(j.getEp());
journal.setIss(j.getIss());
journal.setIssnLinking(j.getIssnLinking());
journal.setIssnOnline(j.getIssnOnline());
journal.setIssnPrinted(j.getIssnPrinted());
journal.setName(j.getName());
journal.setSp(j.getSp());
journal.setVol(j.getVol());
journal.setDataInfo(mapDataInfo(j.getDataInfo()));
return journal;
}
public static Author mapAuthor(FieldTypeProtos.Author author) {
final Author entity = new Author();
entity.setFullname(author.getFullname());
entity.setName(author.getName());
entity.setSurname(author.getSurname());
entity.setRank(author.getRank());
entity
.setPid(
author
.getPidList()
.stream()
.map(
kv -> {
final StructuredProperty sp = new StructuredProperty();
sp.setValue(kv.getValue());
final Qualifier q = new Qualifier();
q.setClassid(kv.getKey());
q.setClassname(kv.getKey());
sp.setQualifier(q);
return sp;
})
.collect(Collectors.toList()));
entity
.setAffiliation(
author
.getAffiliationList()
.stream()
.map(ProtoConverter::mapStringField)
.collect(Collectors.toList()));
return entity;
}
public static GeoLocation mapGeolocation(ResultProtos.Result.GeoLocation geoLocation) {
final GeoLocation entity = new GeoLocation();
entity.setPoint(geoLocation.getPoint());
entity.setBox(geoLocation.getBox());
entity.setPlace(geoLocation.getPlace());
return entity;
}
}

172
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/migration/TransformActions.java Normal file
View File

@ -0,0 +1,172 @@
package eu.dnetlib.dhp.actionmanager.migration;
import static eu.dnetlib.dhp.common.SparkSessionSupport.runWithSparkSession;
import java.io.IOException;
import java.io.Serializable;
import java.util.LinkedList;
import java.util.Objects;
import java.util.Optional;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.SparkSession;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.base.Splitter;
import com.google.common.collect.Lists;
import com.google.protobuf.InvalidProtocolBufferException;
import eu.dnetlib.data.proto.OafProtos;
import eu.dnetlib.dhp.application.ArgumentApplicationParser;
import eu.dnetlib.dhp.schema.action.AtomicAction;
import eu.dnetlib.dhp.schema.oaf.*;
import eu.dnetlib.dhp.utils.ISLookupClientFactory;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpException;
import eu.dnetlib.enabling.is.lookup.rmi.ISLookUpService;
import scala.Tuple2;
public class TransformActions implements Serializable {
private static final Logger log = LoggerFactory.getLogger(TransformActions.class);
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private static final String SEPARATOR = "/";
public static void main(String[] args) throws Exception {
final ArgumentApplicationParser parser = new ArgumentApplicationParser(
IOUtils
.toString(
MigrateActionSet.class
.getResourceAsStream(
"/eu/dnetlib/dhp/actionmanager/migration/transform_actionsets_parameters.json")));
parser.parseArgument(args);
Boolean isSparkSessionManaged = Optional
.ofNullable(parser.get("isSparkSessionManaged"))
.map(Boolean::valueOf)
.orElse(Boolean.TRUE);
log.info("isSparkSessionManaged: {}", isSparkSessionManaged);
final String isLookupUrl = parser.get("isLookupUrl");
log.info("isLookupUrl: {}", isLookupUrl);
final String inputPaths = parser.get("inputPaths");
if (StringUtils.isBlank(inputPaths)) {
throw new RuntimeException("empty inputPaths");
}
log.info("inputPaths: {}", inputPaths);
final String targetBaseDir = getTargetBaseDir(isLookupUrl);
SparkConf conf = new SparkConf();
runWithSparkSession(
conf, isSparkSessionManaged, spark -> transformActions(inputPaths, targetBaseDir, spark));
}
private static void transformActions(String inputPaths, String targetBaseDir, SparkSession spark)
throws IOException {
final JavaSparkContext sc = JavaSparkContext.fromSparkContext(spark.sparkContext());
final FileSystem fs = FileSystem.get(spark.sparkContext().hadoopConfiguration());
for (String sourcePath : Lists.newArrayList(Splitter.on(",").split(inputPaths))) {
LinkedList<String> pathQ = Lists.newLinkedList(Splitter.on(SEPARATOR).split(sourcePath));
final String rawset = pathQ.pollLast();
final String actionSetDirectory = pathQ.pollLast();
final Path targetDirectory = new Path(targetBaseDir + SEPARATOR + actionSetDirectory + SEPARATOR + rawset);
if (fs.exists(targetDirectory)) {
log.info("found target directory '{}", targetDirectory);
fs.delete(targetDirectory, true);
log.info("deleted target directory '{}", targetDirectory);
}
log.info("transforming actions from '{}' to '{}'", sourcePath, targetDirectory);
sc
.sequenceFile(sourcePath, Text.class, Text.class)
.map(a -> eu.dnetlib.actionmanager.actions.AtomicAction.fromJSON(a._2().toString()))
.map(TransformActions::doTransform)
.filter(Objects::nonNull)
.mapToPair(
a -> new Tuple2<>(a.getClazz().toString(), OBJECT_MAPPER.writeValueAsString(a)))
.mapToPair(t -> new Tuple2(new Text(t._1()), new Text(t._2())))
.saveAsNewAPIHadoopFile(
targetDirectory.toString(),
Text.class,
Text.class,
SequenceFileOutputFormat.class,
sc.hadoopConfiguration());
}
}
private static AtomicAction doTransform(eu.dnetlib.actionmanager.actions.AtomicAction aa)
throws InvalidProtocolBufferException {
// dedup similarity relations had empty target value, don't migrate them
if (aa.getTargetValue().length == 0) {
return null;
}
final OafProtos.Oaf proto_oaf = OafProtos.Oaf.parseFrom(aa.getTargetValue());
final Oaf oaf = ProtoConverter.convert(proto_oaf);
switch (proto_oaf.getKind()) {
case entity:
switch (proto_oaf.getEntity().getType()) {
case datasource:
return new AtomicAction<>(Datasource.class, (Datasource) oaf);
case organization:
return new AtomicAction<>(Organization.class, (Organization) oaf);
case project:
return new AtomicAction<>(Project.class, (Project) oaf);
case result:
final String resulttypeid = proto_oaf
.getEntity()
.getResult()
.getMetadata()
.getResulttype()
.getClassid();
switch (resulttypeid) {
case "publication":
return new AtomicAction<>(Publication.class, (Publication) oaf);
case "software":
return new AtomicAction<>(Software.class, (Software) oaf);
case "other":
return new AtomicAction<>(OtherResearchProduct.class, (OtherResearchProduct) oaf);
case "dataset":
return new AtomicAction<>(Dataset.class, (Dataset) oaf);
default:
// can be an update, where the resulttype is not specified
return new AtomicAction<>(Result.class, (Result) oaf);
}
default:
throw new IllegalArgumentException(
"invalid entity type: " + proto_oaf.getEntity().getType());
}
case relation:
return new AtomicAction<>(Relation.class, (Relation) oaf);
default:
throw new IllegalArgumentException("invalid kind: " + proto_oaf.getKind());
}
}
private static String getTargetBaseDir(String isLookupUrl) throws ISLookUpException {
ISLookUpService isLookUp = ISLookupClientFactory.getLookUpService(isLookupUrl);
String XQUERY = "collection('/db/DRIVER/ServiceResources/ActionManagerServiceResourceType')//SERVICE_PROPERTIES/PROPERTY[@key = 'basePath']/@value/string()";
return isLookUp.getResourceProfileByQuery(XQUERY);
}
}

204
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/partition/PartitionActionSetsByPayloadTypeJob.java
View File

@ -1,9 +1,13 @@
package eu.dnetlib.dhp.actionmanager.partition; package eu.dnetlib.dhp.actionmanager.partition;
import eu.dnetlib.dhp.actionmanager.ISClient; import static eu.dnetlib.dhp.common.SparkSessionSupport.runWithSparkSession;
import eu.dnetlib.dhp.common.HdfsSupport; import static org.apache.spark.sql.functions.*;
import eu.dnetlib.dhp.actionmanager.promote.PromoteActionPayloadForGraphTableJob;
import eu.dnetlib.dhp.application.ArgumentApplicationParser; import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import org.apache.commons.io.IOUtils; import org.apache.commons.io.IOUtils;
import org.apache.hadoop.io.Text; import org.apache.hadoop.io.Text;
import org.apache.spark.SparkConf; import org.apache.spark.SparkConf;
@ -14,111 +18,127 @@ import org.apache.spark.sql.types.*;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import java.util.Arrays; import eu.dnetlib.dhp.actionmanager.ISClient;
import java.util.Collections; import eu.dnetlib.dhp.actionmanager.promote.PromoteActionPayloadForGraphTableJob;
import java.util.List; import eu.dnetlib.dhp.application.ArgumentApplicationParser;
import java.util.Optional; import eu.dnetlib.dhp.common.HdfsSupport;
import static eu.dnetlib.dhp.common.SparkSessionSupport.runWithSparkSession; /** Partitions given set of action sets by payload type. */
import static org.apache.spark.sql.functions.*;
/**
* Partitions given set of action sets by payload type.
*/
public class PartitionActionSetsByPayloadTypeJob { public class PartitionActionSetsByPayloadTypeJob {
private static final Logger logger = LoggerFactory.getLogger(PartitionActionSetsByPayloadTypeJob.class); private static final Logger logger = LoggerFactory.getLogger(PartitionActionSetsByPayloadTypeJob.class);
private static final StructType KV_SCHEMA = StructType$.MODULE$.apply( private static final StructType KV_SCHEMA = StructType$.MODULE$
Arrays.asList( .apply(
StructField$.MODULE$.apply("key", DataTypes.StringType, false, Metadata.empty()), Arrays
StructField$.MODULE$.apply("value", DataTypes.StringType, false, Metadata.empty()) .asList(
)); StructField$.MODULE$.apply("key", DataTypes.StringType, false, Metadata.empty()),
StructField$.MODULE$.apply("value", DataTypes.StringType, false, Metadata.empty())));
private static final StructType ATOMIC_ACTION_SCHEMA = StructType$.MODULE$.apply( private static final StructType ATOMIC_ACTION_SCHEMA = StructType$.MODULE$
Arrays.asList( .apply(
StructField$.MODULE$.apply("clazz", DataTypes.StringType, false, Metadata.empty()), Arrays
StructField$.MODULE$.apply("payload", DataTypes.StringType, false, Metadata.empty()) .asList(
)); StructField$.MODULE$.apply("clazz", DataTypes.StringType, false, Metadata.empty()),
StructField$.MODULE$
.apply(
"payload", DataTypes.StringType, false, Metadata.empty())));
public static void main(String[] args) throws Exception { private ISClient isClient;
String jsonConfiguration = IOUtils.toString(
PromoteActionPayloadForGraphTableJob.class
.getResourceAsStream("/eu/dnetlib/dhp/actionmanager/partition/partition_action_sets_by_payload_type_input_parameters.json"));
final ArgumentApplicationParser parser = new ArgumentApplicationParser(jsonConfiguration);
parser.parseArgument(args);
Boolean isSparkSessionManaged = Optional public PartitionActionSetsByPayloadTypeJob(String isLookupUrl) {
.ofNullable(parser.get("isSparkSessionManaged")) this.isClient = new ISClient(isLookupUrl);
.map(Boolean::valueOf) }
.orElse(Boolean.TRUE);
logger.info("isSparkSessionManaged: {}", isSparkSessionManaged);
String inputActionSetIds = parser.get("inputActionSetIds"); public PartitionActionSetsByPayloadTypeJob() {
logger.info("inputActionSetIds: {}", inputActionSetIds); }
String outputPath = parser.get("outputPath"); public static void main(String[] args) throws Exception {
logger.info("outputPath: {}", outputPath); String jsonConfiguration = IOUtils
.toString(
PromoteActionPayloadForGraphTableJob.class
.getResourceAsStream(
"/eu/dnetlib/dhp/actionmanager/partition/partition_action_sets_by_payload_type_input_parameters.json"));
final ArgumentApplicationParser parser = new ArgumentApplicationParser(jsonConfiguration);
parser.parseArgument(args);
String isLookupUrl = parser.get("isLookupUrl"); Boolean isSparkSessionManaged = Optional
logger.info("isLookupUrl: {}", isLookupUrl); .ofNullable(parser.get("isSparkSessionManaged"))
.map(Boolean::valueOf)
.orElse(Boolean.TRUE);
logger.info("isSparkSessionManaged: {}", isSparkSessionManaged);
List<String> inputActionSetPaths = ISClient.getLatestRawsetPaths(isLookupUrl, inputActionSetIds); String inputActionSetIds = parser.get("inputActionSetIds");
logger.info("inputActionSetPaths: {}", String.join(",", inputActionSetPaths)); logger.info("inputActionSetIds: {}", inputActionSetIds);
SparkConf conf = new SparkConf(); String outputPath = parser.get("outputPath");
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer"); logger.info("outputPath: {}", outputPath);
runWithSparkSession(conf, isSparkSessionManaged, String isLookupUrl = parser.get("isLookupUrl");
spark -> { logger.info("isLookupUrl: {}", isLookupUrl);
removeOutputDir(spark, outputPath);
readAndWriteActionSetsFromPaths(spark, inputActionSetPaths, outputPath);
});
}
private static void removeOutputDir(SparkSession spark, new PartitionActionSetsByPayloadTypeJob(isLookupUrl)
String path) { .run(isSparkSessionManaged, inputActionSetIds, outputPath);
HdfsSupport.remove(path, spark.sparkContext().hadoopConfiguration()); }
}
private static void readAndWriteActionSetsFromPaths(SparkSession spark, protected void run(Boolean isSparkSessionManaged, String inputActionSetIds, String outputPath) {
List<String> inputActionSetPaths,
String outputPath) {
inputActionSetPaths
.forEach(inputActionSetPath -> {
Dataset<Row> actionDS = readActionSetFromPath(spark, inputActionSetPath);
saveActions(actionDS, outputPath);
});
}
private static Dataset<Row> readActionSetFromPath(SparkSession spark, List<String> inputActionSetPaths = getIsClient().getLatestRawsetPaths(inputActionSetIds);
String path) { logger.info("inputActionSetPaths: {}", String.join(",", inputActionSetPaths));
logger.info("Reading actions from path: {}", path);
List<String> files = HdfsSupport.listFiles(path, spark.sparkContext().hadoopConfiguration()); SparkConf conf = new SparkConf();
logger.info("Found files: {}", String.join(",", files)); conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer");
JavaSparkContext sc = JavaSparkContext.fromSparkContext(spark.sparkContext()); runWithSparkSession(
return files conf,
.stream() isSparkSessionManaged,
.map(file -> { spark -> {
JavaRDD<Row> rdd = sc removeOutputDir(spark, outputPath);
.sequenceFile(file, Text.class, Text.class) readAndWriteActionSetsFromPaths(spark, inputActionSetPaths, outputPath);
.map(x -> RowFactory.create(x._1().toString(), x._2().toString())); });
return spark.createDataFrame(rdd, KV_SCHEMA) }
.withColumn("atomic_action", from_json(col("value"), ATOMIC_ACTION_SCHEMA))
.select(expr("atomic_action.*"));
})
.reduce(spark.createDataFrame(Collections.emptyList(), ATOMIC_ACTION_SCHEMA), Dataset::union);
}
private static void saveActions(Dataset<Row> actionDS, private static void removeOutputDir(SparkSession spark, String path) {
String path) { HdfsSupport.remove(path, spark.sparkContext().hadoopConfiguration());
logger.info("Saving actions to path: {}", path); }
actionDS
.write() private static void readAndWriteActionSetsFromPaths(
.partitionBy("clazz") SparkSession spark, List<String> inputActionSetPaths, String outputPath) {
.mode(SaveMode.Append) inputActionSetPaths
.parquet(path); .stream()
} .filter(path -> HdfsSupport.exists(path, spark.sparkContext().hadoopConfiguration()))
.forEach(
inputActionSetPath -> {
Dataset<Row> actionDS = readActionSetFromPath(spark, inputActionSetPath);
saveActions(actionDS, outputPath);
});
}
private static Dataset<Row> readActionSetFromPath(SparkSession spark, String path) {
logger.info("Reading actions from path: {}", path);
JavaSparkContext sc = JavaSparkContext.fromSparkContext(spark.sparkContext());
JavaRDD<Row> rdd = sc
.sequenceFile(path, Text.class, Text.class)
.map(x -> RowFactory.create(x._1().toString(), x._2().toString()));
return spark
.createDataFrame(rdd, KV_SCHEMA)
.withColumn("atomic_action", from_json(col("value"), ATOMIC_ACTION_SCHEMA))
.select(expr("atomic_action.*"));
}
private static void saveActions(Dataset<Row> actionDS, String path) {
logger.info("Saving actions to path: {}", path);
actionDS.write().partitionBy("clazz").mode(SaveMode.Append).parquet(path);
}
public ISClient getIsClient() {
return isClient;
}
public void setIsClient(ISClient isClient) {
this.isClient = isClient;
}
} }

133
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/promote/MergeAndGet.java
View File

@ -1,74 +1,87 @@
package eu.dnetlib.dhp.actionmanager.promote; package eu.dnetlib.dhp.actionmanager.promote;
import static eu.dnetlib.dhp.schema.common.ModelSupport.isSubClass;
import java.util.function.BiFunction;
import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.SerializableSupplier; import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.SerializableSupplier;
import eu.dnetlib.dhp.schema.oaf.Oaf; import eu.dnetlib.dhp.schema.oaf.Oaf;
import eu.dnetlib.dhp.schema.oaf.OafEntity; import eu.dnetlib.dhp.schema.oaf.OafEntity;
import eu.dnetlib.dhp.schema.oaf.Relation; import eu.dnetlib.dhp.schema.oaf.Relation;
import java.util.function.BiFunction; /** OAF model merging support. */
import static eu.dnetlib.dhp.schema.common.ModelSupport.isSubClass;
/**
* OAF model merging support.
*/
public class MergeAndGet { public class MergeAndGet {
private MergeAndGet() { private MergeAndGet() {
} }
/** /**
* Strategy for merging OAF model objects. * Strategy for merging OAF model objects.
* <p> * <p>
* MERGE_FROM_AND_GET: use OAF 'mergeFrom' method * MERGE_FROM_AND_GET: use OAF 'mergeFrom' method SELECT_NEWER_AND_GET: use last update timestamp to return newer
* SELECT_NEWER_AND_GET: use last update timestamp to return newer instance * instance
*/ */
public enum Strategy { public enum Strategy {
MERGE_FROM_AND_GET, SELECT_NEWER_AND_GET MERGE_FROM_AND_GET, SELECT_NEWER_AND_GET
} }
/** /**
* Returns a function for merging OAF model objects. * Returns a function for merging OAF model objects.
* *
* @param strategy Strategy to be used to merge objects * @param strategy Strategy to be used to merge objects
* @param <G> Graph table type * @param <G> Graph table type
* @param <A> Action payload type * @param <A> Action payload type
* @return BiFunction to be used to merge OAF objects * @return BiFunction to be used to merge OAF objects
*/ */
public static <G extends Oaf, A extends Oaf> SerializableSupplier<BiFunction<G, A, G>> functionFor(Strategy strategy) { public static <G extends Oaf, A extends Oaf> SerializableSupplier<BiFunction<G, A, G>> functionFor(
switch (strategy) { Strategy strategy) {
case MERGE_FROM_AND_GET: switch (strategy) {
return () -> MergeAndGet::mergeFromAndGet; case MERGE_FROM_AND_GET:
case SELECT_NEWER_AND_GET: return () -> MergeAndGet::mergeFromAndGet;
return () -> MergeAndGet::selectNewerAndGet; case SELECT_NEWER_AND_GET:
} return () -> MergeAndGet::selectNewerAndGet;
throw new RuntimeException(); }
} throw new RuntimeException();
}
private static <G extends Oaf, A extends Oaf> G mergeFromAndGet(G x, A y) { private static <G extends Oaf, A extends Oaf> G mergeFromAndGet(G x, A y) {
if (isSubClass(x, Relation.class) && isSubClass(y, Relation.class)) { if (isSubClass(x, Relation.class) && isSubClass(y, Relation.class)) {
((Relation) x).mergeFrom((Relation) y); ((Relation) x).mergeFrom((Relation) y);
return x; return x;
} else if (isSubClass(x, OafEntity.class) && isSubClass(y, OafEntity.class) && isSubClass(x, y)) { } else if (isSubClass(x, OafEntity.class)
((OafEntity) x).mergeFrom((OafEntity) y); && isSubClass(y, OafEntity.class)
return x; && isSubClass(x, y)) {
} ((OafEntity) x).mergeFrom((OafEntity) y);
throw new RuntimeException(String.format("MERGE_FROM_AND_GET incompatible types: %s, %s", return x;
x.getClass().getCanonicalName(), y.getClass().getCanonicalName())); }
} throw new RuntimeException(
String
.format(
"MERGE_FROM_AND_GET incompatible types: %s, %s",
x.getClass().getCanonicalName(), y.getClass().getCanonicalName()));
}
private static <G extends Oaf, A extends Oaf> G selectNewerAndGet(G x, A y) { private static <G extends Oaf, A extends Oaf> G selectNewerAndGet(G x, A y) {
if (x.getClass().equals(y.getClass()) && x.getLastupdatetimestamp() > y.getLastupdatetimestamp()) { if (x.getClass().equals(y.getClass())
return x; && x.getLastupdatetimestamp() > y.getLastupdatetimestamp()) {
} else if (x.getClass().equals(y.getClass()) && x.getLastupdatetimestamp() < y.getLastupdatetimestamp()) { return x;
return (G) y; } else if (x.getClass().equals(y.getClass())
} else if (isSubClass(x, y) && x.getLastupdatetimestamp() > y.getLastupdatetimestamp()) { && x.getLastupdatetimestamp() < y.getLastupdatetimestamp()) {
return x; return (G) y;
} else if (isSubClass(x, y) && x.getLastupdatetimestamp() < y.getLastupdatetimestamp()) { } else if (isSubClass(x, y) && x.getLastupdatetimestamp() > y.getLastupdatetimestamp()) {
throw new RuntimeException(String.format("SELECT_NEWER_AND_GET cannot return right type when it is not the same as left type: %s, %s", return x;
x.getClass().getCanonicalName(), y.getClass().getCanonicalName())); } else if (isSubClass(x, y) && x.getLastupdatetimestamp() < y.getLastupdatetimestamp()) {
} throw new RuntimeException(
throw new RuntimeException(String.format("SELECT_NEWER_AND_GET cannot be used when left is not subtype of right: %s, %s", String
x.getClass().getCanonicalName(), y.getClass().getCanonicalName())); .format(
} "SELECT_NEWER_AND_GET cannot return right type when it is not the same as left type: %s, %s",
x.getClass().getCanonicalName(), y.getClass().getCanonicalName()));
}
throw new RuntimeException(
String
.format(
"SELECT_NEWER_AND_GET cannot be used when left is not subtype of right: %s, %s",
x.getClass().getCanonicalName(), y.getClass().getCanonicalName()));
}
} }

381
dhp-workflows/dhp-actionmanager/src/main/java/eu/dnetlib/dhp/actionmanager/promote/PromoteActionPayloadForGraphTableJob.java
View File

@ -1,11 +1,14 @@
package eu.dnetlib.dhp.actionmanager.promote; package eu.dnetlib.dhp.actionmanager.promote;
import com.fasterxml.jackson.databind.ObjectMapper; import static eu.dnetlib.dhp.common.SparkSessionSupport.runWithSparkSession;
import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.SerializableSupplier; import static eu.dnetlib.dhp.schema.common.ModelSupport.isSubClass;
import eu.dnetlib.dhp.common.HdfsSupport;
import eu.dnetlib.dhp.application.ArgumentApplicationParser; import java.util.Objects;
import eu.dnetlib.dhp.schema.common.ModelSupport; import java.util.Optional;
import eu.dnetlib.dhp.schema.oaf.*; import java.util.function.BiFunction;
import java.util.function.Function;
import org.apache.commons.io.IOUtils; import org.apache.commons.io.IOUtils;
import org.apache.spark.SparkConf; import org.apache.spark.SparkConf;
import org.apache.spark.api.java.function.MapFunction; import org.apache.spark.api.java.function.MapFunction;
@ -16,231 +19,207 @@ import org.apache.spark.sql.SparkSession;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import java.util.Objects; import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.Optional;
import java.util.function.BiFunction;
import java.util.function.Function;
import static eu.dnetlib.dhp.schema.common.ModelSupport.isSubClass; import eu.dnetlib.dhp.application.ArgumentApplicationParser;
import static eu.dnetlib.dhp.common.SparkSessionSupport.runWithSparkSession; import eu.dnetlib.dhp.common.FunctionalInterfaceSupport.SerializableSupplier;
import eu.dnetlib.dhp.common.HdfsSupport;
import eu.dnetlib.dhp.schema.common.ModelSupport;
import eu.dnetlib.dhp.schema.oaf.*;
/** /** Applies a given action payload file to graph table of compatible type. */
* Applies a given action payload file to graph table of compatible type.
*/
public class PromoteActionPayloadForGraphTableJob { public class PromoteActionPayloadForGraphTableJob {
private static final Logger logger = LoggerFactory.getLogger(PromoteActionPayloadForGraphTableJob.class); private static final Logger logger = LoggerFactory.getLogger(PromoteActionPayloadForGraphTableJob.class);
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
public static void main(String[] args) throws Exception { public static void main(String[] args) throws Exception {
String jsonConfiguration = IOUtils.toString( String jsonConfiguration = IOUtils
PromoteActionPayloadForGraphTableJob.class .toString(
.getResourceAsStream("/eu/dnetlib/dhp/actionmanager/promote/promote_action_payload_for_graph_table_input_parameters.json")); PromoteActionPayloadForGraphTableJob.class
final ArgumentApplicationParser parser = new ArgumentApplicationParser(jsonConfiguration); .getResourceAsStream(
parser.parseArgument(args); "/eu/dnetlib/dhp/actionmanager/promote/promote_action_payload_for_graph_table_input_parameters.json"));
final ArgumentApplicationParser parser = new ArgumentApplicationParser(jsonConfiguration);
parser.parseArgument(args);
Boolean isSparkSessionManaged = Optional Boolean isSparkSessionManaged = Optional
.ofNullable(parser.get("isSparkSessionManaged")) .ofNullable(parser.get("isSparkSessionManaged"))
.map(Boolean::valueOf) .map(Boolean::valueOf)
.orElse(Boolean.TRUE); .orElse(Boolean.TRUE);
logger.info("isSparkSessionManaged: {}", isSparkSessionManaged); logger.info("isSparkSessionManaged: {}", isSparkSessionManaged);
String inputGraphTablePath = parser.get("inputGraphTablePath"); String inputGraphTablePath = parser.get("inputGraphTablePath");
logger.info("inputGraphTablePath: {}", inputGraphTablePath); logger.info("inputGraphTablePath: {}", inputGraphTablePath);
String graphTableClassName = parser.get("graphTableClassName"); String graphTableClassName = parser.get("graphTableClassName");
logger.info("graphTableClassName: {}", graphTableClassName); logger.info("graphTableClassName: {}", graphTableClassName);
String inputActionPayloadPath = parser.get("inputActionPayloadPath"); String inputActionPayloadPath = parser.get("inputActionPayloadPath");
logger.info("inputActionPayloadPath: {}", inputActionPayloadPath); logger.info("inputActionPayloadPath: {}", inputActionPayloadPath);
String actionPayloadClassName = parser.get("actionPayloadClassName"); String actionPayloadClassName = parser.get("actionPayloadClassName");
logger.info("actionPayloadClassName: {}", actionPayloadClassName); logger.info("actionPayloadClassName: {}", actionPayloadClassName);
String outputGraphTablePath = parser.get("outputGraphTablePath"); String outputGraphTablePath = parser.get("outputGraphTablePath");
logger.info("outputGraphTablePath: {}", outputGraphTablePath); logger.info("outputGraphTablePath: {}", outputGraphTablePath);
MergeAndGet.Strategy strategy = MergeAndGet.Strategy.valueOf(parser.get("mergeAndGetStrategy").toUpperCase()); MergeAndGet.Strategy strategy = MergeAndGet.Strategy.valueOf(parser.get("mergeAndGetStrategy").toUpperCase());
logger.info("strategy: {}", strategy); logger.info("strategy: {}", strategy);
Class<? extends Oaf> rowClazz = (Class<? extends Oaf>) Class.forName(graphTableClassName); Class<? extends Oaf> rowClazz = (Class<? extends Oaf>) Class.forName(graphTableClassName);
Class<? extends Oaf> actionPayloadClazz = (Class<? extends Oaf>) Class.forName(actionPayloadClassName); Class<? extends Oaf> actionPayloadClazz = (Class<? extends Oaf>) Class.forName(actionPayloadClassName);
throwIfGraphTableClassIsNotSubClassOfActionPayloadClass(rowClazz, actionPayloadClazz); throwIfGraphTableClassIsNotSubClassOfActionPayloadClass(rowClazz, actionPayloadClazz);
SparkConf conf = new SparkConf(); SparkConf conf = new SparkConf();
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer"); conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer");
conf.registerKryoClasses(ModelSupport.getOafModelClasses()); conf.registerKryoClasses(ModelSupport.getOafModelClasses());
runWithSparkSession(conf, isSparkSessionManaged, runWithSparkSession(
spark -> { conf,
removeOutputDir(spark, outputGraphTablePath); isSparkSessionManaged,
promoteActionPayloadForGraphTable(spark, spark -> {
inputGraphTablePath, removeOutputDir(spark, outputGraphTablePath);
inputActionPayloadPath, promoteActionPayloadForGraphTable(
outputGraphTablePath, spark,
strategy, inputGraphTablePath,
rowClazz, inputActionPayloadPath,
actionPayloadClazz); outputGraphTablePath,
}); strategy,
} rowClazz,
actionPayloadClazz);
});
}
private static void throwIfGraphTableClassIsNotSubClassOfActionPayloadClass(Class<? extends Oaf> rowClazz, private static void throwIfGraphTableClassIsNotSubClassOfActionPayloadClass(
Class<? extends Oaf> actionPayloadClazz) { Class<? extends Oaf> rowClazz, Class<? extends Oaf> actionPayloadClazz) {
if (!isSubClass(rowClazz, actionPayloadClazz)) { if (!isSubClass(rowClazz, actionPayloadClazz)) {
String msg = String.format("graph table class is not a subclass of action payload class: graph=%s, action=%s", String msg = String
rowClazz.getCanonicalName(), actionPayloadClazz.getCanonicalName()); .format(
throw new RuntimeException(msg); "graph table class is not a subclass of action payload class: graph=%s, action=%s",
} rowClazz.getCanonicalName(), actionPayloadClazz.getCanonicalName());
} throw new RuntimeException(msg);
}
}
private static void removeOutputDir(SparkSession spark, private static void removeOutputDir(SparkSession spark, String path) {
String path) { HdfsSupport.remove(path, spark.sparkContext().hadoopConfiguration());
HdfsSupport.remove(path, spark.sparkContext().hadoopConfiguration()); }
}
private static <G extends Oaf, A extends Oaf> void promoteActionPayloadForGraphTable(SparkSession spark, private static <G extends Oaf, A extends Oaf> void promoteActionPayloadForGraphTable(
String inputGraphTablePath, SparkSession spark,
String inputActionPayloadPath, String inputGraphTablePath,
String outputGraphTablePath, String inputActionPayloadPath,
MergeAndGet.Strategy strategy, String outputGraphTablePath,
Class<G> rowClazz, MergeAndGet.Strategy strategy,
Class<A> actionPayloadClazz) { Class<G> rowClazz,
Dataset<G> rowDS = readGraphTable(spark, inputGraphTablePath, rowClazz); Class<A> actionPayloadClazz) {
Dataset<A> actionPayloadDS = readActionPayload(spark, inputActionPayloadPath, actionPayloadClazz); Dataset<G> rowDS = readGraphTable(spark, inputGraphTablePath, rowClazz);
Dataset<A> actionPayloadDS = readActionPayload(spark, inputActionPayloadPath, actionPayloadClazz);
Dataset<G> result = promoteActionPayloadForGraphTable(rowDS, actionPayloadDS, strategy, rowClazz, actionPayloadClazz) Dataset<G> result = promoteActionPayloadForGraphTable(
.map((MapFunction<G, G>) value -> value, Encoders.bean(rowClazz)); rowDS, actionPayloadDS, strategy, rowClazz, actionPayloadClazz)
.map((MapFunction<G, G>) value -> value, Encoders.bean(rowClazz));
saveGraphTable(result, outputGraphTablePath); saveGraphTable(result, outputGraphTablePath);
} }
private static <G extends Oaf> Dataset<G> readGraphTable(SparkSession spark, private static <G extends Oaf> Dataset<G> readGraphTable(
String path, SparkSession spark, String path, Class<G> rowClazz) {
Class<G> rowClazz) { logger.info("Reading graph table from path: {}", path);
logger.info("Reading graph table from path: {}", path);
return spark
.read()
.parquet(path)
.as(Encoders.bean(rowClazz));
}
private static <A extends Oaf> Dataset<A> readActionPayload(SparkSession spark, return spark
String path, .read()
Class<A> actionPayloadClazz) { .textFile(path)
logger.info("Reading action payload from path: {}", path); .map(
return spark (MapFunction<String, G>) value -> OBJECT_MAPPER.readValue(value, rowClazz),
.read() Encoders.bean(rowClazz));
.parquet(path)
.map((MapFunction<Row, A>) value -> OBJECT_MAPPER.readValue(value.<String>getAs("payload"),
actionPayloadClazz), Encoders.bean(actionPayloadClazz));
}
private static <G extends Oaf, A extends Oaf> Dataset<G> promoteActionPayloadForGraphTable(Dataset<G> rowDS, /*
Dataset<A> actionPayloadDS, * return spark .read() .parquet(path) .as(Encoders.bean(rowClazz));
MergeAndGet.Strategy strategy, */
Class<G> rowClazz, }
Class<A> actionPayloadClazz) {
logger.info("Promoting action payload for graph table: payload={}, table={}", actionPayloadClazz.getSimpleName(), rowClazz.getSimpleName());
SerializableSupplier<Function<G, String>> rowIdFn = PromoteActionPayloadForGraphTableJob::idFn; private static <A extends Oaf> Dataset<A> readActionPayload(
SerializableSupplier<Function<A, String>> actionPayloadIdFn = PromoteActionPayloadForGraphTableJob::idFn; SparkSession spark, String path, Class<A> actionPayloadClazz) {
SerializableSupplier<BiFunction<G, A, G>> mergeRowWithActionPayloadAndGetFn = MergeAndGet.functionFor(strategy); logger.info("Reading action payload from path: {}", path);
SerializableSupplier<BiFunction<G, G, G>> mergeRowsAndGetFn = MergeAndGet.functionFor(strategy); return spark
SerializableSupplier<G> zeroFn = zeroFn(rowClazz); .read()
SerializableSupplier<Function<G, Boolean>> isNotZeroFn = PromoteActionPayloadForGraphTableJob::isNotZeroFnUsingIdOrSource; .parquet(path)
.map(
(MapFunction<Row, A>) value -> OBJECT_MAPPER
.readValue(value.<String> getAs("payload"), actionPayloadClazz),
Encoders.bean(actionPayloadClazz));
}
Dataset<G> joinedAndMerged = PromoteActionPayloadFunctions private static <G extends Oaf, A extends Oaf> Dataset<G> promoteActionPayloadForGraphTable(
.joinGraphTableWithActionPayloadAndMerge( Dataset<G> rowDS,
rowDS, Dataset<A> actionPayloadDS,
actionPayloadDS, MergeAndGet.Strategy strategy,
rowIdFn, Class<G> rowClazz,
actionPayloadIdFn, Class<A> actionPayloadClazz) {
mergeRowWithActionPayloadAndGetFn, logger
rowClazz, .info(
actionPayloadClazz); "Promoting action payload for graph table: payload={}, table={}",
actionPayloadClazz.getSimpleName(),
rowClazz.getSimpleName());
return PromoteActionPayloadFunctions SerializableSupplier<Function<G, String>> rowIdFn = ModelSupport::idFn;
.groupGraphTableByIdAndMerge( SerializableSupplier<Function<A, String>> actionPayloadIdFn = ModelSupport::idFn;
joinedAndMerged, SerializableSupplier<BiFunction<G, A, G>> mergeRowWithActionPayloadAndGetFn = MergeAndGet.functionFor(strategy);
rowIdFn, SerializableSupplier<BiFunction<G, G, G>> mergeRowsAndGetFn = MergeAndGet.functionFor(strategy);
mergeRowsAndGetFn, SerializableSupplier<G> zeroFn = zeroFn(rowClazz);
zeroFn, SerializableSupplier<Function<G, Boolean>> isNotZeroFn = PromoteActionPayloadForGraphTableJob::isNotZeroFnUsingIdOrSource;
isNotZeroFn,
rowClazz);
}
private static <T extends Oaf> Function<T, String> idFn() { Dataset<G> joinedAndMerged = PromoteActionPayloadFunctions
return x -> { .joinGraphTableWithActionPayloadAndMerge(
if (isSubClass(x, Relation.class)) { rowDS,
return idFnForRelation(x); actionPayloadDS,
} rowIdFn,
return idFnForOafEntity(x); actionPayloadIdFn,
}; mergeRowWithActionPayloadAndGetFn,
} rowClazz,
actionPayloadClazz);
private static <T extends Oaf> String idFnForRelation(T t) { return PromoteActionPayloadFunctions
Relation r = (Relation) t; .groupGraphTableByIdAndMerge(
return Optional.ofNullable(r.getSource()) joinedAndMerged, rowIdFn, mergeRowsAndGetFn, zeroFn, isNotZeroFn, rowClazz);
.map(source -> Optional.ofNullable(r.getTarget()) }
.map(target -> Optional.ofNullable(r.getRelType())
.map(relType -> Optional.ofNullable(r.getSubRelType())
.map(subRelType -> Optional.ofNullable(r.getRelClass())
.map(relClass -> String.join(source, target, relType, subRelType, relClass))
.orElse(String.join(source, target, relType, subRelType))
)
.orElse(String.join(source, target, relType))
)
.orElse(String.join(source, target))
)
.orElse(source)
)
.orElse(null);
}
private static <T extends Oaf> String idFnForOafEntity(T t) { private static <T extends Oaf> SerializableSupplier<T> zeroFn(Class<T> clazz) {
return ((OafEntity) t).getId(); switch (clazz.getCanonicalName()) {
} case "eu.dnetlib.dhp.schema.oaf.Dataset":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Dataset());
case "eu.dnetlib.dhp.schema.oaf.Datasource":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Datasource());
case "eu.dnetlib.dhp.schema.oaf.Organization":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Organization());
case "eu.dnetlib.dhp.schema.oaf.OtherResearchProduct":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.OtherResearchProduct());
case "eu.dnetlib.dhp.schema.oaf.Project":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Project());
case "eu.dnetlib.dhp.schema.oaf.Publication":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Publication());
case "eu.dnetlib.dhp.schema.oaf.Relation":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Relation());
case "eu.dnetlib.dhp.schema.oaf.Software":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Software());
default:
throw new RuntimeException("unknown class: " + clazz.getCanonicalName());
}
}
private static <T extends Oaf> SerializableSupplier<T> zeroFn(Class<T> clazz) { private static <T extends Oaf> Function<T, Boolean> isNotZeroFnUsingIdOrSource() {
switch (clazz.getCanonicalName()) { return t -> {
case "eu.dnetlib.dhp.schema.oaf.Dataset": if (isSubClass(t, Relation.class)) {
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Dataset()); return Objects.nonNull(((Relation) t).getSource());
case "eu.dnetlib.dhp.schema.oaf.Datasource": }
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Datasource()); return Objects.nonNull(((OafEntity) t).getId());
case "eu.dnetlib.dhp.schema.oaf.Organization": };
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Organization()); }
case "eu.dnetlib.dhp.schema.oaf.OtherResearchProduct":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.OtherResearchProduct());
case "eu.dnetlib.dhp.schema.oaf.Project":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Project());
case "eu.dnetlib.dhp.schema.oaf.Publication":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Publication());
case "eu.dnetlib.dhp.schema.oaf.Relation":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Relation());
case "eu.dnetlib.dhp.schema.oaf.Software":
return () -> clazz.cast(new eu.dnetlib.dhp.schema.oaf.Software());
default:
throw new RuntimeException("unknown class: " + clazz.getCanonicalName());
}
}
private static <T extends Oaf> Function<T, Boolean> isNotZeroFnUsingIdOrSource() { private static <G extends Oaf> void saveGraphTable(Dataset<G> result, String path) {
return t -> { logger.info("Saving graph table to path: {}", path);
if (isSubClass(t, Relation.class)) { result.toJSON().write().option("compression", "gzip").text(path);
return Objects.nonNull(((Relation) t).getSource()); }
}
return Objects.nonNull(((OafEntity) t).getId());
};
}
private static <G extends Oaf> void saveGraphTable(Dataset<G> result,
String path) {
logger.info("Saving graph table to path: {}", path);
result
.toJSON()
.write()
.option("compression", "gzip")
.text(path);
}
} }

Some files were not shown because too many files have changed in this diff Show More