First commit
This commit is contained in:
commit
eae69f8ee4
|
@ -0,0 +1,13 @@
|
||||||
|
This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
# Changelog for "conductor-setup"
|
||||||
|
|
||||||
|
## [v0.1.0-SNAPSHOT]
|
||||||
|
|
||||||
|
- First release. It provides Conductor HA with 2 instances. (#19689).<br>
|
||||||
|
Currently the Docker Swamrs Stack deploys:
|
||||||
|
- 4 Dynomites nodes (2 shards with 1 replication each one, handled by Dynomite directly) backed by Redis DB in the same container
|
||||||
|
- 2 Conductor server nodes with 2 replicas handled by Swarm
|
||||||
|
- 2 Conductor UI nodes with 2 replicas handled by Swarm
|
||||||
|
- 1 Elasticsearch node
|
||||||
|
|
|
@ -0,0 +1,26 @@
|
||||||
|
# Acknowledgments
|
||||||
|
|
||||||
|
The projects leading to this software have received funding from a series of European Union programmes including:
|
||||||
|
|
||||||
|
- the Sixth Framework Programme for Research and Technological Development
|
||||||
|
- [DILIGENT](https://cordis.europa.eu/project/id/004260) (grant no. 004260).
|
||||||
|
- the Seventh Framework Programme for research, technological development and demonstration
|
||||||
|
- [D4Science](https://cordis.europa.eu/project/id/212488) (grant no. 212488);
|
||||||
|
- [D4Science-II](https://cordis.europa.eu/project/id/239019) (grant no.239019);
|
||||||
|
- [ENVRI](https://cordis.europa.eu/project/id/283465) (grant no. 283465);
|
||||||
|
- [iMarine](https://cordis.europa.eu/project/id/283644) (grant no. 283644);
|
||||||
|
- [EUBrazilOpenBio](https://cordis.europa.eu/project/id/288754) (grant no. 288754).
|
||||||
|
- the H2020 research and innovation programme
|
||||||
|
- [SoBigData](https://cordis.europa.eu/project/id/654024) (grant no. 654024);
|
||||||
|
- [PARTHENOS](https://cordis.europa.eu/project/id/654119) (grant no. 654119);
|
||||||
|
- [EGI-Engage](https://cordis.europa.eu/project/id/654142) (grant no. 654142);
|
||||||
|
- [ENVRI PLUS](https://cordis.europa.eu/project/id/654182) (grant no. 654182);
|
||||||
|
- [BlueBRIDGE](https://cordis.europa.eu/project/id/675680) (grant no. 675680);
|
||||||
|
- [PerformFISH](https://cordis.europa.eu/project/id/727610) (grant no. 727610);
|
||||||
|
- [AGINFRA PLUS](https://cordis.europa.eu/project/id/731001) (grant no. 731001);
|
||||||
|
- [DESIRA](https://cordis.europa.eu/project/id/818194) (grant no. 818194);
|
||||||
|
- [ARIADNEplus](https://cordis.europa.eu/project/id/823914) (grant no. 823914);
|
||||||
|
- [RISIS 2](https://cordis.europa.eu/project/id/824091) (grant no. 824091);
|
||||||
|
- [EOSC-Pillar](https://cordis.europa.eu/project/id/857650) (grant no. 857650);
|
||||||
|
- [Blue Cloud](https://cordis.europa.eu/project/id/862409) (grant no. 862409);
|
||||||
|
- [SoBigData-PlusPlus](https://cordis.europa.eu/project/id/871042) (grant no. 871042);
|
|
@ -0,0 +1,312 @@
|
||||||
|
#European Union Public Licence V.1.1
|
||||||
|
|
||||||
|
##*EUPL © the European Community 2007*
|
||||||
|
|
||||||
|
|
||||||
|
This **European Union Public Licence** (the **“EUPL”**) applies to the Work or Software
|
||||||
|
(as defined below) which is provided under the terms of this Licence. Any use of
|
||||||
|
the Work, other than as authorised under this Licence is prohibited (to the
|
||||||
|
extent such use is covered by a right of the copyright holder of the Work).
|
||||||
|
|
||||||
|
The Original Work is provided under the terms of this Licence when the Licensor
|
||||||
|
(as defined below) has placed the following notice immediately following the
|
||||||
|
copyright notice for the Original Work:
|
||||||
|
|
||||||
|
**Licensed under the EUPL V.1.1**
|
||||||
|
|
||||||
|
or has expressed by any other mean his willingness to license under the EUPL.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##1. Definitions
|
||||||
|
|
||||||
|
In this Licence, the following terms have the following meaning:
|
||||||
|
|
||||||
|
- The Licence: this Licence.
|
||||||
|
|
||||||
|
- The Original Work or the Software: the software distributed and/or
|
||||||
|
communicated by the Licensor under this Licence, available as Source Code and
|
||||||
|
also as Executable Code as the case may be.
|
||||||
|
|
||||||
|
- Derivative Works: the works or software that could be created by the Licensee,
|
||||||
|
based upon the Original Work or modifications thereof. This Licence does not
|
||||||
|
define the extent of modification or dependence on the Original Work required
|
||||||
|
in order to classify a work as a Derivative Work; this extent is determined by
|
||||||
|
copyright law applicable in the country mentioned in Article 15.
|
||||||
|
|
||||||
|
- The Work: the Original Work and/or its Derivative Works.
|
||||||
|
|
||||||
|
- The Source Code: the human-readable form of the Work which is the most
|
||||||
|
convenient for people to study and modify.
|
||||||
|
|
||||||
|
- The Executable Code: any code which has generally been compiled and which is
|
||||||
|
meant to be interpreted by a computer as a program.
|
||||||
|
|
||||||
|
- The Licensor: the natural or legal person that distributes and/or communicates
|
||||||
|
the Work under the Licence.
|
||||||
|
|
||||||
|
- Contributor(s): any natural or legal person who modifies the Work under the
|
||||||
|
Licence, or otherwise contributes to the creation of a Derivative Work.
|
||||||
|
|
||||||
|
- The Licensee or “You”: any natural or legal person who makes any usage of the
|
||||||
|
Software under the terms of the Licence.
|
||||||
|
|
||||||
|
- Distribution and/or Communication: any act of selling, giving, lending,
|
||||||
|
renting, distributing, communicating, transmitting, or otherwise making
|
||||||
|
available, on-line or off-line, copies of the Work or providing access to its
|
||||||
|
essential functionalities at the disposal of any other natural or legal
|
||||||
|
person.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##2. Scope of the rights granted by the Licence
|
||||||
|
|
||||||
|
The Licensor hereby grants You a world-wide, royalty-free, non-exclusive,
|
||||||
|
sub-licensable licence to do the following, for the duration of copyright vested
|
||||||
|
in the Original Work:
|
||||||
|
|
||||||
|
- use the Work in any circumstance and for all usage, reproduce the Work, modify
|
||||||
|
- the Original Work, and make Derivative Works based upon the Work, communicate
|
||||||
|
- to the public, including the right to make available or display the Work or
|
||||||
|
- copies thereof to the public and perform publicly, as the case may be, the
|
||||||
|
- Work, distribute the Work or copies thereof, lend and rent the Work or copies
|
||||||
|
- thereof, sub-license rights in the Work or copies thereof.
|
||||||
|
|
||||||
|
Those rights can be exercised on any media, supports and formats, whether now
|
||||||
|
known or later invented, as far as the applicable law permits so.
|
||||||
|
|
||||||
|
In the countries where moral rights apply, the Licensor waives his right to
|
||||||
|
exercise his moral right to the extent allowed by law in order to make effective
|
||||||
|
the licence of the economic rights here above listed.
|
||||||
|
|
||||||
|
The Licensor grants to the Licensee royalty-free, non exclusive usage rights to
|
||||||
|
any patents held by the Licensor, to the extent necessary to make use of the
|
||||||
|
rights granted on the Work under this Licence.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##3. Communication of the Source Code
|
||||||
|
|
||||||
|
The Licensor may provide the Work either in its Source Code form, or as
|
||||||
|
Executable Code. If the Work is provided as Executable Code, the Licensor
|
||||||
|
provides in addition a machine-readable copy of the Source Code of the Work
|
||||||
|
along with each copy of the Work that the Licensor distributes or indicates, in
|
||||||
|
a notice following the copyright notice attached to the Work, a repository where
|
||||||
|
the Source Code is easily and freely accessible for as long as the Licensor
|
||||||
|
continues to distribute and/or communicate the Work.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##4. Limitations on copyright
|
||||||
|
|
||||||
|
Nothing in this Licence is intended to deprive the Licensee of the benefits from
|
||||||
|
any exception or limitation to the exclusive rights of the rights owners in the
|
||||||
|
Original Work or Software, of the exhaustion of those rights or of other
|
||||||
|
applicable limitations thereto.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##5. Obligations of the Licensee
|
||||||
|
|
||||||
|
The grant of the rights mentioned above is subject to some restrictions and
|
||||||
|
obligations imposed on the Licensee. Those obligations are the following:
|
||||||
|
|
||||||
|
Attribution right: the Licensee shall keep intact all copyright, patent or
|
||||||
|
trademarks notices and all notices that refer to the Licence and to the
|
||||||
|
disclaimer of warranties. The Licensee must include a copy of such notices and a
|
||||||
|
copy of the Licence with every copy of the Work he/she distributes and/or
|
||||||
|
communicates. The Licensee must cause any Derivative Work to carry prominent
|
||||||
|
notices stating that the Work has been modified and the date of modification.
|
||||||
|
|
||||||
|
Copyleft clause: If the Licensee distributes and/or communicates copies of the
|
||||||
|
Original Works or Derivative Works based upon the Original Work, this
|
||||||
|
Distribution and/or Communication will be done under the terms of this Licence
|
||||||
|
or of a later version of this Licence unless the Original Work is expressly
|
||||||
|
distributed only under this version of the Licence. The Licensee (becoming
|
||||||
|
Licensor) cannot offer or impose any additional terms or conditions on the Work
|
||||||
|
or Derivative Work that alter or restrict the terms of the Licence.
|
||||||
|
|
||||||
|
Compatibility clause: If the Licensee Distributes and/or Communicates Derivative
|
||||||
|
Works or copies thereof based upon both the Original Work and another work
|
||||||
|
licensed under a Compatible Licence, this Distribution and/or Communication can
|
||||||
|
be done under the terms of this Compatible Licence. For the sake of this clause,
|
||||||
|
“Compatible Licence” refers to the licences listed in the appendix attached to
|
||||||
|
this Licence. Should the Licensee’s obligations under the Compatible Licence
|
||||||
|
conflict with his/her obligations under this Licence, the obligations of the
|
||||||
|
Compatible Licence shall prevail.
|
||||||
|
|
||||||
|
Provision of Source Code: When distributing and/or communicating copies of the
|
||||||
|
Work, the Licensee will provide a machine-readable copy of the Source Code or
|
||||||
|
indicate a repository where this Source will be easily and freely available for
|
||||||
|
as long as the Licensee continues to distribute and/or communicate the Work.
|
||||||
|
|
||||||
|
Legal Protection: This Licence does not grant permission to use the trade names,
|
||||||
|
trademarks, service marks, or names of the Licensor, except as required for
|
||||||
|
reasonable and customary use in describing the origin of the Work and
|
||||||
|
reproducing the content of the copyright notice.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##6. Chain of Authorship
|
||||||
|
|
||||||
|
The original Licensor warrants that the copyright in the Original Work granted
|
||||||
|
hereunder is owned by him/her or licensed to him/her and that he/she has the
|
||||||
|
power and authority to grant the Licence.
|
||||||
|
|
||||||
|
Each Contributor warrants that the copyright in the modifications he/she brings
|
||||||
|
to the Work are owned by him/her or licensed to him/her and that he/she has the
|
||||||
|
power and authority to grant the Licence.
|
||||||
|
|
||||||
|
Each time You accept the Licence, the original Licensor and subsequent
|
||||||
|
Contributors grant You a licence to their contributions to the Work, under the
|
||||||
|
terms of this Licence.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##7. Disclaimer of Warranty
|
||||||
|
|
||||||
|
The Work is a work in progress, which is continuously improved by numerous
|
||||||
|
contributors. It is not a finished work and may therefore contain defects or
|
||||||
|
“bugs” inherent to this type of software development.
|
||||||
|
|
||||||
|
For the above reason, the Work is provided under the Licence on an “as is” basis
|
||||||
|
and without warranties of any kind concerning the Work, including without
|
||||||
|
limitation merchantability, fitness for a particular purpose, absence of defects
|
||||||
|
or errors, accuracy, non-infringement of intellectual property rights other than
|
||||||
|
copyright as stated in Article 6 of this Licence.
|
||||||
|
|
||||||
|
This disclaimer of warranty is an essential part of the Licence and a condition
|
||||||
|
for the grant of any rights to the Work.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##8. Disclaimer of Liability
|
||||||
|
|
||||||
|
Except in the cases of wilful misconduct or damages directly caused to natural
|
||||||
|
persons, the Licensor will in no event be liable for any direct or indirect,
|
||||||
|
material or moral, damages of any kind, arising out of the Licence or of the use
|
||||||
|
of the Work, including without limitation, damages for loss of goodwill, work
|
||||||
|
stoppage, computer failure or malfunction, loss of data or any commercial
|
||||||
|
damage, even if the Licensor has been advised of the possibility of such
|
||||||
|
damage. However, the Licensor will be liable under statutory product liability
|
||||||
|
laws as far such laws apply to the Work.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##9. Additional agreements
|
||||||
|
|
||||||
|
While distributing the Original Work or Derivative Works, You may choose to
|
||||||
|
conclude an additional agreement to offer, and charge a fee for, acceptance of
|
||||||
|
support, warranty, indemnity, or other liability obligations and/or services
|
||||||
|
consistent with this Licence. However, in accepting such obligations, You may
|
||||||
|
act only on your own behalf and on your sole responsibility, not on behalf of
|
||||||
|
the original Licensor or any other Contributor, and only if You agree to
|
||||||
|
indemnify, defend, and hold each Contributor harmless for any liability incurred
|
||||||
|
by, or claims asserted against such Contributor by the fact You have accepted
|
||||||
|
any such warranty or additional liability.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##10. Acceptance of the Licence
|
||||||
|
|
||||||
|
The provisions of this Licence can be accepted by clicking on an icon “I agree”
|
||||||
|
placed under the bottom of a window displaying the text of this Licence or by
|
||||||
|
affirming consent in any other similar way, in accordance with the rules of
|
||||||
|
applicable law. Clicking on that icon indicates your clear and irrevocable
|
||||||
|
acceptance of this Licence and all of its terms and conditions.
|
||||||
|
|
||||||
|
Similarly, you irrevocably accept this Licence and all of its terms and
|
||||||
|
conditions by exercising any rights granted to You by Article 2 of this Licence,
|
||||||
|
such as the use of the Work, the creation by You of a Derivative Work or the
|
||||||
|
Distribution and/or Communication by You of the Work or copies thereof.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##11. Information to the public
|
||||||
|
|
||||||
|
In case of any Distribution and/or Communication of the Work by means of
|
||||||
|
electronic communication by You (for example, by offering to download the Work
|
||||||
|
from a remote location) the distribution channel or media (for example, a
|
||||||
|
website) must at least provide to the public the information requested by the
|
||||||
|
applicable law regarding the Licensor, the Licence and the way it may be
|
||||||
|
accessible, concluded, stored and reproduced by the Licensee.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##12. Termination of the Licence
|
||||||
|
|
||||||
|
The Licence and the rights granted hereunder will terminate automatically upon
|
||||||
|
any breach by the Licensee of the terms of the Licence.
|
||||||
|
|
||||||
|
Such a termination will not terminate the licences of any person who has
|
||||||
|
received the Work from the Licensee under the Licence, provided such persons
|
||||||
|
remain in full compliance with the Licence.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##13. Miscellaneous
|
||||||
|
|
||||||
|
Without prejudice of Article 9 above, the Licence represents the complete
|
||||||
|
agreement between the Parties as to the Work licensed hereunder.
|
||||||
|
|
||||||
|
If any provision of the Licence is invalid or unenforceable under applicable
|
||||||
|
law, this will not affect the validity or enforceability of the Licence as a
|
||||||
|
whole. Such provision will be construed and/or reformed so as necessary to make
|
||||||
|
it valid and enforceable.
|
||||||
|
|
||||||
|
The European Commission may publish other linguistic versions and/or new
|
||||||
|
versions of this Licence, so far this is required and reasonable, without
|
||||||
|
reducing the scope of the rights granted by the Licence. New versions of the
|
||||||
|
Licence will be published with a unique version number.
|
||||||
|
|
||||||
|
All linguistic versions of this Licence, approved by the European Commission,
|
||||||
|
have identical value. Parties can take advantage of the linguistic version of
|
||||||
|
their choice.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##14. Jurisdiction
|
||||||
|
|
||||||
|
Any litigation resulting from the interpretation of this License, arising
|
||||||
|
between the European Commission, as a Licensor, and any Licensee, will be
|
||||||
|
subject to the jurisdiction of the Court of Justice of the European Communities,
|
||||||
|
as laid down in article 238 of the Treaty establishing the European Community.
|
||||||
|
|
||||||
|
Any litigation arising between Parties, other than the European Commission, and
|
||||||
|
resulting from the interpretation of this License, will be subject to the
|
||||||
|
exclusive jurisdiction of the competent court where the Licensor resides or
|
||||||
|
conducts its primary business.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
##15. Applicable Law
|
||||||
|
|
||||||
|
This Licence shall be governed by the law of the European Union country where
|
||||||
|
the Licensor resides or has his registered office.
|
||||||
|
|
||||||
|
This licence shall be governed by the Belgian law if:
|
||||||
|
|
||||||
|
- a litigation arises between the European Commission, as a Licensor, and any
|
||||||
|
- Licensee; the Licensor, other than the European Commission, has no residence
|
||||||
|
- or registered office inside a European Union country.
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
##Appendix
|
||||||
|
|
||||||
|
|
||||||
|
**“Compatible Licences”** according to article 5 EUPL are:
|
||||||
|
|
||||||
|
|
||||||
|
- GNU General Public License (GNU GPL) v. 2
|
||||||
|
|
||||||
|
- Open Software License (OSL) v. 2.1, v. 3.0
|
||||||
|
|
||||||
|
- Common Public License v. 1.0
|
||||||
|
|
||||||
|
- Eclipse Public License v. 1.0
|
||||||
|
|
||||||
|
- Cecill v. 2.0
|
||||||
|
|
|
@ -0,0 +1,57 @@
|
||||||
|
# Conductor Setup
|
||||||
|
|
||||||
|
**Conductor Setup** is composed by a Docker image script that should be used to build the autodynomite image, and 3 different Docker Compose Swarm YAML files to deploy the Conductor in HA.
|
||||||
|
|
||||||
|
## Structure of the project
|
||||||
|
|
||||||
|
The AutoDynomite Docker image script file is present in `dynomite` folder.
|
||||||
|
The Docker Compose Swarm files are present in the `stack` folder.
|
||||||
|
|
||||||
|
## Built With
|
||||||
|
|
||||||
|
* [Docker](https://www.docker.com)
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
The provided Docker stack files provide the following configuration:
|
||||||
|
- 4 Dynomites nodes (2 shards with 1 replication each one, handled by Dynomite directly) backed by Redis DB in the same container
|
||||||
|
- 2 Conductor Server nodes with 2 replicas handled by Swarm
|
||||||
|
- 2 Conductor UI nodes with 2 replicas handled by Swarm
|
||||||
|
- 1 Elasticsearch node
|
||||||
|
|
||||||
|
Build the Docker `autodinomite` image with the `Dockerfile` present in the dynomite folder and launch the three Docker Compose Swarm files is sequence:
|
||||||
|
- dynomite-swarm.yaml
|
||||||
|
- elasticsearch-swarm.yaml
|
||||||
|
- conductor-swarm.yaml
|
||||||
|
|
||||||
|
The command to be executed should looks like: `docker stack deploy -c dynomite-swarm.yaml -c elasticsearch-swarm.yaml -c conductor-swarm.yaml [your stack name]`
|
||||||
|
|
||||||
|
If you plan to deploy more than 4 nodes for dynomite you should modify the `dynomite-swarm.yaml` and the `seeds.list` files as per your needs.
|
||||||
|
The `conductor-swarm-config.properties` should be let unmodified.
|
||||||
|
|
||||||
|
## Change log
|
||||||
|
|
||||||
|
See [CHANGELOG.md](CHANGELOG.md).
|
||||||
|
|
||||||
|
## Authors
|
||||||
|
|
||||||
|
* **Marco Lettere** ([Nubisware S.r.l.](http://www.nubisware.com))
|
||||||
|
* **Mauro Mugnaini** ([Nubisware S.r.l.](http://www.nubisware.com))
|
||||||
|
|
||||||
|
## How to Cite this Software
|
||||||
|
[Intentionally left blank]
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This project is licensed under the EUPL V.1.1 License - see the [LICENSE.md](LICENSE.md) file for details.
|
||||||
|
|
||||||
|
## About the gCube Framework
|
||||||
|
This software is part of the [gCubeFramework](https://www.gcube-system.org/ "gCubeFramework"): an
|
||||||
|
open-source software toolkit used for building and operating Hybrid Data
|
||||||
|
Infrastructures enabling the dynamic deployment of Virtual Research Environments
|
||||||
|
by favouring the realisation of reuse oriented policies.
|
||||||
|
|
||||||
|
The projects leading to this software have received funding from a series of European Union programmes see [FUNDING.md](FUNDING.md)
|
||||||
|
|
||||||
|
## Acknowledgments
|
||||||
|
[Intentionally left blank]
|
|
@ -0,0 +1,63 @@
|
||||||
|
#
|
||||||
|
# Dockerfile derived from the good work of 'Iannis Papapanagiotou' (https://github.com/ipapapa/DynomiteDocker/blob/master/Dockerfile)
|
||||||
|
# It uses the startup.sh script to generate the dynomite conf YAML based on seeds.conf file provided via 'docker swarm configs' directive
|
||||||
|
# as '/dynomite/seeds.conf' and the DYNO_NODE env. variable that identifies the node itself (that must exactly match one of the lines in the seeds file)
|
||||||
|
#
|
||||||
|
# Set the base image to Ubuntu
|
||||||
|
FROM ubuntu:14.04
|
||||||
|
|
||||||
|
# File Author / Maintainer
|
||||||
|
MAINTAINER Marco Lettere, Mauro Mugnaini - Nubisware S.r.l.
|
||||||
|
|
||||||
|
# Update the repository sources list and Install package Build Essential
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
autoconf \
|
||||||
|
build-essential \
|
||||||
|
dh-autoreconf \
|
||||||
|
git \
|
||||||
|
libssl-dev \
|
||||||
|
libtool \
|
||||||
|
python-software-properties\
|
||||||
|
redis-server \
|
||||||
|
tcl8.5
|
||||||
|
|
||||||
|
# Clone the Dynomite Git
|
||||||
|
RUN git clone https://github.com/Netflix/dynomite.git
|
||||||
|
RUN echo 'Git repo has been cloned in your Docker VM'
|
||||||
|
WORKDIR dynomite/
|
||||||
|
|
||||||
|
# Autoreconf
|
||||||
|
RUN autoreconf -fvi \
|
||||||
|
&& ./configure --enable-debug=log \
|
||||||
|
&& CFLAGS="-ggdb3 -O0" ./configure --enable-debug=full \
|
||||||
|
&& make \
|
||||||
|
&& make install
|
||||||
|
|
||||||
|
##################### INSTALLATION ENDS #####################
|
||||||
|
|
||||||
|
# Expose the peer port
|
||||||
|
#RUN echo 'Exposing peer port 8101'
|
||||||
|
EXPOSE 8101
|
||||||
|
|
||||||
|
# Expose the underlying Redis port
|
||||||
|
#RUN echo 'Exposing Redis port 22122'
|
||||||
|
EXPOSE 22122
|
||||||
|
|
||||||
|
# Expose the stats/admin port
|
||||||
|
#RUN echo 'Exposing stats/admin port 22222'
|
||||||
|
EXPOSE 22222
|
||||||
|
|
||||||
|
# Default port to acccess Dynomite
|
||||||
|
#RUN echo 'Exposing client port for Dynomite 8102'
|
||||||
|
EXPOSE 8102
|
||||||
|
|
||||||
|
|
||||||
|
# Setting overcommit for Redis to be able to do BGSAVE/BGREWRITEAOF
|
||||||
|
RUN sysctl vm.overcommit_memory=1
|
||||||
|
|
||||||
|
# Set the entry-point to be the startup script
|
||||||
|
ENTRYPOINT ["/dynomite/startup.sh"]
|
||||||
|
|
||||||
|
#RUN echo 'copy startup file...'
|
||||||
|
COPY scripts/startup.sh ./startup.sh
|
||||||
|
#RUN echo '... done'
|
|
@ -0,0 +1,31 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# GENERATE a dynomite yaml configuration based on DYNO_NODE env. variable and 'seeds.list' file.
|
||||||
|
|
||||||
|
NODE=${DYNO_NODE:-"dynomite:8101:rack-1:datacenter-1:0"}
|
||||||
|
IFS=':' read -r -a NODEandPORTandRACKandDATACENTERandTOKEN <<< "$NODE"
|
||||||
|
|
||||||
|
echo "I am $NODE"
|
||||||
|
|
||||||
|
{
|
||||||
|
printf 'dyn_o_mite:\n'
|
||||||
|
printf ' datacenter: %s\n' "${NODEandPORTandRACKandDATACENTERandTOKEN[3]}"
|
||||||
|
printf ' rack: %s\n' "${NODEandPORTandRACKandDATACENTERandTOKEN[2]}"
|
||||||
|
printf ' dyn_listen: 0.0.0.0:8101\n'
|
||||||
|
printf ' listen: 0.0.0.0:8102\n'
|
||||||
|
printf ' dyn_seed_provider: simple_provider\n'
|
||||||
|
printf ' dyn_seeds:\n'
|
||||||
|
while IFS= read -r SEED_LINE; do
|
||||||
|
if [ "$SEED_LINE" != "$NODE" ]; then
|
||||||
|
printf ' - %s\n' "$SEED_LINE";
|
||||||
|
fi
|
||||||
|
done < seeds.list
|
||||||
|
printf ' tokens: %s\n' "${NODEandPORTandRACKandDATACENTERandTOKEN[4]}"
|
||||||
|
printf ' servers:\n'
|
||||||
|
printf ' - 127.0.0.1:22122:1\n'
|
||||||
|
} > /dynomite/auto_dynomite.yml
|
||||||
|
|
||||||
|
#Start redis server on 22122
|
||||||
|
redis-server --port 22122 &
|
||||||
|
|
||||||
|
src/dynomite --conf-file=/dynomite/auto_dynomite.yml -v11
|
|
@ -0,0 +1,58 @@
|
||||||
|
# Servers.
|
||||||
|
conductor.jetty.server.enabled=true
|
||||||
|
conductor.grpc.server.enabled=false
|
||||||
|
|
||||||
|
# Database persistence model. Possible values are memory, redis, and dynomite.
|
||||||
|
# If ommitted, the persistence used is memory
|
||||||
|
#
|
||||||
|
# memory : The data is stored in memory and lost when the server dies. Useful for testing or demo
|
||||||
|
# redis : non-Dynomite based redis instance
|
||||||
|
# dynomite : Dynomite cluster. Use this for HA configuration.
|
||||||
|
|
||||||
|
db=dynomite
|
||||||
|
|
||||||
|
# Dynomite Cluster details.
|
||||||
|
# format is host:port:rack separated by semicolon
|
||||||
|
workflow.dynomite.cluster.hosts=dynomite1:8102:us-east-1b;dynomite2:8102:us-east-1b;dynomite3:8102:us-east-2b;dynomite4:8102:us-east-2b
|
||||||
|
|
||||||
|
# Dynomite cluster name
|
||||||
|
workflow.dynomite.cluster.name=dyno1
|
||||||
|
|
||||||
|
# Namespace for the keys stored in Dynomite/Redis
|
||||||
|
workflow.namespace.prefix=conductor
|
||||||
|
|
||||||
|
# Namespace prefix for the dyno queues
|
||||||
|
workflow.namespace.queue.prefix=conductor_queues
|
||||||
|
|
||||||
|
# No. of threads allocated to dyno-queues (optional)
|
||||||
|
queues.dynomite.threads=10
|
||||||
|
|
||||||
|
# Non-quorum port used to connect to local redis. Used by dyno-queues.
|
||||||
|
# When using redis directly, set this to the same port as redis server
|
||||||
|
# For Dynomite, this is 22122 by default or the local redis-server port used by Dynomite.
|
||||||
|
queues.dynomite.nonQuorum.port=22122
|
||||||
|
|
||||||
|
# Elastic search instance type. Possible values are memory and external.
|
||||||
|
# If not specified, the instance type will be embedded in memory
|
||||||
|
#
|
||||||
|
# memory: The instance is created in memory and lost when the server dies. Useful for development and testing.
|
||||||
|
# external: Elastic search instance runs outside of the server. Data is persisted and does not get lost when
|
||||||
|
# the server dies. Useful for more stable environments like staging or production.
|
||||||
|
workflow.elasticsearch.instanceType=external
|
||||||
|
|
||||||
|
# Transport address to elasticsearch
|
||||||
|
workflow.elasticsearch.url=elasticsearch:9300
|
||||||
|
|
||||||
|
# Name of the elasticsearch cluster
|
||||||
|
workflow.elasticsearch.index.name=conductor
|
||||||
|
|
||||||
|
# Additional modules (optional)
|
||||||
|
# conductor.additional.modules=class_extending_com.google.inject.AbstractModule
|
||||||
|
|
||||||
|
# Additional modules for metrics collection (optional)
|
||||||
|
# conductor.additional.modules=com.netflix.conductor.contribs.metrics.MetricsRegistryModule,com.netflix.conductor.contribs.metrics.LoggingMetricsModule
|
||||||
|
# com.netflix.conductor.contribs.metrics.LoggingMetricsModule.reportPeriodSeconds=15
|
||||||
|
|
||||||
|
# Load sample kitchen sink workflow
|
||||||
|
loadSample=false
|
||||||
|
|
|
@ -0,0 +1,59 @@
|
||||||
|
version: '3.6'
|
||||||
|
|
||||||
|
services:
|
||||||
|
conductor-server:
|
||||||
|
environment:
|
||||||
|
- CONFIG_PROP=conductor-swarm-config.properties
|
||||||
|
image: nubisware/conductor-server
|
||||||
|
networks:
|
||||||
|
- conductor-network
|
||||||
|
ports:
|
||||||
|
- "8080:8080"
|
||||||
|
depends_on:
|
||||||
|
- elasticsearch
|
||||||
|
- dynomite1
|
||||||
|
- dynomite2
|
||||||
|
deploy:
|
||||||
|
mode: replicated
|
||||||
|
replicas: 2
|
||||||
|
#endpoint_mode: dnsrr
|
||||||
|
placement:
|
||||||
|
constraints: [node.role == worker]
|
||||||
|
restart_policy:
|
||||||
|
condition: on-failure
|
||||||
|
delay: 5s
|
||||||
|
max_attempts: 3
|
||||||
|
window: 120s
|
||||||
|
configs:
|
||||||
|
- source: swarm-config
|
||||||
|
target: /app/config/conductor-swarm-config.properties
|
||||||
|
|
||||||
|
logging:
|
||||||
|
driver: "journald"
|
||||||
|
|
||||||
|
conductor-ui:
|
||||||
|
environment:
|
||||||
|
- WF_SERVER=http://conductor-server:8080/api/
|
||||||
|
image: nubisware/conductor-ui
|
||||||
|
networks:
|
||||||
|
- conductor-network
|
||||||
|
ports:
|
||||||
|
- "5000:5000"
|
||||||
|
deploy:
|
||||||
|
mode: replicated
|
||||||
|
replicas: 2
|
||||||
|
#endpoint_mode: dnsrr
|
||||||
|
placement:
|
||||||
|
constraints: [node.role == worker]
|
||||||
|
restart_policy:
|
||||||
|
condition: on-failure
|
||||||
|
delay: 5s
|
||||||
|
max_attempts: 3
|
||||||
|
window: 120s
|
||||||
|
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
||||||
|
|
||||||
|
configs:
|
||||||
|
swarm-config:
|
||||||
|
file: ./conductor-swarm-config.properties
|
|
@ -0,0 +1,101 @@
|
||||||
|
version: '3.6'
|
||||||
|
|
||||||
|
services:
|
||||||
|
dynomite1:
|
||||||
|
environment:
|
||||||
|
- DYNO_NODE=dynomite1:8101:rack-1:d4s:0
|
||||||
|
image: nubisware/autodynomite:latest
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
||||||
|
logging:
|
||||||
|
driver: "journald"
|
||||||
|
deploy:
|
||||||
|
mode: replicated
|
||||||
|
replicas: 1
|
||||||
|
endpoint_mode: dnsrr
|
||||||
|
placement:
|
||||||
|
constraints: [node.role == worker]
|
||||||
|
restart_policy:
|
||||||
|
condition: on-failure
|
||||||
|
delay: 5s
|
||||||
|
max_attempts: 3
|
||||||
|
window: 120s
|
||||||
|
configs:
|
||||||
|
- source: seeds.list
|
||||||
|
target: /dynomite/seeds.list
|
||||||
|
|
||||||
|
dynomite2:
|
||||||
|
environment:
|
||||||
|
- DYNO_NODE=dynomite2:8101:rack-1:d4s:2147483647
|
||||||
|
image: nubisware/autodynomite:latest
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
||||||
|
logging:
|
||||||
|
driver: "journald"
|
||||||
|
deploy:
|
||||||
|
mode: replicated
|
||||||
|
replicas: 1
|
||||||
|
endpoint_mode: dnsrr
|
||||||
|
placement:
|
||||||
|
constraints: [node.role == worker]
|
||||||
|
restart_policy:
|
||||||
|
condition: on-failure
|
||||||
|
delay: 5s
|
||||||
|
max_attempts: 3
|
||||||
|
window: 120s
|
||||||
|
configs:
|
||||||
|
- source: seeds.list
|
||||||
|
target: /dynomite/seeds.list
|
||||||
|
|
||||||
|
dynomite3:
|
||||||
|
environment:
|
||||||
|
- DYNO_NODE=dynomite3:8101:rack-3:d4s:0
|
||||||
|
image: nubisware/autodynomite:latest
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
||||||
|
logging:
|
||||||
|
driver: "journald"
|
||||||
|
deploy:
|
||||||
|
mode: replicated
|
||||||
|
replicas: 1
|
||||||
|
endpoint_mode: dnsrr
|
||||||
|
placement:
|
||||||
|
constraints: [node.role == worker]
|
||||||
|
restart_policy:
|
||||||
|
condition: on-failure
|
||||||
|
delay: 5s
|
||||||
|
max_attempts: 3
|
||||||
|
window: 120s
|
||||||
|
configs:
|
||||||
|
- source: seeds.list
|
||||||
|
target: /dynomite/seeds.list
|
||||||
|
|
||||||
|
dynomite4:
|
||||||
|
environment:
|
||||||
|
- DYNO_NODE=dynomite4:8101:rack-2:d4s:2147483647
|
||||||
|
image: nubisware/autodynomite:latest
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
||||||
|
logging:
|
||||||
|
driver: "journald"
|
||||||
|
deploy:
|
||||||
|
mode: replicated
|
||||||
|
replicas: 1
|
||||||
|
endpoint_mode: dnsrr
|
||||||
|
placement:
|
||||||
|
constraints: [node.role == worker]
|
||||||
|
restart_policy:
|
||||||
|
condition: on-failure
|
||||||
|
delay: 5s
|
||||||
|
max_attempts: 3
|
||||||
|
window: 120s
|
||||||
|
configs:
|
||||||
|
- source: seeds.list
|
||||||
|
target: /dynomite/seeds.list
|
||||||
|
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
||||||
|
|
||||||
|
configs:
|
||||||
|
seeds.list:
|
||||||
|
file: ./seeds.list
|
|
@ -0,0 +1,31 @@
|
||||||
|
version: '3.6'
|
||||||
|
|
||||||
|
services:
|
||||||
|
|
||||||
|
elasticsearch:
|
||||||
|
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.8
|
||||||
|
environment:
|
||||||
|
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
|
||||||
|
- transport.host=0.0.0.0
|
||||||
|
- discovery.type=single-node
|
||||||
|
- xpack.security.enabled=false
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
||||||
|
aliases:
|
||||||
|
- es
|
||||||
|
logging:
|
||||||
|
driver: "journald"
|
||||||
|
deploy:
|
||||||
|
mode: replicated
|
||||||
|
replicas: 1
|
||||||
|
#endpoint_mode: dnsrr
|
||||||
|
placement:
|
||||||
|
constraints: [node.role == worker]
|
||||||
|
restart_policy:
|
||||||
|
condition: on-failure
|
||||||
|
delay: 5s
|
||||||
|
max_attempts: 3
|
||||||
|
window: 120s
|
||||||
|
|
||||||
|
networks:
|
||||||
|
conductor-network:
|
|
@ -0,0 +1,4 @@
|
||||||
|
dynomite1:8101:rack-1:d4s:0
|
||||||
|
dynomite2:8101:rack-1:d4s:2147483647
|
||||||
|
dynomite3:8101:rack-2:d4s:0
|
||||||
|
dynomite4:8101:rack-2:d4s:2147483647
|
Loading…
Reference in New Issue