Commit 393c0822 authored by Daniele Venzano's avatar Daniele Venzano Committed by GitHub

Merge pull request #53 from DistributedSystemsGroup/documentation-fixes

Expand and update install documentation
parents a78ba80e ad7fa56d
......@@ -18,7 +18,7 @@ Common options:
* ``debug = <true|false>`` : enable or disable debug log output
* ``api-listen-uri = tcp://*:4850`` : ZeroMQ server connection string, used for the master listening endpoint
* ``deployment-name = devel`` : name of this Zoe deployment. Can be used to have multiple Zoe deployments using the same Swarm (devel and prod, for example)
* ``workspace-deployment-path`` : path appended to the workspace path to distinguish this deployment. If unspecified is equal to the deployment name
* ``workspace-deployment-path`` : path appended to the ``workspace-base-path`` to distinguish this deployment. If left unspecified it is equal to the deployment name
* ``influxdb-dbname = zoe`` : Name of the InfluxDB database to use for storing metrics
* ``influxdb-url = http://localhost:8086`` : URL of the InfluxDB service (ex. )
* ``influxdb-enable = False`` : Enable metric output toward influxDB
......
......@@ -9,12 +9,10 @@ Multiple deployment options are available:
* Deployment scripts
* Manual install
Please refer to `zoe-deploy <https://github.com/DistributedSystemsGroup/zoe-deploy>`_ repository for deployment scripts.
The section below describes how to install Zoe manually.
Manual install
--------------
Overview
--------
Zoe components:
......@@ -22,31 +20,96 @@ Zoe components:
* API
* command-line client
Zoe is written in Python and uses the ``requirements.txt`` file to list the package dependencies needed for all components of Zoe. Not all of them are needed in all cases, for example you need the ``kazoo`` library only if you use Zookeeper to manage Swarm high availability.
Optional components:
Optionally an API manager can be configured in front of the Zoe API to provide SSL termination and usage statistics:
* :ref:`api-manager-label`
Overview
--------
Most of the ZApps expose a number of interfaces (web, REST and others) to the user. Zoe configures the active back-end to expose these ports, but does not perform any additional action to configure routing or DNS to make the ports accessible. Keeping in mind that the back-end network configuration is outside Zoe's competence area, here there a non-exhaustive list of the possible configurations:
* expose the hosts running the containers by using public IP addresses
* use a proxy, like the one developed for Zoe: :ref:`proxy`
* use back-end network plugins to build custom topologies
Stand-alone environment for development and testing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A simple deployment for development and testing is possible with just:
* A Docker Engine
* Zoe
Please note: since Zoe will use the Swarm API to talk to the Docker Engine, the addresses of the exposed ports for running ZApp may be ``0.0.0.0``, causing the links generated by the web interface and the command line tool to be wrong. ZApps can be accessed by using ``127.0.0.1`` instead.
Example of distributed environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For running heavier workloads and distributed applications, you need a real container cluster. In this example we will use Docker Swarm, as it is simpler to setup than Kubernetes.
Software:
* Docker Swarm
* Optional: ZooKeeper for Docker Engine discovery by Swarm (etcd and consul are also supported by Swarm)
* Zoe
* NFS (or another distributed filesystem like CephFS)
Swarm supports several discovery mechanisms and you should select the most appropriate for your environment. An explicit host list is the most simple to setup, while ZooKeeper provides a resilient and dynamic way of attaching Docker Engines to Swarm.
Topology:
* On node running Zoe, a Docker Engine and the following containers:
ZApps, usually, expose a number of interfaces (web, REST and others) to the user. Docker Swarm does not provide an easy way to manage this situation: the port can be statically allocated, but the IP address is chosen arbitrarily by Swarm and there is no discovery mechanism (DNS) exposed to the outside of Swarm.
* Swarm manager
* Docker registry (Optional, but strongly suggested)
* Optional `gateway containers <https://github.com/DistributedSystemsGroup/gateway-containers>`_ for SSH and/or SOCKS proxies
In the interest of keeping dependencies few and easy to manage, we do not rely on external plugins for networking of volumes.
With the functionality that is built-in into Docker and Swarm there is no good, automated, way to solve the problem of accessing services running inside an overlay network from outside. We decided to leave the network configuration entirely in the hands of who is in charge of doing the deployment: Zoe expects a Docker network name and will connect all containers on that network. How that network is configured is outside Zoe's competence area.
* Three nodes for ZooKeeper: if used only for Swarm the load will be very low and ZooKeeper can be co-located with Zoe and the Docker Engines
* At least one more node with a Docker Engine
* A file server running NFS: depending on the workload it can be co-located with Zoe and the Swarm manager
* Another node running a Swarm manager, if you want to enable HA for Swarm
As an example of a simple, robust configuration, we use a standard Swarm configuration, with private and closed overlay networks. We create one overlay network for use by Zoe and spawn two containers attached to it: one is a SOCKS proxy and the other is an SSH gateway. Thanks to LDAP users can use the SSH gateway to create tunnels and copy files from/to their workspace.
These gateway containers are maintained outside of Zoe, at this Github repository: https://github.com/DistributedSystemsGroup/gateway-containers
To configure Swarm and the Registry, please refer to the documentation available on the Docker website.
Zoe requires a shared filesystem, visible from all Docker hosts. Each user has a workspace directory visible from all its running ZApps. The workspace is used to save Jupyter notebooks, copy data from/to HDFS, provide binaries to MPI and Spark applications. Again, there are several plugins for Docker that offer a variety of volume backends: we have chosen the simplest deployment option, by using a shared filesystem mounted on all the hosts to provide workspaces.
To configure container networking, we suggest the standard Swarm overlay network.
Users need to put data and binaries in a place accessible by Zoe and need to be able to access the results and the logs generated by running ZApp.
Zoe uses the concept of workspaces: each user has a private directory that is attached to all the containers of each ZApp belonging to her in a well-known location. This filesystem can be accessed by a special gateway container spawned by the administrator (see `gateway containers <https://github.com/DistributedSystemsGroup/gateway-containers>`_) or by other methods (direct mount on user machines, webdav, web file managers).
Zoe expects the network filesystem to be mounted in the same location on all hosts registered in Swarm. This location is specified in the ``workspace-base-path`` Zoe configuration item. Zoe will create a directory under it named as ``deployment-name`` by default or ``workspace-deployment-path`` if specified. Under it a new directory will be created for each user accessing Zoe.
Refer to the manual install section below for more details and links to external resources.
Choosing the back-end
---------------------
At this time Zoe supports three back-ends:
* Docker engine: useful for running Zoe on a single PC, it is the simplest to setup.
* Docker Swarm: simple to install, additional features like SSL, high-availability and dynamic host discovery can be added as needed. Please note that Zoe does not support the new Swarm Mode of Docker Engine as the API is too limited.
* Kubernetes: the most complex to setup, we suggest using it only if you already have (or need) a Kubernetes setup for running other software.
Docker compose
--------------
In the root of the repository you can find a ``docker-compose.yml`` file that should help get you started.
Look also at the deployment scripts below, as they provide an option for a simple, zoe-on-a-laptop, install.
Deployment scripts
------------------
Refer to `zoe-deploy <https://github.com/DistributedSystemsGroup/zoe-deploy>`_ repository for automated deployment scripts for configurations with Swarm or Kubernetes back-ends.
Manual install
--------------
This section shows how to install the components outlined in the distributed environment outlined above. A lot of other options and possibilities exist for deploying Zoe.
Requirements
------------
^^^^^^^^^^^^
* Python 3. Development happens on Python 3.4, but we test also for Python 3.5 on Travis-CI.
* Docker Swarm (we have not yet tested the new distributed swarm-in-docker available in Docker 1.12)
* A shared filesystem, mounted on all hosts part of the Swarm. Internally we use CEPH-FS, but NFS is also a valid solution.
* Python 3. Development happens on Python 3.4 and 3.5.
* Docker Swarm
* A shared filesystem, mounted on all hosts part of the Swarm.
Optional:
......@@ -54,7 +117,7 @@ Optional:
* A logging pipeline able to receive GELF-formatted logs, or a Kafka broker
Swarm/Docker
------------
^^^^^^^^^^^^
Install Docker and the Swarm container:
......@@ -78,20 +141,22 @@ A few sample ZApps have their images available on the Docker Hub. We strongly su
Zoe
---
Zoe is written in Python and uses the ``requirements.txt`` file to list the package dependencies needed for all components of Zoe. Not all of them are needed in all cases, for example you need the ``kazoo`` library only if you use Zookeeper to manage Swarm high availability.
Currently this is the recommended procedure, once the initial Swarm setup has been done:
1. Clone the zoe repository
2. Install Python package dependencies: ``pip3 install -r requirements.txt``
3. Create new configuration files for the master and the api processes (:ref:`config_file`), you will need also access to a postgres database
4. Setup supervisor to manage Zoe processes: in the ``scripts/supervisor/`` directory you can find the configuration file for supervisor. You need to modify the paths to point to where you cloned Zoe and the user (Zoe does not need special privileges).
4. Setup supervisor to manage Zoe processes: in the ``contrib/supervisor/`` directory you can find the configuration file for supervisor. You need to modify the paths to point to where you cloned Zoe and the user (Zoe does not need special privileges).
5. Start running ZApps!
In case of troubles, check the logs for errors. Zoe basic functionality can be tested via the ``zoe.py stats`` command. It will query the ``zoe-api`` process, that in turn will query the ``zoe-master`` process.
.. _api-manager-label:
API Manager
-----------
API Managers
------------
To provide TLS termination, authentication, load balancing, metrics, and other services to the Zoe API, you can use an API manager in front of the Zoe API. For example:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment