Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Note
iconfalse
titleDocker Deployment v2.0

May 2021: On behalf of the Ed-Fi community, we are pleased to announce the release of Ed-Fi-ODS Docker Deployment 2.0 which includes support for deploying the latest Ed-Fi ODS / API v5.2 version and Admin App 2v2.2. In addition, this release delivers two major features:

  • Configures applications on different virtual paths to allow standard https port for all applications.
  • Adds external SQL Server support.

Overview 

The Ed-Fi ODS Docker deployment scripts install and configure components of the Ed-Fi ODS / API on Docker containers. The Docker Compose files provided automate the numerous configuration steps required to get the ODS / API up and running on Linux containers backed by PostgreSQL 11 or an external SQL Server. These scripts can be used for scenarios where a default ODS / API is needed without extensions or customizations. The following configurations are supported out of the box.  

Sandbox Configuration 


Deck of Cards
historyfalse
idsandox config


Card
idPostgreSQLSandboxConfiguration
labelPostgreSQL


Card
idSQLServerSandboxConfiguration
labelSQL Server


Shared Instance Configuration 

Deck of Cards
historyfalse
idshared config


Card
idPostgreSQLSharedConfiguration
labelPostgreSQL


Card
idSQLServerSharedConfiguration
labelSQL Server


Deploying to Docker Containers

Docker Containers have the added benefit of running anywhere (e.g., VMs, on premises, in the cloud), which is a huge advantage for both development and deployment. It has been adopted by leading cloud providers, including, Google Cloud, Amazon Web Services (AWS), and Microsoft Azure. For simplicity, the steps below describe how to install on ODS / API on Docker Desktop.

Table of Contents
maxLevel3
minLevel3

Step 1. Download the Source Code or Clone the Repo

The Ed-Fi ODS Docker deployment source code is contained in the Ed-Fi repository hosted by GitHub. A link to the repository is provided in the download panel on the right. You can either clone the repository or download the source code as a ZIP file. 

Step 2. Set up Docker Environment 

To work with the material in this repository, set up a Docker environment by referring to the article Set Up Your Docker Environment

Step 3. Setup Your Environment Variables

Configure your deployments using environment file. The repository includes .env.example, which lists the supported environment variables.  Copy .env.example file and name it .env. Update the values as desired.

Info
iconfalse

Note: Compose supports declaring default environment variables in an environment file named .env, placed in the folder where the docker-compose command is executed (current working directory). If you desire a different name or location, the following command can be used with docker-compose during deployment.

Code Block
languagebash
docker-compose --env-file .env.dev -f (docker-compose-filename) up



A sample .env setting for PostgreSQL backed deployment is provided below as an example and not be used as is in your deployment. 

Deck of Cards
historyfalse
idenv setting


Card
idPostgreSQLEnv
labelPostgreSQL


Code Block
ADMIN_USER=admin@test.com
ADMIN_PASSWORD=Admin#1
LOGS_FOLDER=c:/tmp/logs
MINIMAL_KEY=minimal
MINIMAL_SECRET=minimalSecret
POPULATED_KEY=populated
POPULATED_SECRET=populatedSecret
POSTGRES_USER=postgres
POSTGRES_PASSWORD=P@ssw0rd
PGBOUNCER_LISTEN_PORT=6432
ODS_VIRTUAL_NAME=api
SANDBOX_ADMIN_VIRTUAL_NAME=admin
TAG=v2

# The following are only needed for Admin App
ADMINAPP_VIRTUAL_NAME=adminapp
API_INTERNAL_URL=<http://{ods api container's hostname}>
API_EXTERNAL_URL=https://localhost/api
ENCRYPTION_KEY=<base64-encoded 256-bit key>



Card
idSQLServerEnv
labelSQL Server


Code Block
ADMIN_USER=admin@test.com
ADMIN_PASSWORD=Admin#1
LOGS_FOLDER=c:/tmp/logs
MINIMAL_KEY=minimal
MINIMAL_SECRET=minimalSecret
POPULATED_KEY=populated
POPULATED_SECRET=populatedSecret
SQLSERVER_DATASOURCE=<DNS or IP Address of the SQL Server Instance, i.e. sql.somedns.org or 10.1.5.9,1433
SQLSERVER_USER=<SQL Username with access to SQL Server Ed-Fi databases, edfiadmin>
SQLSERVER_PASSWORD=<SQL Password for the SQLSERVER_USER with access to SQL Server Ed-Fi databases, password123!>
ODS_VIRTUAL_NAME=api
SANDBOX_ADMIN_VIRTUAL_NAME=admin
TAG=v2

# The following are only needed for Admin App
ADMINAPP_VIRTUAL_NAME=adminapp
API_INTERNAL_URL=<http://{ods api container's hostname}>
API_EXTERNAL_URL=https://localhost/api
ENCRYPTION_KEY=<base64-encoded 256-bit key>



When including the Admin App in your deployment, such as with the Shared Instance Configuration:

  • API_INTERNAL_URL should be set to http://{ODS API container's hostname}
  • API_EXTERNAL_URL should be set to the ODS API gateway URL.
  • The AES encryption key for the ENCRYPTION_KEY environment variable must be set to a newly generated value for the deployment environment. Such a unique value can be generated using the following code snippet at a Linux command prompt or using Git Bash, if deploying on local Windows host.
Code Block
languagebash
titleGenerate key on Linux
openssl rand -base64 32

Step 4. Choose Your Deployment Options 

In the step you need to choose between deploying a Sandbox or Shared Instance environment.

Sandbox Environment

The Docker Compose file for Sandbox environment installs three primary Ed-Fi applications in separate containers behind NGINX proxy: 

  • Ed-Fi ODS / API
  • SwaggerUI
  • Sandbox Admin

EdFi_Admin and EdFi_Security databases are installed on one PostgreSQL 11 container. ODS sandboxes are installed on a separate PostgreSQL 11 container.

If you desire to deploy sandbox environment, navigate to the root directory of docker deployment scripts and run the following script: 

Deck of Cards
historyfalse
idsandbox compose up


Card
labelPostgreSQL


Code Block
languagebash
docker-compose -f ./compose/pgsql/compose-sandbox-env.yml --env-file .\.env up -d



Card
labelSQL Server


Code Block
languagebash
docker-compose -f ./compose/mssql/compose-sandbox-env.yml --env-file .\.env up -d



Shared Instance Environment

The Docker Compose for Shared Instance environment installs the following behind NGINX proxy.

  • Ed-Fi ODS / API
  • Admin App 

EdFi_Admin and EdFi_Security databases are installed on one PostgreSQL 11 container. EdFi_Ods database is installed on a separate PostgreSQL 11 container.

If you desire to deploy Shared Instance environment, navigate to the root directory of docker deployment scripts and run the following script:

Deck of Cards
historyfalse
idshared compose up


Card
labelPostgreSQL


Code Block
languagebash
docker-compose -f ./compose/pgsql/compose-shared-instance-env.yml --env-file .\.env up -d



Card
labelSQL Server


Code Block
languagebash
docker-compose -f ./compose/mssql/compose-shared-instance-env.yml --env-file .\.env up -d




Info

Alternatively, you can execute sandbox-env-up.ps1 or shared-instance-env-up.ps1 PowerShell scripts if deploying on Windows host. 

These compose files pull down the images from Docker Hub. The repository contains two additional compose files compose-sandbox-env-build.yml and compose-shared-instance-env-build.yml that work with the Dockerfiles directly for customizations.

Step 5. Provide SSL certificate

The deployments require a valid SSL certificate to function. A self-signed certificate can be used for a Non-Production environment. The repository includes generate-cert.sh script that can be used to generate a self-signed certificate and place it in ssl folder under root directory to be used by the running Gateway container.

If deploying on local Windows host, you will either need Git Bash or WSL to run generate-cert.sh.

Using Git Bash

  • Start a Git Bash Session
  • Run the following commands:
Code Block
languagebash
export MSYS_NO_PATHCONV=1
cd '{your repo root}'
./generate-cert.sh

Using WSL

  • Enable WSL and install a Linux from the Microsoft Store.
  • Start a WSL session.
  • Change directory to the Web -Gateway (Web-Gateway-Sandbox) folder under Ed-Fi-ODS-Docker repository folder.
  • Convert the script from CRLF to LF endings.
  • Run script generate-cert.sh (i.e., ./generate-cert.sh).

Step 6. Verify Your Deployments 

Open your Docker Desktop instance and view running container instances. The following image shows sandbox deployment: 

You can also verify deployed applications by browsing to: 

The following image shows shared instance deployment:

You can also verify deployed applications by browsing to: 

Step 7. Restarting the ODS/API

After the first time setup for Admin App, the user is prompted to restart the ODS/API. For Docker deployments, the ODS/API can be restarted by clicking on the restart button (see image) next to the ODS/API container on your Docker Desktop instance.

Step 8. Accessing Log Files 

The ODS/API, Admin App and SandboxAdmin write logs to a mounted folder within their docker containers. The environment file described in the Step 3 lets you configure log location for easy access. For example, you could set LOGS_FOLDER variable to c:/tmp/logs for windows hosts or to ~/tmp/logs for Linux and MacOS hosts. 

Tearing down Docker Deployments 

If deployed as a sandbox environment, navigate to the root directory of Docker deployment scripts, and run the following script: 

Deck of Cards
historyfalse
idsandbox compose down


Card
labelPostgreSQL


Code Block
languagebash
docker-compose -f ./compose/pgsql/compose-sandbox-env.yml --env-file .\.env down -v --remove-orphans



Card
labelSQL Server


Code Block
languagebash
docker-compose -f ./compose/mssql/compose-sandbox-env.yml --env-file .\.env down -v --remove-orphans



If deployed as a shared instance environment, navigate to the root directory of Docker deployment scripts and run the following script: 

Deck of Cards
historyfalse
idshared compose down


Card
labelPostgreSQL


Code Block
languagebash
docker-compose -f ./compose/pgsql/compose-shared-instance-env.yml --env-file .\.env down -v --remove-orphans



Card
labelSQL Server


Code Block
languagebash
docker-compose -f ./compose/mssql/compose-shared-instance-env.yml --env-file .\.env down -v --remove-orphans




Warning

Using -v option with the docker-compose will remove volumes declared in the compose file, which includes the database volumes. Do not use that option if you intend to keep the data in the ODS. 

Use rmi command to cleanup local copies of Images. 

Info

Alternatively, you can execute sandbox-env-clean.ps1 or shared-instance-env-clean.ps1 PowerShell scripts if you are using Windows host. 



Panel
borderColor#fec43d
bgColor#ffedc4
titleColor#000
borderWidth1
titleBGColor#fec43d
borderStylesolid
titleDownloads

The following links contain relevant source code and published images:

Repository: Ed-Fi-ODS-Docker
Images: https://hub.docker.com/u/edfialliance