Meeting 1 - 2020-07-22

Participants

  • Britto Augustine, Arizona DOE
  • Stephen Fuqua, Ed-Fi Alliance
  • Jean-Francois Guertin, EdWire
  • Eric Jansson, Ed-Fi Alliance
  • Erik Joranlie, EdAnalytics
  • Vinaya Mayya, Ed-Fi Alliance
  • Jim McKay, Certica
  • Laura Michaels, Palm Beach County School District
  • Patrick Yoho, InnovateEDU

Agenda

  1. Introductions (5 mins)
  2. Review origins and goals of SIG (5 mins)
  3. Discussion on previously-identified concerns and questions (40 minutes):
    1. Is there any call for using Windows containers in addition to (but not instead of) Linux containers?
    2. Corollary: is there demand for Windows-supported container setup for ODS/API for recent previously-released 3.x code?
    3. Do we need to build in more fault tolerance / circuit breakers to handle the more ephemeral nature of container uptime?
    4. Should the standard architecture include an external caching service that can be shared across API container instances?
    5. Should the standard architecture include tools for aggregating and visualizing log data, including custom applications, HTTP, and database?
  4. Brief description of planned community use cases (20 minutes).
  5. Next steps (5 minutes).

Meetings Notes

Any missing containers in proposed architecture? Is this a good starting point?

  • Consider a blue-green deployment, staging environment that is simpler. → Patrick Yoho particularly concerned.
  • Maybe just a single environment for all Ed-Fi platform tools? Start with an MVP?
  • Putting together a HELM chart? Maybe that comes later, as likely not needed for an MVP.
  • Brief mention of Apache Camel for message-oriented facade in front of the API, no detailed conversation.
  • Conclusions
    • Go simplest route with Docker Compose first, then look at Kubernetes.
    • Make a list of use cases and then prioritize these - that will define an MVP.

Operating system?

  • Proposal: to go Linux first.
    • Look into musl-based distributions - these have performance advantages https://www.musl-libc.org/.
    • Could use Alpine-based distribution
  • Any demand for Windows-based?
    • Unclear, but probably low - look to community to validate this over time.
    • Might be useful for ODS/API 3.3, 3.4, and 5.0.0, which will not work on Linux.
  • Conclusions:
    • Start with Linux.
    • Will consider Windows down the road as there is demand.

Fault tolerance?

  • Can be costly.
  • Likely won't go into production use initially - may start with ancillary use cases (e.g.vendor integration)
  • Mostly this is handed off to coordination tools.
  • Conclusion: not needed initially / low investment

External caching service?

  • Current ODS architecture not suitable to a shared cache architecture.
  • Current cache not heavily used (e.g. descriptors) and would migrate to containers OK.
  • Counterpoint: Student IDs are in that cache, so it can get large for large implementations.
  • As this cache is in memory, it could get duplicated many times over.
  • Current caching implementation does not sync well with updates in the database.
    • But a shared / distributed cache should solve that, since all API instances would update the shared cache when updating the database.
  • Conclusions:
    • There is some need, as larger implementations will see issues and/or need workarounds (e.g. SEA).
    • Possible action: estimate externalizing the cache(s) in the ODS/API.

Logging?

  • Is this a service needed, or implementation-specific?
  • Conclusion: No - logging is likely implementation specific
    Focus on ease of injecting the configurations the implementation wants

Use Cases?

  • Identified scenarios
    • Single-tenant with shared instance.
    • Single-tenant with year-specific instances.
    • Multi-tenant via new district-specific instances.
    • State-based shared instance
    • State-based year-specific instance
    • Extension support
  • With the year-specific  and district-specific settings, prefer to have one set of running containers for the installation, rather than a set of separate containers for every district (think about a SaaS provider).

Is Admin App container needed?

  • Conclusion: helpful, but not required, platform containers useful without Admin App.

Parking Lot

  • Comment: I think one of the most important things for docker-compose or helm solutions is balancing configuration vs simpleness and making sure that documentation is really strong. +1
  • Easily switch between internal or external PostgreSQL.
    • Probably multiple sample docker-compose files.
  • Does anyone need XML support, thus need/want a client-side bulk loader image?

Next Steps

  • Update the technical vision with more use case information and map out iterative MVP's for 2020 and beyond. Share with SIG members.
  • Further requirements definition will be handled via ad hoc communication, but we will reconvene the SIG if the Alliance or community members see value