Thursday, April 12, 2018

9:30 AM - 11:30 AM

Assessment Working Group
Las Palmas B
The working group will discuss key topics in the current assessment exchange model and specifications, and propose changes where necessary to meet community needs. Technical topics (among others) will include: enumerations, identity, and the taxonomy of provider types in the current certification.

These broad session notes attempt to capture the spirit of the discussion and should not be interpreted as a transcript. Although Ed-Fi Alliance staff were involved in capturing these observations, the notes below should not be construed as official, complete, or 100% accurate.


Notes to accompany the slides


  • Governance process has been working well in several areas: participatory, seeks out expertise, how to make decisions and execute upon them for addressing needs of the community.

  • BUT, the scale and growth of the movement and community caused the consideration to review and research governance models that will build the structure to allow Ed-Fi to grow larger more successfully while maintaining community involvement and voice balanced with fidelity and sustainability.

    • Ensure alignment of Ed-Fi direction with Ed-Fi community needs.

  • Governance is multi-tiered, and will be a 2-way process of review/clarification/approval, etc.

Questions/discussion


  • As the governance process evolves, what about the license? Is it a topic for governance to consider changes to licensing to move toward more open source and correlated impact/changes to IP?

    • Yes, the governance structure is exactly why and how this would be discussed / considered / worked through

  • How do we expect vendors to commit and work toward these standards/certifications before workgroups are spun up to inform those efforts?

    • Refer to the TAO of the Ed-Fi Alliance. We start and begin work to effect change and participation and involvement and then fine tune, iterate and improve the work in these emerging areas.

  • How do we protect and balance the demand side (agencies/customers) with the supply side (vendors/providers) such that as we seek sustainable directions they aren’t co-opted by powerful supply side interests?

    • We will prioritize raising awareness on the demand side along with commitment to participate, pull in domain expertise to flesh out use cases for agency IT groups, and provide a demand side balance with the supply side perspectives.

  • How do we get involved?

    • See Nancy Wilson to join the sign on sheet today. All attendees today will get an email next week about next steps. Can also contact anyone on the Ed-Fi team by email, etc.

  • Priority Use Cases and examples from attendees

    • Teacher developed assessments and item level data with teacher authored items, need to be able to do stats analysis, scale scores, and return that back post analysis for learning/instructional intervention

      • Which answer chosen, not just right/wrong

      • TEI items

      • Cross reference standards with questions, like a blueprint

    • Type of assessment, mapped to multiple standards (CC, but also SEL, behavior frame, etc.) and mapped to multiple scores, and variable scores

    • Looking at multiple grade bands. A standard that spans multiple grades.

    • Competency frameworks. See Chris sturgis & Susan Patrick  “competency works” at iNacol



    • Item types; constructed response versus selected response

    • Learning standards, broken down to learning targets

    • CASE specs. How to we support or interact/intersect?

    • Gamification data; usage/utilization type data

    • Competencies vs standards

    • Use of student assessment data for teacher effectiveness assessment and program evaluation

      • This is an area to be very careful with. It’s a lightning rod issue politically that could ignite a huge pushback from unions, states, etc.

    • Self assessment and SEL

    • Performance based assessments

  • Question around how flexible the tech stack is to support the various types of items/data needed by these assessment considerations

    • Yes, flexible. Can describe these things in metadata, point to actual URI of the content (eg. like portfolio items, etc) BUT, fleshing out these requirements could really help to justify those tech stack needs.

  • Eric Jansson will be point of contact for this work. eric.jansson@ed-fi.org

Wrap up


Open questions and areas of development for workgroup consideration


  1. Identifiers for learning standards

  2. Diversity of ecosystem metadata

    1. Who owns controlled vocabularies?

  3. Key values

  4. One API or Two?

  5. Relationships to “gradebook” data

Summary notes


  1. General positive reaction to governance/workgroup model.

    1. Lack of participatory model can be a block to adoption.

  2. Concern re maintaining active agency input for specifications, both due to inequities in finances but also due to difficulty for agencies to finance participation.

  3. Developed an initial list of use cases, and will likely need to prioritize these as there was considerable diversity.

  4. A number of domain data model issues, including:

    1. Representation of grade bands

    2. Applicability to other kinds of assessments, like SEL assessments

    3. Connecting back to item-level data and responses, including for technology-enhanced items