Ed-Fi Working Draft 2: Learning App Tool Usage
Technical Suite: Suite 3
Status: Active


By: Ed-Fi Assessment Work Group
Contact: eric.jansson@ed-fi.org
Publication Date: June 22, 2020


Synopsis


These materials propose a simple design for capturing and exchanging data on learning app usage. The designs here focus on simplified metrics around learning app usage by individuals such as times, durations, basic indicators of activity or content area, and learning standards alignment. This design is not focused on detailed telemetry or "event stream" capture either in terms of the granularity of interactions or the periodicity of the capture.

Contents

General Discussion


The number of options for software tools to deliver and assess instruction as well as to provide interventions has grown exponentially, but within this growth teachers and district administrators do not have access to basic data on the usage of these tools. A set of basic data on learning tool usage would assist in making decisions regarding redeploying, increasing, sustaining, or transitioning from these tools within school district programs. 

There is also a growing policy movement to require administrators to use school improvement funding for programs that meet evidence standards, such as that released under the 2015 authorization of ESSA. As a result of these changes, instructional programs are increasingly conducting and providing research-based evidence as a matter of general practice. 

The designs here focus on capture transfer of what might be termed "granular aggregates."

  • They are "aggregates" in the sense that they are not focused on event stream type transmissions with the objective of itemizing in detail all the activity the student performed in the learning app (e.g. "logged on", "watched video, "responded to question", etc.).
  • However, the entities are "granular" in the sense that they can be scoped to the lowest level – the most granular level – needed for the primary use cases of consumers. 

As such, the design uses simplified concepts such as "TimeOnSystem" and "LearningAppActivity" that represent more coarse-grained elements in the student's interaction.

Use Cases


  • Use Case 1. As a school district administrator or principal, I want to be able to assess the amount of time various learning tools are used and the learning standards addressed or assessed during those interactions. I can use this information to assess if various tools are being employed, and will have more data points to inform analysis of whether such tools are effective. The data is only required on a daily basis (i.e., it could be supplied nightly and does not need to be streamed in real-time) and trackable at a classroom level so that I can assess the impacts of programs and usage across classrooms that use such programs or interventions in different amounts.
  • Use Case 2. As a teacher using multiple sources of online content, I want to be able to assess the content elements that my students are engaging with as well as how these contribute to various learning objectives for my state and district. I can use this information to discuss and recommend additional content to students, as well as to help my organization understand better how to adopt and expand usage of strong online content. 

Working Draft Details


The proposed data model is shown below. There are three main entities:

  • LearningApp represents a learning tool or application.
  • LearningAppUsage represents a period of application usage. This entity could be employed to capture various time periods, and the concept is that the periodicity of this element can be set to match the use case.  
    • For example: in some cases, it might make sense for the learning app usage to be reported daily, but in others it might make sense to report weekly or monthly, for less used tools or to allow the transmission to be less of a burden for the providers or agencies who receive the data.
  • LearningAppSession captures a students engagement with an app in a single session - a consistent but short term period of usage that probably corresponds to usage in a classroom or usage for a particular day's homework. 

Proposed key fields are shown in italics. Student would be part of the key on LearningAppUsage, and LearningAppSession would inherit all key fields of LearningAppUsage.

Figure 1: data model diagram (click to expand).

API Binding

It is assumed that each of these entities would be bound to a REST-ful API providing CRUD operations. Authorization could be driven by either the provider namespace (the namespace on LearningApp) or by access to the students records (via the Student reference on the supplied entities).