Three-Year Review Process

Frequently Asked Questions: Approved Online Learning Provider Three-Year Review Process

Why must Approved Online Learning Providers participate in a Three-Year Review Process?

This process is based on Minnesota Statutes, section 124D.095: Online Learning Option Act (Subdivision 7), which charges the Minnesota Department of Education with oversight and implementation of a rigorous standards-based process for initial approval as well as ongoing approval of each public online learning provider. In the statute, “The commissioner, using research-based standards of quality for online learning programs, must review all approved online learning providers on a cyclical, three-year basis. Approved online learning providers annually must submit program data to, confirm statements of assurances for, and provide program updates including a current course list to the commissioner.”

View the Approved Online Learning Provider Three-Year Review Process.

What are the initial benefits to the Approved Online Providers who undergo the self-study and develop the digital portfolios? What are the benefits to MDE in leading and reviewing the standards-based, reflective self-study?

• Integration of the new Three-Year, Self-Study process into the practice of assurances and updates.
• Transparent, objective and standardized means for reviewing and renewing a provider that is both iterative and longitudinal in nature.
• Increased integrity, trust, and shared knowledge through a renewal of assurances that is based on standards, continuous- improvement focused, and quality-oriented.
• Increased knowledge of and accountability for quality assurance of approved online providers in Minnesota.
• Quality state-approved programs and courses available to the public, Online Learning peers, and Minnesota stakeholders.

What logic model will the Three-Year Review Process follow? What are the intended outcomes?

“Inputs” in this Three-Year Review Process include elements foundational to the development of a reflective self-study process. These elements include documents, workshop materials, and presentations all based on the Guiding Principles for the Self-Study Process An additional component of this process is a digital portfolio embedded with the iNACOL National Standards for Quality Online Programs.

Outputs in the first year of this Three-Year Review Process are provider self-reflective studies that culminate in a standards-based digital portfolio. The digital portfolio is the provider to demonstrate alignment with research-based standards (iNACOL) and compliance with Minnesota Statutes, section 124D.095: Online Learning Option Act. The end result of the process is an exhibit of reflection, supporting data, documents, and other artifacts that demonstrate how the provider meets or exceeds the standards and complies with department standards and statute language. This process builds on the annual updates and assurance requirements already in place.

Initial Outcomes in the Three-Year Review Process include standards-based analysis of the data, documents, and other artifacts the provider selects to demonstrate and/or measure against the results of their organization’s involvement in delivering online learning to students in Minnesota. Use of iNACOL National Standards for Quality Online Programs provides an objective and standardized way to report out and review results of the study. Building on the annual reporting process, the three-year review gives providers and the department a longitudinal look at approved online learning providers as they evaluate and measure themselves against standards at more than one point in time.

View the Logic Model of the Three-Year Review process.

Why was eFolio selected as the technology for providers to use in this process?

The rationale for selecting eFolio to use in the Three-Year Review process is as follows:

• Standardizes the MDE Three-Year Review Process through a common technology that establishes clear standards connected to the provider renewal of approval process.
• Allows approved providers to develop an online system to document, manage, and display resources for program assurances and Three-Year Review purposes.
• Permits integration of common standards in the review and provides a vehicle to exhibit data and documents as a demonstration of alignment and compliance.
• Encourages a process in which the program can be showcased and accountability can be demonstrated, shared, and updated on a continual basis.
• Creates a provider-owned site so public, secure, and/or password protected information is managed by the provider.
• Facilitates provider input during the study and supports asynchronous review of self-study work by reviewers and potentially other stakeholders.
• Provides to each of the approved online providers and supported at no cost.
• Allows documentation of performance to external stakeholders as well as to support the accreditation process.

Can I use another web-based tool for this process instead of eFolio?

No. The department uses eFolio to provide an objective and standardized means for reviewing the quality of an online learning program and its related services. Approved online providers are required to produce a self-study using eFolio Technology to exhibit data, documents, and artifacts that demonstrate alignment with standards, and compliance with Minnesota Statutes, section 124D.095: Online Learning Option Act. However, approved online providers may integrate other technology and/or web tools into this eFolio site.

When is the finished eFolio due to the department?

eFolio sites will be open for review on August 1 of each self-study year. Once the eFolio site is open for review, it then should be easily accessible for the reviewers. For this reason, self-checks along the way are good and you may want your process to include a review by others in your organization and/or trusted peers. The self-study is a showcase and should be developed accordingly. Approved providers are encouraged, but not required, to create their own support network of peers and colleagues during the self-study process.

How is the process connected to the annual assurances and program data reporting that are due each July?

Approved Online Providers involved in Self-Study during FY2012-2013 will integrate 2012 Annual Assurances and annual program data in their self-study site. In successive years, the annual program data, but not the additional assurances, will be required on July 15 of the third year of a provider’s Three-Year Review.

What kinds of support can I expect in this process?

Technology and Self-Study Support Available:

A. MDE will provide Wednesday and& Friday webinars on eFolio technology and the self-study process. View the webinar dates and topics to support the Three-Year Review Process.
B. View the eFolio Training website that will be used to support users. This site will be embedded into the demo site. Visit the Demo Three-Year Review Clone Site.
C. Information on the eFolio Help Desk and FAQ site links will be provided to all providers via the public calendar and embedded into the MDE Clone Demo site.

Are there eFolio samples of Three-Year Reviews I can look at?

There is no eFolio Self-Study that has been completed in Minnesota by an approved online provider yet. Currently, 12 of the 30 approved providers are working on the first eFolio sites for the new Three-Year Review Process. These are due on August 1, 2013. However, there are models of eFolio used in Minnesota two-year colleges. A few eFolio sites are provided as samples:

Lake Superior College Art Department (a program review site),
Mesabi Community & Technical College (an accreditation review site),
St. Cloud Technical & Community College (an eSystems Portfolio).

Is the department going to rank providers in the Three-Year Review Process?

No. The Three-Year Review is built on a continuous quality improvement process model. Results of the review for re-approval of an Online Learning (OLL) Provider will be on the MDE public list of Approved Providers. This list will include the OLL’s, initial date of approval, provider self-study year date, and date re-approved. This list will not rank or rate providers.

What is the Continuous Quality Improvement Process (CQI)?

CQI is a strategic approach to meeting or exceeding stakeholder expectations. In this Three-Year Review process, the CQI is based on: 1) meeting or exceeding the needs of all stakeholders; 2) involving approved online learning providers to reflect on online educational processes; and, 3) analyzing good data to drive organizational decision making. The Three-Year Review process, longitudinal in nature, builds on the annual updates required for Approved Online Providers and provides additional assurance of compliance with existing state and federal education statutes.

Who will review the eFolios submitted by the approved providers in this process?

The members of the review team have yet to be determined, but all reviewers will have an appropriate educational background that includes expertise in online learning, knowledge of research-based standards for quality assurance in online teaching and learning, and the technology skills needed to review an electronic portfolio. MDE anticipates a need for up to three review teams made up of three to five reviewers in place by August 1, 2013. The process for recruiting and training reviewers is incorporated in the new Three-Year Review Process.

What rubric(s) will be used to guide the review team in its review of the Self-Study eFolio? How can I get a copy of the rubric(s) that will be used by the review team?

The review team will be provided with the MDE Three-Year Review Rubric. You may download a copy of the rubric on the Online Learning Providers page located on the MDE website. The rubric will be used to assess the Self-Study eFolios submitted for review in the following five areas:

• Overall Rating of Self-Study
• Institutional Standards
• Teaching and Learning Standards
• Support Standards
• Evaluation Standards

How will the results of the MDE assigned review be communicated?

If the provider has confirmed alignment with research-based standards (iNACOL) and compliance with Minnesota Statutes, section 124D.095: Online Learning Option Act, the review team will recommend one of the following approval options:

• Approve for an additional three years; provider is only required to submit annual reports/data/assurances to MDE.
• Approve for an additional three years; however, approval is contingent on submission of annual reports and data to MDE to address specific review concerns.

What must an Approved Provider do if they receive an approval that is “contingent on submission of annual reports and data to MDE “to address specific review concerns?”

Submission of annual reports and data to MDE that address the review team concerns is required. However, the department will appraise all concerns presented by the assigned review team. In this appraisal of the concerns under Subdivision 7, (f) , (1) and (2), it may determine a need for further review and/or a need to create a compliance plan for the provider that could include withholding funds under Minnesota Statutes, sections 124D.095, 124D.10, subdivision 8, and 127A.42. In the unlikely event that funds are withheld, the department is required to notify an online learning provider in writing about withholding funds and provide detailed calculations.