UNCLASSIFIED

Skip to content

GitLab

Open
Created by charles.heaton.2@charles.heaton.2Owner0 of 2 tasks completed0/2 tasks

Process: How to handle and analyze Survey results:8

Description

After we get back information from a Training survey's (https://gitlab.com/90cos/public/training-roadmaps/-/issues/14) how do we capture discrepancies on that information when there is a potential conflict with the provider?

Scenario:

  1. A training source intended to teach material at a particular level and indicate this in their "Mapped" level
  2. We survey the course #47 to measure the level it's actually training at
  3. The survey results disagree with the mapping (intended)
  4. How do we detect (or document) this discrepancy? (Purpose of this ticket

Potential solutions:

  • Option B: We control the mapped level (<-- this is less data).

Considerations

Determining true level/mapping with multiple data points

Point in time vs trend: How do we measure for a particular instance of the training? Individual vs group:

How do we (90COS) determine the aggregated mapped level especially as time and instance will matter. A course could be given 4 times. the first two times it was level A, then once at level B, then latest at level C... what do we say the level is?

Provider's perspective

Training providers will likely see this as accusatory, damaging, or confrontational so will state/ask some of the below. We need to be prepared for this eventuality with either answers, or better, with transparent information to help create a feedback loop they appreciate (not fight).

  1. They (90COS) are saying I'm not doing my job (not training what I'm claiming)!
  2. How do I correct this claim that I'm not training what I'm advertising?
  3. Where are they getting their information from?

Potential solution:

  • Analysis will always be human driven but we should develop the report(s)/dashboards used to enable this analysis.
  • Aggregating and exposing our feedback and survey data can help enable providers to improve their products.

Test Plan

Option 1: Fake data and table topping

  1. Mock out two clusters of surveys to simulate two courses given at different times
  2. Mock out surveys which present information

Option 2: Just do it!

  1. Start giving the surveys and test on the fly.

Acceptance Criteria

  • AC 1 - Prototype a report
  • AC 2 - pick a test strategy
Edited by charles.heaton.2

UNCLASSIFIED