ADP — Instructor Insights

Questions this session

—

Avg. answer confidence

—

Frustrated sentiment

—

Deferred to instructor

—

Attendance · Teams

—

Avg attention · Teams

—

Sentiment over session

a trend across the session, not a snapshot

Positive Frustrated start → now

Concept struggle heatmap

taught concepts ranked by repeat questions + low confidence

Microsoft Teams signals illustrative — not connected

Attention across the session (engagement vs. comprehension)

Illustrative — signals reflect the proposed analytics design. Aggregated at the class level; never individual evaluation.

CT101 “Time Fundamentals” — feedback aggregated across self-serve (standalone) and instructor-led (ILT) queries. Class/cohort level only.

Total queries

—

Self-serve

—

In ILT

—

Deflection rate

—

Avg confidence

—

Frustration

—

What we can derive from self-serve (standalone) queries

Topic demand & question volume per concept
Where self-serve learners struggle (repeat questions + low confidence)
Content gaps vs teaching gaps, per lesson
Sentiment & frustration trends over days/weeks
Deflection rate — % resolved without a human

What we can derive from ILT (in-training) queries

Live class pulse, sentiment dip timing, recap nudges
Deferred (“not yet covered”) topics learners want early
Concept struggle during the session → on-the-spot recap
Cohort-over-cohort comparison for the same module
(+ Microsoft Teams: attendance, attention, engagement)

By lesson — demand, sentiment & gaps

self-serve vs ILT split, frustration, confidence, and the gap signal

Lesson	Questions	Self-serve / ILT	Frustration	Avg conf	Deferred	Signal

Top struggle concepts (whole module)

ranked by repeat questions + frustration + low confidence

Module trend over time

weekly question volume & frustration

VolumeFrustration %

Cohort comparison (longitudinal)

the same module across cohorts — spot perennially hard concepts vs one-offs

Cohort	Queries	Frustration	Avg conf	Deflection	Concepts

Illustrative module feedback — powered by the same query stream as the tutor. Class/cohort level only.

—

Repeat rate (still-stuck proxy)

—

Resolved first try

—

Distinct concepts asked

—

Low-confidence / no-answer

Quiz mastery by concept

scores + most common wrong answer (weakest first)

Concept	Mastery	Attempts	Common wrong answer

Repeated concepts (still stuck)

asked again and again — recap candidates

Low-confidence / no-answer log ⬇ Export CSV

verbatim queries where the tutor was weak or had no answer — the content-gap goldmine for the content team

Query	Lesson	Concept	Conf	Decision

Content utilization

cited often (promote) vs never cited (prune / teach harder)

Hold-topic demand (in-training)

what learners ask about before it's taught — a pacing signal

Decision-useful analytics across standalone + ILT. Class/cohort level only.

LLM grounding barometer

how well the tutor stayed grounded — and correctly refused — instead of hallucinating

—

Avg. answer confidence

—

Grounded answers

—

Low-confidence (review)

—

Correctly held

Recent answers

coverage decision · grounding confidence · cited course content?

Question	Decision	Confidence	Grounded