Buzz: Questions & Answers

Hi Shelly,

Thanks for the detailed description.

Buzz picks the set of questions for an assessment attempt when the student clicks Start, not when they first open the activity. Opening the activity, even many times, doesn’t create an attempt or lock in any version of the questions.

We checked this on the current release with “Allow save and continue” turned off, like your assessment. As a student, we opened an assessment several times without clicking Start. Then we edited one of its questions. When the student went back and clicked Start, they got the updated question.

So a student who only opened the activity on September 17 should have gotten the September 22 version when they started on October 15. Without more details to investigate, I am not sure what did occur in the particular student report. But after my own testing, I confirmed Buzz is working correctly for the scenario you provided. 

Assuming the student did select START and was simply reviewing questions and backing out before submitting-  if there is a pool of questions, will they get the same questions each time they view? 

We tested this as well, using an assessment that draws its questions from a pool in another activity, with “Allow save and continue” turned off.

• With “Allow save and continue” disabled in the activity settings, a student can’t look at the questions and back out without submitting. Once a student clicks Start, leaving the assessment submits the attempt. Buzz warns: “If you leave, your answers will be submitted.” Closing the page or going somewhere else submits it too.
  
  
• Each new attempt draws a new set from the pool. If retakes are allowed, every Retake picks a new random set of questions, so the student won’t necessarily see the same questions each time.
  
  
• New attempts use the current version of the questions. After we updated the questions in the source activity in our testing, the test student’s next attempt showed the updated questions.

Hi Brian,

Thanks so much.

Our team conducted a few tests as well. Please see these results.

Background:

• Seven controlled tests in Buzz to investigate how single-attempt assessments behave when a student closes the browser without submitting.
• Every assessment was configured to allow exactly one attempt. We varied the Allow save and continue setting, question pooling, and password protection to determine whether these settings affected the student's ability to resume an interrupted attempt.
• For each test, proxied in as a student, started the assessment, reviewed the questions, selected answers for questions, closed the browser window without submitting, and returned to the assessment.
• All pooled assessments were configured to select seven questions, with a minimum of one question per learning objective.

Test Configurations and Results: Using Microsoft Edge -Version 155.0.4283.62 (Official build) (64-bit)

RESULTS (we have video of the tests, if needed)

1. 21 questions, no bank | Save & Continue: OFF | Password: NO
   Result: Continued; answers retained.

2. 21 questions, no bank | Save & Continue: ON | Password: NO
   Result: Continued; answers retained.

3. 7 pooled questions | Save & Continue: ON | Password: NO
   Result: Continued; same questions and saved answers.

4. 7 pooled questions | Save & Continue: OFF | Password: NO
   Result: Continued; same questions and saved answers.

5. 7 pooled questions | Save & Continue: OFF | Password: YES
   Result: Continued; same questions, answers cleared.

6. 7 pooled questions | Save & Continue: ON | Password: NO
   Result: Continued; same questions and saved answers.

7. 7 pooled questions | Save & Continue: ON | Password: YES
   Result: Continued; same questions and saved answers.

• Regardless of whether Allow save and continue was enabled or disabled, closing the browser did not prevent the student from returning to the assessment.
• In every test, Buzz displayed Continue rather than treating the assessment as a completed attempt.

• When the student returned, the same questions appeared in the same order. This included assessments that drew seven questions from a pool.

• The only exception was Test 5, where Allow save and continue was disabled and password protection was enabled. The questions remained the same, but previously selected answers were cleared.
• In both password-protected tests, the password was required again when the student returned.

• The Score/Activity history screens recorded the student's initial visit and subsequent return.
• I also checked Submission history for Tests 4 and 7. Both displayed "No history to display."
• Test 4 is especially relevant because it used pooled questions with Allow save and continue disabled. Despite this configuration, the student resumed the assessment with the original questions and answers, and no submission was recorded.

1. Is a single-attempt assessment expected to remain accessible through Continue after the browser closes, even when Allow save and continue is disabled?
2. Should closing the browser automatically submit the attempt, or is that behavior limited to navigating away within Buzz?
3. Why are answers retained when saving is disabled in Tests 1 and 4?
4. Why does the combination of password protection and disabled saving clear answers in Test 5, while enabling saving preserves them in Test 7?
5. Are there additional settings or conditions that could explain the differences between your testing and ours?