Last modified: Mar 11, 2014
Recording
These slides accompany a recorded video: Play Video
Example of Early Analysis
This lesson works through an example of the early stages of analysis.
Mistakes will be made!
One of my pet peeves about reading how to do analysis and design in textbooks is that they always make the right decisions at each step.
It’s important to realize that designers do make mistakes and need to back up and reconsider things. (That’s why we have the “V” in the ADIV workflow!)
So I try to be honest and record my analysis examples as a stream-of-consciousness of what I actually went through considering the problem for the first time, including the mistakes.
ODU offers a number of courses via the internet. A common requirement among these courses is for a system of online assessment. An assessment is any form of graded question-and-answer activity. Examples include exams, quizzes, exercises, and self-assessments. In preparation for automating such a system, our group has undertaken a study of assessment techniques in traditional classrooms.
An assessment can contain a number of questions. Questions come in many forms, including true/false, single-choice from among multiple alternatives, multiple choices, fill-in-the-blank, and essay. There may be other forms as well.
Students take assessments that are administered by instructors. The students’ responses to each question are collected by the instructor, who grades them by comparison to a rubric for each question. The instructor may also elect to provide feedback (written comments), particularly about incorrect responses.
A total score for the assessment is computed by the instructor. If this is a self-assessment, the score is for informational purposes only. For other kinds of assessments, the instructor records the score in his/her grade book.
Information is returned to the student about their performance. At a minimum, the student would learn of their score and any instructor-provided feedback. Depending upon the instructor, students may also receive the questions, a copy of their own responses, and the instructor’s correct answer.
For the initial list, mark up the description, looking for noun phrases and verb phrases.
ODU offers a number of courses via the Internet. A common requirement among these courses is for a system of on-line assessment. An assessment is any form of graded question-and-answer activity. Examples include exams, quizzes, exercises, and self-assessments. In preparation for automating such a system, our group has undertaken a study of assessment techniques in traditional classrooms.
An assessment can contain a number of questions. Questions come in many forms, including true/false, single-choice from among multiple alternatives, multiple choices, fill-in-the-blank, and essay. There may be other forms as well.
Students take assessments that are administered by instructors. The students’ responses to each question are collected by the instructor, who grades them by comparison to a rubric for each question. The instructor may also elect to provide feedback (written comments), particularly about incorrect responses.
A total score for the assessment is computed by the instructor. If this is a self-assessment, the score is for informational purposes only. For other kinds of assessments, the instructor records the score in his/her grade book.
Information is returned to the student about their performance. At a minimum, the student would learn of their score and any instructor-provided feedback. Depending upon the instructor, students may also receive the questions, a copy of their own responses, and the instructor’s correct answer.
Start by drawing up CRC cards.
Assessment | |
---|---|
Exam | |
---|---|
Quiz | |
---|---|
Exercise | |
---|---|
Self-Assessment | |
---|---|
Question | |
---|---|
True/False Question | |
---|---|
Single-Choice Question | |
---|---|
Multiple Choices Question | |
---|---|
Fill-In-The-Blank Question | |
---|---|
Essay Question | |
---|---|
Student | |
---|---|
Instructor | |
---|---|
Response | |
---|---|
Rubric | |
---|---|
Feedback | |
---|---|
Score | |
---|---|
Grade Book | |
---|---|
Information | |
---|---|
Performance | |
---|---|
Correct Answer | |
---|---|
Since all of the various kinds of assessments are likely to have similar responsibilities and collaborators, let’s stack their cards for now and treat them as a unit.
We’ll do the same with the various kinds of questions.
Assessment | |
---|---|
Question | |
---|---|
Student | |
---|---|
Instructor | |
---|---|
Response | |
---|---|
Rubric | |
---|---|
Feedback | |
---|---|
Score | |
---|---|
Grade Book | |
---|---|
Information | |
---|---|
Performance | |
---|---|
Correct Answer | |
---|---|
Now fill in the operations known so far:
contain (questions)
take (assessment),
administer,
collect (responses),
grade,
provide (feedback),
compute (score),
record (score),
return (information),
An assessment can contain a number of questions.
This is really a statement about attributes of an assessment
Assessment | |
---|---|
Has questions | |
Students take assessments that are administered by instructors.
Is this really two separate operations?
The language (plurals) is a bit tricky.
Instructors administer an assessment to an entire class.
Each student individually takes the assessment.
Instructor | |
---|---|
administer assessment to group of Students | |
Taking or Administering?
Surprised that I put that in Instructor?
Remember the basic rule: if A does B to C, then “do B” is usually a responsibility of C
What’s involved in administering an assessment?
The problem statement tells us:
Students take assessments that are administered by instructors. The students’ responses to each question are collected by the instructor, who grades them … The instructor may also elect to provide feedback (written comments), particularly about incorrect responses.
A total score for the assessment is computed by the instructor. … Information is returned to the student about their performance.
We’re looking at the instructor’s method for administering an assessment.
taking an assessment
So we add the student’s role into our model:
Student | |
---|---|
take an Assessment | Assessment |
Instructor | |
---|---|
administer assessment to group of Students | Student |
The students’ responses to each question are collected by the instructor
This is really just describing the output from the request sent to students asking them to take the assessment.
Student | |
---|---|
take an Assessment: Response | Assessment |
the instructor, who grades them by comparison to a rubric for each question.
Response | |
---|---|
grade | |
Instructor | |
---|---|
administer assessment to group of Students | Student |
Response | |
Working with Rubrics
We are told there is a separate rubric for each question. So the “comparison” is between a response to a single question and a rubric.
This highlights the distinction between the response to an assessment and the responses to individual questions.
Response | |
---|---|
has QuestionResponses? | Rubric |
grade all question responses via a seq of Rubrics | |
QuestionResponse? | |
---|---|
grade(Rubric): score | |
Wait a minute…
Response | |
---|---|
has QuestionResponses? | Rubric |
grade all question responses via a seq of Rubrics | |
QuestionResponse? | |
---|---|
grade(Rubric): score | |
At this point, sanity reasserts itself * ITRW, when a student returns an exam sheet or a bluebook (the Response), those things don’t grade themselves. * They’re just paper * And while some anthropomorphism is common in OO modeling, that may be going a little too far.
What’s the alternative?
Response | |
---|---|
has QuestionResponses? | Rubric |
grade a seq of Questions via a seq of Rubrics | |
QuestionResponse? | |
---|---|
grade(Rubric): score | |
Looking for Variant Behavior
One reason that we really want to model the grading process is that we know that we have many different kinds of questions:
Question | |
---|---|
True/False Question | |
---|---|
Single-Choice Question | |
---|---|
Multiple Choices Question | |
---|---|
Fill-In-The-Blank Question | |
---|---|
Essay Question | |
---|---|
and we suspect that the grading method varies from one type of question to another.
Question | |
---|---|
grade(Rubric) | Rubric |
But I Don’t Like That Either
… and here’s why:
So maybe it’s the rubrics that capture this behavior
Grading - revised
Response | |
---|---|
has QuestionResponses? | Rubric |
grade a seq of Questions via a seq of Rubrics | |
QuestionResponse? | |
---|---|
grade(Rubric): score | |
Question | |
---|---|
grade(Rubric) | Rubric |
Rubric | |
---|---|
grade(QuestionResponse?): score | |
I’m much happier with that
We’re probably going to wind up with a small deck of Rubric cards for different variants.
The instructor may also elect to provide feedback (written comments), particularly about incorrect responses.
GradedQuestionResponse? | |
---|---|
Has score, feedback | |
A total score for the assessment is computed by the instructor.
GradedResponse? | |
---|---|
has GradedQuestionResponses | |
compute total score | |
recording grades
the instructor records the score in his/her grade book.
GradeBook | |
---|---|
record a score for a Student on an Assessment | |
Instructor | |
---|---|
administer assessment to group of Students | Student |
Response | |
Grade Book | |
returning information
Information is returned to the student about their performance.
It’s a pretty good bet that we don’t want a class with as vague a name as “Information”.
The clue is the description: “At a minimum, the student would learn of their score and any instructor-provided feedback.”
Student | |
---|---|
take an Assessment: Response | Assessment |
receive a GradedResponse | |
The Story So Far
Assessment | |
---|---|
Has questions |
GradeBook | |
---|---|
record a score for a Student on an Assessment | |
GradedQuestionResponse? | |
---|---|
Has score, feedback | |
GradedResponse? | |
---|---|
has GradedQuestionResponses? | |
compute total score | |
Instructor | |
---|---|
administer assessment to group of Students | Student |
Response | |
Grade Book | |
Question | |
---|---|
grade(Rubric) | Rubric |
QuestionResponse? | |
---|---|
grade(Rubric): score | |
Response | |
---|---|
has QuestionResponses? | Rubric |
grade a seq of Questions via a seq of Rubrics | |
Rubric | |
---|---|
grade(QuestionResponse): score | |
Student | |
---|---|
take an Assessment: Response | Assessment |
receive a GradedResponse | |
We might (cautiously) question whether some of the empty cards represent classes that we need to retain in the model.
Right now, we have as many questions as answers.
but finding useful questions is part of the process
We can’t go much further without more info