Two Approaches for Enabling an Adaptive Distance Learning System
Guillaume Nerzic, Shunichi Toida, Christian Wild, and Mohammad Zubair
Department of Computer Science, Old Dominion University,
Norfolk, VA 23529-0162, USA
Abstract.The wide spread availability of computers has not only created a need for well-trained and computer-literate people but has also provided new opportunities in the practice of education. There is a demand for just-in-time training, relevance of material to current needs, and life-long education. Combined with rapidly developing computer and network technology, this requires new models for instructional presentation, delivery and utilization of knowledge. In particular, there is a need for asynchronous distance learning systems at the undergraduate level that are adaptive to the diverse background of students. In this paper, we discuss two approaches that we are exploring to enable an adaptive distance learning environment. First approach, which is based on a course knowledge base is being tested for a course covering discrete mathematics. The second approach, which is based on the spiral model, is being tested for the beginning programming sequence. These two courses form the cornerstones of background material needed by community colleges student who transfer into a BS Computer Science program as well as for traditional students.
The changing mission of institutes of higher education has led to increased
interest in the use of emerging network technologies to support distance
education programs. These changes have been brought about by the need for
life long education in fast moving industries, desire for second career
choices, and the spiraling costs of higher education. Aside from perceived
cost advantages of utilizing computer and communications technologies for
education and training, distance education purports to offer self-paced
just-in-time learning which provides the efficient delivery of information
to the students as they need that information. However to reach promise
requires new learning models and new methods of organizing, delivering
and managing instructional material [Benyon 1997].
In this paper, we discuss two approaches to an adaptive distance learning
environment. These two approaches arose out of a need to support two cornerstone
courses in the undergraduate computer science curriculum. One course covers
discrete mathematics, which forms the theoretical basis for computer science.
The other course is an introduction for programming, which forms the practical
basis for computer science. Because of the great difference between these
two courses, we proposed and implemented different approaches. The first
approach is based on a course knowledge base in which the instructional
material is organized as a dependency graph between the instructional topics.
Students can explore any material for which they have the prerequisites.
At each stage, the student can be evaluated to determine their preparation
for more advanced material. Key to this approach is careful management
of the student's progress through the material based on demonstrated preparation
in the prerequisite material. Using this approach, we have designed course
material for a discrete structures course. The second approach, based on
the spiral model, is being tested in the introductory programming course
in C++. The key objective of this approach is to handle the diversity in
backgrounds of students who traditionally enroll in this course. The spiral
model uses a data base of related material which differs in difficulty,
assumed background and delivery modes. While we originally believed these
two courses to be sufficiently different to warrant two different approaches,
we now believe that the two approaches should be combined to provide a
general infrastructure for adaptive web based learning.
The rest of the paper is organized as follows. In the next section,
a system overview is given which outlines the features in a common learning
environment. Next, a description of the Knowledge Based Model developed
for the discrete structures course is given. The Spiral Model is presented
next which is followed by a discussion of future directions and conclusions.
The objective of this project is to provide any-time,
any-where instruction adaptive to students' background with the state of
the art technologies in Web, multimedia, and artificial intelligence.
Conceptually the general model consists of the following
six components: 1) Student Profile, 2) Knowledge Base, 3) Course of Study,
4) Query-Answer Unit, 5) Diagnosis Unit, and 6) Tracking Agent.
The "Student Profile" stores student record information
and state of their progress in the subject. The "Knowledge Base" is a collection
of "Knowledge Modules" on the subject. There may be several modules for
each major topic such as logic, set theory, programming syntax etc. A "Knowledge
Module" can consist of a) "Topic" b) "Learning Task" with immediate grading
and explanation of answers c) "Quiz" which is graded off-line or d) "Examples".
Any prerequisite relationships between knowledge modules are also represented
by a dependency graph. Different modules covering the same material will
be geared to student with different backgrounds or learning styles. From
a "Topic", students will be able to access related "Topics" (for instance
simpler or more advanced presentations of the material, or presented with
a different style), "Learning Tasks", and "Examples". The "Course of Study"
component is a suggested route through the knowledge modules designed by
the instructor(s) for certain "stereotypical" students. The "Course of
Study" also sets learning goals which prescribe the number and difficulty
of "Learning Tasks" and "Quizzes" the student must complete. Scheduled
meetings with the instructor (perhaps over a video conferencing system)
for oral exams or presentations are also part of the "Course of Study".
The operation of the system is controlled by the "Tracking
Agent". When the user begins a session, the "Tracking Agent" interacts
with the "Student Profile" and "Course of Study" to suggest appropriate
"Knowledge Modules" for the student's attention. Student's answers to "Learning
Tasks" or "Quizzes" are sent to the "Diagnosis Unit" for analysis. The
results of this analysis are recorded in the "Student Profile" and used
to further guide the student by suggesting remedial material or by skipping
ahead in the "Course of Study". Student's questions are handled by the
"Query-Answer Unit" that can analyze and automatically answer certain classes
of queries or forwarded to the instructor as appropriate.
In order to gain a better understanding of the issues
involved in building adaptive web-base course, we explored two different
approaches focused on two different courses. We also choose different implementation
technologies to gain a better understanding of their suitability to support
this project.
In this approach, the individual topics are related by a dependency graph which captures the pre-requisite structure between these topics. Students entering the web site are allowed to access any topic for which they have the necessary prerequisites. For each topic there are learning tasks at the end of the presentation. These are simple yes-no questions and are intended for testing the basic understanding of concepts/techniques presented. Each student's answer is corrected and commented on with canned comments. The number of questions varies according to student's performance. More elaborate exercises are given at the end of each section/chapter. Questions there are not necessarily multiple choice. The student is often going to be asked to compose an answer and type it in. Student's answers are evaluated and the results together with comments are fed back. We feel that this kind of exercises is very important in learning. Artificial intelligence is sufficiently advanced in certain areas such as automated reasoning for logic problems for example and it can be utilized to realize non-multiple choice exercises for certain subjects such as prepositional and predicate logic. In ARTS, the automatic theorem prover TPS [Andrews 1997] developed at CMU is used to check the correctness of student's answers to non-multiple choice questions on prepositional and first order predicate logic. Student's questions are handled mainly off-line by e-mail, chat room, and FAQ. Periodic synchronized recitation sessions are going to complement these. It is also our plan to explore possibilities of automatic processing of questions at least for certain types of questions in certain subject areas. The works of Pilkington [Pilk 1992][Pilk 1996] and Cawsey [Caws1998] for example are quite instructive here.
The block level architecture of the Asynchronous Remote Tutoring System (ARTS) prototype developed for the knowledge-based approach is shown below.
Figure 1. A Block-Level Architecture of Asynchronous Remote Tutoring System.
The proposed architecture is a Web based system. The ARTS client is based on Java and runs on a standard Web browser that supports Java. The client displays the system and course information, gathers data from the user and manages the user's state. The ARTS server is also based on Java and works with the Web server. A request by a client, in our case Java applet running on the browser, is handled by the Thread Manager. Depending on the request, the Thread Manager passes the request to the User Manager or the Course Manager. The User Manager manager the set of user profiles. The Course Manager is responsible for managing all the courses. The Knowledge Base is the collection of knowledge on the subject and consists of a number of modules. There is one module for each major subject such as logic, set theory, relation, function etc. for the discrete structures course for example. Each module is represented by Concept Map which relates different concepts in a prerequisite structure represented by a dependency graph. This graph will be rooted in primitive concepts (for a particular subject) and will contain dependency links between concepts. In addition an optional modeling link will allow adaptation of the material to students with different backgrounds. For instance, a topic in discrete structures, such as predicates, might be explained in several ways depending on the mathematical background of the student. Although a specific background may be a required pre-requisite of the course, we would like to take advantage of any additional background of a student in presenting a concept. Thus a given concept will have presentation material based on different backgrounds and at different levels of difficulty. The Profiles represent both static (student record information) and dynamic (state of their progress in the subject). The dynamic state is determined by the assessment and curriculum tests. The student profile can be mapped onto both the concept map and curriculum map to visualize the student's progress. The Course Agent is the most important object in the whole system. It implements different course navigation policies. This object, a Java applet, is downloaded from the Course Manager server module. Please note that the Course Agent is specific to a course. The Course Agent communicates directly with the server to obtain the current topic object and displays this object on the screen. It is responsible for validating the request for a topic. It writes its state to the dynamic profile allocated for the current course. The dynamic profile, as discussed previously, extends from the Dynamic Profile super-class and is implement to cater for the needs of a particular Course Agent implementation. When the user logs out of a course the Dynamic Profile is transmitted to the server to be stored on permanent storage. Because the Course Agent may behave in different ways, it also has its own interface, implemented inside the Interface Drawer object.
The knowledge-based approach has been adopted for CS 281 Discrete Structures course. At the moment the Concept Map has been completed and topic objects for various contents with learning tasks have been completed for Logic module covering propositional and predicate logic. Also work is under way for incorporating TPS [Andrews 1997] to diagnose student's answers to logic quiz questions.
The activity of programming computers is fundamentally an exercise in problem solving. Problem solving is characterized by the integrated use of a set of skills, each of which contributes towards the overall solution. The role of each skill in the overall process forms a conceptual model of the problem solving activity. The complexity of computing systems in general and software development environments in particular means that this learning process will take place with only an incomplete understanding of the many skills necessary to complete a given programming task. In fact the existence of different competence levels among practitioners is characteristic of a field in which expertise is gained over many years of learning, improving skills and integrating those skills into more and more sophisticated conceptual models and theories. When learning a complex subject, it is often desirable, and even necessary, to use conceptual models that are inaccurate or overly simplistic in some particulars. Since problem-solving skills are best honed by doing, a student needs to acquire a certain minimal set of competence in that set of skills necessary to perform a group of problems in the domain of study. The spiral instructional model organizes the subject material into a series of Competency Circles that allow the student to tackle a given set of problems. Thus the learning process is driven by a need-to-know in order to solve a given set of problems with a given set of skills and conceptual models. It is the instructor's role to define the Competency Circles and to define a series of problem solving activities through which the student will navigate towards increased competency.
In our initial implementation, we envision four
roles: author, instructor, teaching assistant and student. The author is
responsible for creating instructional material for the course. This material
will be the core of an electronic textbook, organized along the principles
outlined below. The instructor is responsible for organizing the instructional
material into a particular course given to a specific set of students.
Presumably the instructor organizes the material based on his or her instructional
style and the needs and capabilities of the particular class. The instructor
is free to use and adapt the material provided by the author and can supplement
this material with that of his/her own. The instructor is responsible for
setting the objectives for a particular course. The teaching assistant
is responsible for analyzing the progress of the students and providing
feedback to the instructor about each student's performance. The student
is responsible for the real job of learning.
Topics Database: The subject material
is organized as a set of information topic explanations, learning tasks,
examples, and tests implemented as HTML pagelets. Each item in the database
is indexed in the following ways:
In addition, each entry has the URL of the actual
content material. In order to address different audiences, there may be
several explanations of the same topic, each suitable for a different background
or learning style. For the fall 1998 semester, we have adopted a format
for explanation pages, which provides hyperlinks to other explanations
of the same topic but at different levels, hyperlinks to additional examples
and hyperlinks to related learning tasks. Since all these hyperlinks go
through the webpage synthesis engine, all material in the subject database
related to this topic is available from a given page, even those pagelets
that have been added later.
Curriculum: Defines competency circles
by gathering topics into a problem solving skill set. The curriculum is
defined as HTML web pages with hyperlinks to associated information from
the topics database. Each HTML page can correspond to a chapter in a traditional
textbook, to a lecture that might be drawn from several information sources
or to exercises and projects for the student. Such curriculum material
can be generated by the author based on an understanding of the subject
material or by the instructor based on personal instructing style, an understanding
of student population, goals of the academic program, and institutional
policies. The current implementation uses regular the anchor tag in HTML
to reference the synthesis CGI script with a query string giving the parameters
of the data base search in the topics database. For example, to refer to
a learning task pagelet on while loops in C++ you would use the following
HTML tag
<a href="synthesis.cgi?topic=loops&subtopic=while&type=LearningTask">
while loops </a>
The actual pagelet returned depends on the synthesis
engine (to be described). One of the advantages of the current implementation
is that the instructor can design the curriculum without being aware of
the specific material in the topics database. New material can be added
at any time. Since the hyperlink is to a CGI script instead of an html
page, new material can be accessed from existing links as soon as it is
available.
Web Page Synthesizer: This is the heart
of the adaptive learning system. The current implementation of this component
uses a rather straightforward algorithm in choosing which material to present
next. The fields in the query string of the HTML anchor tag are used to
query the subject database. If multiple entries are found, a web page listing
all choices is shown with an indication of which pagelets the student has
already visited. We are investigating ways to automate the choice of the
next page to show the student based on an assessment of the student's current
progress.
A prototype implementation of this model is being tested in the Fall 1998 semester along side a traditional lecture presentation of the same material. There are 10 students taking the web-based version of this course and 100 students taking the traditional course. However, students taking the traditional course were invited to use the website and at least 40 have elected to do so. Use of the web site is monitored so that usage statistics can be analyzed. A post-course survey will be given to capture student's experiences and to identify good and bad features of the web course. In addition one student with a marketing background in information technology systems has volunteered to do a needs analysis from the student's point of view. Thus, there should be some data and lessons learned available at the end of this course.
Our original decision to develop each course using a
different approach was based on the belief that the two courses were sufficiently
different to justify these different approaches. However as we have gained
experience with both courses, we now feel that an integration of the two
approaches would allow more flexibility in designing, organizing and managing
web courses. For instance, while there is a great diversity of programming
backgrounds among the students taking the introductory programming courses,
there is also quite a variety of mathematical skills among the students
taking the discrete structures course. On the other hand navigation through
the competency circles would be aided with the availability of the dependency
graph in the knowledge based model approach. In fact it should be possible
to generate a course of study from an analysis of the student's background,
achievement on assessment tests, competency skill sets and the dependency
graph. Thus one of the areas for future work is to add a diagnosis program
that will make appropriate choices for the student based on an analysis
of the student's background, progress and learning goals. This facility
could be expanded to include advice on how to proceed at any point, particularly
when the student first logs in (thus answering the question: What do I
do next?). The use of student models and individualized diagnosis by computer
is controversial [Frasson 1996, Gugerty 1997]. However, we plan to look
at a diagnosis system based on stereotypical students. Diagnosis will involve
categorizing the student into one of several well-defined categories. Each
category will contain its own learning strategy, lesson plan, explanations,
learning tasks, etc. The instructor will prepare appropriate curriculum
and associated competency circles for each class of stereotypical students.
The current system uses Java and JavaScript embedded in
the HTML page for automatic and instantaneous evaluation of the student
on learning tasks. Such evaluation is currently limited to multiple choice
questions and simple fill-in-the-blank learning tasks. However, learning
is a creative art that synthesizes a complex technical solution. Since
there are theoretically an arbitrary number of solutions to any given problem,
it is often difficult to provide extensive personal feedback to the student.
Automated programming understanding is area that has received much attention
[Song 1997] and it may be possible to adapt some of
that work to automatically evaluate the programming skills of the student.
For the discrete structures course, the availability of automatic theorem
provers can be used to prove the equivalence of a student's answers to
the correct answers.
The protocol between a browser and a web server is essentially
episodic. Each access to the web site is a separate episode with limited
state information kept between episodes. The current implementation attempts
to inject a degree of continuity by recording and reporting progress. However
more sophisticated methods of dialog management would be desirable. We
plan to investigate how current research in dialog management can be adapted
to a web-based course.
[Andrews 1997] Peter B. Andrews & Matthew Bishop,
TPS: A Tool for Proving Theorems Theorem Proving in Higher-Order Logics,
1997, Murray Hill, New Jersey. Also http://www.cs.cmu.edu/afs/cs.cmu.edu/user/andrews/www/tps.html.
[Benyon 1997] David Benyon, Experience with developing multimedia courseware
for the World Wide Web: the need for better tools
and clear pedagogy, Int. J. Human-Computer Studies (1997),
47, 197-218.
[Caws 1998] Alison Cawsey,
``Planning interactive explanations," in readings in Intelligent User
Interface, Ed. Mark T. Maybury and Wolfgang Wahlster, Morgan Kaufmann,
1998.
[Frasson 1996] Claude Frasson and Esma Aimeur, A comparison of three
learning strategies
in intelligent tutoring systems, J. Educational
Computing Research, Vol. 14(4), pp. 371-383, 1996.
[Gugerty 1997] Leo Gugerty, Non-diagnostic intelligent tutoring systems:
Teaching without student models, Instructional Science, Vol
25, pp. 409-432, 1997.
[Pilk 1992] Rachel M. Pilkington, Question-answering for intelligent
on-line help: The process
of intelligent responding, Cognitive Science, 16(4)
455 - 491, 1992.
[Pilk 1996] Rachel Pilkington and Alec Grierson, Generating explanations
in a simulation-based learning environment,
Int. J. Human-Computer Studies, 45,
527-551, 1996.
[Song 1997] An Intelligent tutoring system for introductory
C language course, J.S. Song, S.H. Hahn, K.Y. Tak, J.H. Kim, Computers
Educ, Vol 28(2), pp. 99-102, 1997.