Rose Garden Promises of Intelligent Tutoring Systems: Blossom or Thorn?

Intelligent tutoring systems (ITS) have been in existence for over a decade. However, few controlled evaluation studies have been conducted comparing the effectiveness of these systems to more traditional instruction methods. Two main promises of ITSs are examined: (1) Engender more effective and efficient learning in relation to traditional formats; and (2) Reduce the range of learning outcome measures where a majority of individuals are elevated to high performance levels. Bloom (1984) has referred to these as the two sigma problem; to achieve two standard deviation improvements with tutoring over traditional instruction methods. Four ITSs are discussed in relation to the two promises. These tutors have undergone systematic, controlled evaluations: (1) The LISP tutor (Anderson Farrell and Sauers, 1984); (2) Smithtown (Shute and Glaser, in press); (3) Sherlock (Lesgold, Lajoie, Bunzo and Eggan, 1990); and (4) The Pascal ITS (Bonar, Cunningham, Beatty and Well, 1988). Results show that these four tutors do accelerate learning with no degradation in final outcome. Suggestions for improvements to the design and evaluation of ITSs are discussed.