Remote Evaluation of Remote Console Information Retrieval System (NASA/RECON)

NASA constantly seeks evaluation from users of its nationwide literature search service based on an interactive information retrieval system. This report explains the technique, which consists of sending out an evaluation form with each literature search, and the results derived from a compilation of the user's responses. In an eleven month period in which evaluation forms went out with 3,001 searches, 33.6% of the forms were completed and returned. The returns showed that 88.5% of the respondents found the searches suitable to their needs, 81% learned of valuable new references from the searches, and 93.5% received the searches in time to meet their needs. The significance of relevance or precision ratio in relation to user satisfaction is discussed, and an extrapolation from the users' responses resulted in a relevance ratio of 9.3%. Some of the general comments found in the responses are analyzed as indicators of what. the users expected from the information retrieval service.