Variations2 Experimental Search: Logs Analysis Findings & Recommendations

Juliet L. Hardesty
Digital Library Program, Indiana University
March 30, 2009
Download Report (PDF, 632K)
View Executive Summary
View Recommendations

Introduction

Variations is a digital music library system that provides online access to selected recordings and scores from the Indiana University Cook Music Library for use by IU Bloomington students, faculty, and staff. Variations2 Experimental Search is a search interface for items that have been cataloged for Variations. This content is small in portion compared to the catalog records contained in IU’s online library catalog (IUCAT) but the cataloging for Variations is more in-depth. By reviewing the logs for the use of this search interface, it is hoped that evidence can be gathered for what users do when searching Variations and how best those actions can be accomplished in a new web search interface. This new web search interface is part of the Variations/FRBR project currently under development.1

Logs analysis was conducted on Variations2 Experimental Search for non-cataloging searches recorded to server logs from March 30 – May 3, 2008. The findings and recommendations that follow are based solely on this analysis and only apply to the creation of an online web search interface for Variations.

Variations2 Experimental Search contains 3 types of search: Basic, Advanced, and Keyword. Basic Search is constructed of the following fields: Creator/Composer, Work Title, Performer/Conductor, and the following filters: Media Format and Key. Advanced Search is constructed of the same fields and filters as Basic Search with the addition of the following fields: Recording/Score Title, Other Contributor, Publisher, and Subject Heading. Keyword Search includes a Keyword search field and the Media Format filter.

This logs analysis report will look at the use of the search forms and not at the success or failure of the search results or the users’ use of those search results. Without having the user present to interpret the search results, any conclusions gathered from the logs regarding the usefulness of a search would be inherently unreliable.

Overall Searches

There were 2,454 non-cataloging searches conducted in 797 sessions from March 30 – May 3, 2008.2 The average number of searches conducted per session was 3, with the top number of searches in a single session being 39.

The following provides the total number of searches for each type of search: Basic (2,172), Advanced (94), and Keyword (188).

Number of Searches March 30 - May 3, 2008 (2,454 Total): 2,172 Basic searches, 94 Advanced searches, 188 Keyword searches

Basic Search was used the most by far, probably due to the fact that it is the default search option. The unique search terms listed for Basic Search show that most searches for Composer were for names of people or groups, most Work Title searches were for titles of music (or at least types of music – “sonata,” “Symphony,” “Concerto”), and most Performer searches were also for names of people or groups. So the audience for this search interface appears to understand the search fields; the labeled search fields appear to mean something in Basic Search.

Keyword Search was the 2nd most-used search, but lagged severely behind Basic Search. It was used twice as much as Advanced Search and shows a mixture of people and group names, musical types/titles, in addition to call numbers, URLs, complex Boolean searches, and instrumentation.

Advanced Search offers more options for searching than Basic Search but those options were used very little and sometimes incorrectly (for example, a search for the call number “LP .T6177 K2.1” was conducted as a Recording/Score Title search). Relatively little difference exists between the use of Advanced Search and Basic Search and can be viewed visually in the following 2 pie charts which show a heavy use of the Creator and Work Title search fields with more limited use of the other fields and filters, particularly those in Advanced Search that are unique to that search form (Recording/Score Title, Other Contributor, Subject Heading, and Publisher).

Advanced Search (94 Total) - Number of Searches per Search Field/Filter: 51 Creator searches, 42 Work Title searches, 15 Performer searches, 8 Media Format searches, 12 Recording/Score Title searches, 4 Other Contributor searches, 4 Subject Heading searches, 3 Publisher searches

Basic Search (2,172 Total) - Number of Searches per Search Field/Filter: 1,745 Creator searches, 1,317 Work Title searches, 179 Performer searches, 116 Media Format searches, 20 Key Letter searches, 20 Key Mode searches, 9 Key Accidental searches

There were many instances of misspellings in the search terms entered throughout the logs (i.e., “Piao Trio”, “la triviata”, “marrige of”, “ainadanar”, “Brhams”, “hoagy carmicheal”, “Medelssohn”). A complete analysis of all misspellings is unknowable since user intentions are unknown, but verifiable misspellings, such as the examples previously listed show in at least 69 of the 2,454 searches. Even though this number appears small (3% of total searches), the effect of a misspelled search term is detrimental to the user’s experience and can lead to unnecessary frustration and failed searches.3 Spelling help or suggestions are currently not provided when search terms are being entered and this does affect users.

Basic searches

Basic Search (2,172 Total) - Top 5 Search Field/Filter Combos: 951 Creator & Work Title searches, 658 Creator searches, 270 Work Title searches, 127 Performer searches, 56 Creator & Work Title & Media Format searches

Most Basic Search searches were for Creator and Work Title (951) with searches for just Creator (658), just Work Title (270) and just Performer (127) following. Overall, however, the spread was pretty even between searches that used 1 search field or filter (1058) and searches that used 2 search fields or filters (1015).

Other interesting facts that might point out topics in courses for that time of year (the final month of the Spring 2008 semester): “Beethoven” was by far the top Creator search (162). “Symphony” was the top Work Title search (44) followed by “Ainadamar” (27).

One puzzling detail in the logs structure, not the data: The search interface shows Media Format listing all format options for searching. However, the logs show searches for “format” and searches for “format(type)” as two distinct search filters. The only 3 selections that were ever made for format(type) were All Recordings (67), All Scores (33), and All Encoded Scores (1). The only 3 selections that were made for format were CD (10), Full Score (3), and Reduced Score (2). All other search fields or filters (Performer, Key Letter, Key Mode, and Key Accidental) were not used much or did not have many unique search terms used repeatedly (Performer had many singularly unique searches).

Advanced searches

Over half of all Advanced Search searches contained Creator (51) while just under half contained Work Title (42). Only half of all the searches for Creator and Work Title combined them together though (27). Of the 94 Advanced Search searches, 61 of them used only 1 search field or filter, 25 used 2 search fields or filters, and 8 searches used 3 or 4 search fields or filters. In relative comparison, similar use can be seen in the Basic Search field/filter combinations. A heavy majority of searches in both Basic Search and Advanced Search used only 1 or 2 fields or filters.

Advanced Search (94 Total) - Searches by Field/Filter Combinations: 61 searches used 1 field/filter, 25 searches used 2 fields/filters, 6 searches used 3 fields/filters, 2 searches used 4 fields/filters

Basic Search (2,172 Total) - Searches by Field/Filter Combinations: 1,058 used 1 field/filter, 1,015 used 2 fields/filters, 78 used 3 fields/filters, 21 used 4 fields/filters

45 sessions made use of Advanced Search and the average number of searches per session was 2 (the highest was 14 searches in one session). No search terms were heavily searched multiple times.

Keyword searches

Media Format was only specified for 13 of the 188 Keyword searches and all 13 of those searches used All Recordings as the selected format. Keyword search was used in 81 different sessions and the average number of keyword searches per session was approximately 2 (the highest was 12 keyword searches in one session). No search term was repeatedly used on a major basis, but “norton anthology” was searched 11 times in Keyword searches, followed by “ainadamar” (8) and “norton” (6).

Recommendations

The following are recommendations for a web search interface for Variations and are based solely on the findings from the logs analysis previously described.

  1. Combine Basic and Advanced into one search form.
    Based on data from the logs, Advanced Search was not used nearly as much as the Basic Search and when Advanced Search was used, it was mostly used the same way as Basic Search (searches for Composer and/or Work Title). Providing the simplest and most efficient interface for searching will entice users to use the new Variations web search. As such, reducing the number of search options from 3 to 2 (a “regular” Search and a Keyword Search) will simplify the interface. As the new search will be a web-based search interface, the additional options currently available only in Advanced Search could be offered not by the default search that is loaded but in the same form via a link for “Advanced Search Options” which could expand to display those additional search fields (Recording/Score Title, Other Contributor, Publisher, and Subject Heading).
  2. Continue using labeled search fields and filters in combined Basic/Advanced Search.
    Offering the options for specific searching through labeled search fields does seem to be helpful to users, according to the terms entered in the search forms. Much of the searching for Creator/Composer involved names of people or groups and searches for Work Title involved some kind of title or at least a type of music. In addition, since Keyword Search conducts a full-text search, the default of a focused search for the composer “glass” or the work title “glass” versus “glass” anywhere in the item record is preferable.4
  3. Provide spelling help and/or suggestions.
    The logs are recording spelling mistakes when users enter search terms into the Variations search form. Any sort of spelling help that can be provided will improve the searching experience. This could include a pre-set list of authority names and titles or a 3rd party spell-checker that would work as users are typing terms into the search fields or a “Did you mean…” link or other spelling suggestions linked for searching after the search has completed, or a combination of any of these options.

Notes

  1. Information on the IMLS grant-funded Variations/FRBR project is available at http://www.dlib.indiana.edu/projects/vfrbr/
  2. Since users must log in to use the system, a user logging in to perform cataloging functions is identified in the logs with a type of session labeled “IU_CATALOGING.” Only searches that did not contain that labeling in the logs were used in this analysis.
  3. Willson, Rebekah and Lisa M. Given. (2008). The Effect of Misspellings on Information Retrieval in Online Public Access Catalogues. In Proceedings of the 36th annual conference of the Canadian Association for Information Science (CAIS), University of British Colombia, Vancouver, June 5-7, 2008. Retrieved March 18, 2009, from http://www.cais-acsi.ca/proceedings/2008/willson_2008.pdf
  4. Keyword Search functionality based on information from Variations2 User Guide: Searching. Retrieved March 18, 2009, from http://variations2.indiana.edu/use/search.html