Juliet L. Hardesty
Digital Library Program, Indiana University
March 30, 2009
Download Report (PDF, 332K)
View Executive Summary
View Recommendations
Variations is a digital music library system that provides online access to selected recordings and scores from the Indiana University Cook Music Library for use by IU Bloomington students, faculty, and staff. Variations2 Experimental Search is a search interface for items that have been cataloged for Variations. This content is small in portion compared to the catalog records contained in IU’s online library catalog (IUCAT) but the cataloging for Variations is more in-depth. By observing and interviewing people using Variations2 Experimental Search, logs analysis can be expanded on and qualitative information can be added to the findings and recommendations for the new web search interface.1 This new web search interface is part of the Variations/FRBR project currently under development.2
Observations and interviews of people using Variations2 Experimental Search were conducted from February 9 – February 16, 2009. Eight participants were observed and interviewed: 2 faculty (one from Jacobs School of Music at Indiana University Bloomington, the other from the IUPUI School of Music and Arts Technology), 4 graduate students, and 2 undergraduate students (all affiliated with the Indiana University Jacobs School of Music). Observations were conducted by a note-taker watching each participant use Variations2 Experimental Search and taking notes by hand; system logs were not consulted for participants’ actions. Figures that follow are minimally accurate but might not reflect the exact activity of the participants. Interviews were conducted subsequent to the observations and notes of the participants’ responses were recorded by the note-taker (see Appendix A for interview questions). The findings and recommendations that follow reflect what occurred in the sessions and build on the findings and recommendations from the logs analysis conducted on non-cataloging searches in the logs from Variations2 from March 30 – May 3, 2008.
In the observations, participants used Basic Search by far the most (69 searches were recorded in Basic Search, 17 in Advanced Search, and 9 in Keyword Search); similar to what was found in the logs analysis. The average number of searches conducted per session was 12, with the top number of searches recorded in one session being 27. Advanced Search was used proportionately more in the observations/interviews than in the logs analysis, outnumbering the use of Keyword Search by almost twice as many uses. All 8 participants used Basic Search, 3 participants used Advanced Search, and 4 participants used Keyword Search, so Basic Search was still given predominance as the default search option, with the other search tabs receiving less attention.
In interviews, participants did not report a preference between Basic, Advanced, or Keyword searches, but the majority did report that any new web search should include Creator/Composer, Work Title, and Performer/Conductor as search fields. These fields are currently available in both Basic and Advanced Search. As the previous charts show, Creator/Composer, Work Title, Performer/Conductor, and Media Format were used the most in both Basic and Advanced Search. In comparison to similar data in the logs analysis, Performer is more prominent in the observation data than in the logs but Creator/Composer and Work Title are still utilized in a majority of searches that were observed, which is similar to what was found in the logs analysis.
Of the 95 searches conducted, 6 of those searches by 4 participants contained misspellings (for example, “german” instead of “gershman,” “Appalachain,” “mozat”). This is more than twice as many misspellings as those noted in the logs (6.3% of searches misspelled in observations versus 2.8% misspelled in the logs). Some of the disparity could be due to knowing the participant’s intentions when conducting observed searches (“german” is technically not a misspelled word) but the essential point is that spelling mistakes occurred and they occurred across multiple participants.
In addition to more than twice as many occurrences of misspellings, the notes from the observations and interviews show the qualitative effect of misspellings on participants. Half of the participants who searched for a misspelled term conducted an additional search before the mistake was noticed, wasting the time and search efforts of the participant. One participant in the interview made special note that a Top Feature for a web search interface should be some sort of spelling aid. That participant noted that not only does a spelling aid help prevent misspellings when entering search terms but suggested search terms can help form better searches. During the observation, this participant conducted an Advanced Search for the Creator/Composer “bruckner” and the Work Title “meta” which brought back 0 results for that search but indicated there was 1 match for the creator “bruckner” and 8 matches for the Work Title “meta.” The participant noted at that time that spelling suggestions would have helped determine that any Work Title searches containing “meta” would not be related to Creator/Composer searches containing “bruckner” and that entire failed search path could have been avoided before any search was even conducted.
One participant observed and interviewed in this study is a blind undergraduate student in the Jacobs School of Music. This participant makes use of screen reading software; the built-in screen reader for Mac OS X (VoiceOver3) was used in this session but this participant reported using the Variations2 Search with screen reading software on Windows (specific software not noted) and on Linux (Orca with Firefox browser on Gnome4). This participant explained many difficulties encountered when using the Variations2 Search, including unlabeled form fields and the inability to focus on search results without moving through word by word. He also reported different kinds of accessibility problems when using different screen reading software on different operating systems, but on Mac OS X this participant was observed having difficulty reading through the search results screen, either to see results or to see a message when 0 results were returned. He also reported that one of the features he would like to have included in an online search in Variations would be a way to “select what you want to see in the search results.” While there is a “Sort By” feature included in the current Variations search, it is only enabled when there are search results showing and otherwise displays as disabled drop-down menus and a disabled checkbox. These form features were never used or found by this participant, so it is possible that there are problems for users of screen readers when form elements toggle between being disabled and enabled on a screen that is not fully reloaded.
This participant also provided information regarding his experience using the Variations Player in addition to the search interface. The Variations Player available in Variations2 for playing digitized musical recordings is only partially usable, according to this participant. This participant using keyboard shortcuts could not control the volume slider on the player. In addition, the image buttons used to stop, play, fast-forward, rewind, and pause do not have “alt” attributes so they are not identifiable controls to this participant without some trial and error testing. The lack of keyboard shortcuts presents a special problem when using screen reading software with the player since the screen reading voice gets in the way of listening to the music and the music playing gets in the way of hearing the screen reader voice.
When told that a new player was being developed using Flash, he said that nearly anything developed using Adobe’s Flash would be inaccessible by him or, he believes, any users using screen reading software. Adobe’s information regarding the accessibility of Flash5 and Flex6 explain that there are ways to develop accessible Flash and Flex software for JAWS and sometimes Window-Eyes (2 screen readers available for Microsoft’s Windows operating system) but these are not standard Flash or Flex development methods and to use Flex applications with JAWS requires that the end-user download and install special scripts (see "Using Flex with JAWS").7
So reports from Adobe claim that accessibility is possible in Flash and Flex applications, but it is not standard when developing and requires extra work and configuration for the user, who may or may not know about those required configuration changes. Even if screen reading software has the capability to handle a certain feature programmed a certain way on a web site, if that capability is not a common use of that screen reading software, the accessibility of that feature is questionable. In addition, these accessible features of Flash and Flex seem to only be available for users of JAWS with Microsoft Windows. That excludes any users of any other screen reading tools used on any other operating systems (similar to developing an application that only works in Internet Explorer for Microsoft Windows). Any development of Flash or Flex applications should only be done if it makes sense for the purpose of development (to showcase, highlight, or demonstrate features) or if there is no other reasonable way to develop the needed interaction (using AJAX adds an unreasonable amount of development time and complexity to the code and there is an accessible development option available using Flash or Flex).
A more formal accessibility evaluation of the Variations2 Experimental Search is being conducted by the University Information Technology Services (UITS) Adaptive Technology Centers (ATC) and should provide a complete list of inaccessible features and recommendations for creating a search that can be used by anyone, including those with disabilities. While the new online search interface will essentially be a development from the ground up, the ATC’s report will provide fundamental accessibility concepts that will be useful for planning functionality and how that functionality should be programmed.
Participants were asked to identify what they liked and disliked most about the current Variations2 Search. 3 participants specifically stated that they liked having Variations available to do a focused search for scores and recordings, as opposed to searching in IUCAT, which requires sifting through non-music search results or knowing “tricks” to return digitized recordings of music. 2 participants liked the layout of the Basic Search screen and thought the options were clear, and 2 participants specifically noted that the “Key” filter was a search feature they really liked. 1 participant mentioned the information buttons (the “I” in the blue circle) in the search results as a favorable feature, requesting that it be a continued in the new search interface, and 3 participants were observed using that link in various search results.
Common dislikes reported were that no instrumentation search field is provided (2 participants reported this) and failed searches (0 results) happen too often with no good way to recover. 5 participants complained that searches failed too often and requested features like linked search suggestions when they are provided and a 0 results message that is accessible in a screen reader. 3 participants also complained that it was hard to discern in the search results between “intermediate” results that lead to a listing of individual items and a search resulting in a direct list of individual items.8
Participants were also asked to describe the 2 or 3 features they would most like to see in an online search for Variations. 7 participants requested that Work Title and Creator/Composer be continued as search fields and also be shown in search results, coinciding with the data gathered from the logs analysis. 4 participants requested that Performer/Conductor be offered as a search field and shown in search results, and 2 participants made the request again for an instrumentation search field. As a result of the complaint regarding intermediate results, one participant asked that the number of hits contained within a set of intermediate results be provided in that search result listing to help indicate that the results being viewed were not the actual items. One participant also requested that a spelling aid be included, giving a voice to the observed negative effect of misspellings on participants’ searches. And finally, 1 faculty participant requested that the new search be constructed so that written instructions to students on how to use it are not necessary.
5 of the 8 participants reported using the Variations2 Experimental Search previously but no one used it regularly. All participants reported using IUCAT to search for digitized musical recordings and scores on average about 3-4 times per week. An online search system capable of finding specially cataloged digitized musical resources will be beneficial to music students and faculty at Indiana University. Based on observations and interviews with participants using Variations2 Experimental Search and findings from the logs analysis of Variations2 Experimental Search, the following are recommendations for creating an online Variations search:
