INFORMATION SERVICES & INSTRUCTION FUNCTIONAL TEAM:
Virtual Reference at Queen's University Library:
An Investigative Report
Submitted by the ISI Team May 2003
Purpose of ReportThe provision of a live online reference service was explicitly identified as a major initiative in the 2002 Queens Library Strategic Plan. A live online reference service would provide an additional method for the provision of reference services to Queen's faculty, students and staff. The ISI Team was asked to explore live online virtual reference and report its findings to the Management Team. This report summarizes the ISI Team's investigation as of April 30, 2003.
Current methods of reference service at Queen's University Library range from the traditional method of face-to-face services, e-mail and telephone. Virtual reference does not replace current and traditional methods of reference service rather it capitalizes on another medium of delivery and communication - the World Wide Web.
There is some confusion as to what constitutes online virtual reference. The word live is included with virtual reference to differentiate this service from other methods of internet mediated reference services such as e-mail. E-mail (asynchronous) is currently used as a method of communication and reference service at Queens University Library. Unlike e-mail, live virtual reference (synchronous) allows patrons to receive information over the internet, in real time.
The rise of live online virtual reference at universities, public libraries and government libraries is a direct outgrowth of the dominance of the internet and parallels the commercial world's 24/7 marketing strategy. In fact, virtual reference software is derived from call-centre software with the explicit advantage of offering service around the clock from any location.
Within academia, the growth and success of the virtual library has created an environment for virtual reference. As collections become accessible remotely, live online virtual reference is a tool to insert a human service element into interactions between user and collection. Contrary to a utopian view of electronic resources, it is still necessary for staff to mediate some proportion of user transactions, even those completed offsite.
Library literature and academic listservs are drowning in testimonials of virtual reference at individual institutions and collaborative efforts among libraries who unite for geographical, logistical, subject or opportunistic reasons. Within Ontario, individual libraries are offering virtual reference (Windsor, Waterloo) and an OCUL pilot project among three institutions (Guelph, Ryerson and York) is underway. Across Canada, partnerships have sprung up opportunistically (UNB and Alberta). The National Library of Canada, seen as a pioneer for virtual reference on the international stage, is launching a national service and has invited libraries across Canada to participate in their initiative. Outside of Canada, there are examples of virtual reference collaborations based upon subject-expertise (engineering libraries, law libraries) and collaborations among libraries who serve a common "user" across a region (state government libraries, branches of a state-wide public library). All of these models offer benefits to their user groups.
While the exact features of virtual reference vary among software products and individual library configurations, the common concept is to provide a mechanism for library users to obtain assistance at their point of need. Virtual reference software also permits library staff to "push pages" or send information to the user that responds to their query. Chat software is the dominant feature that facilitates conversation between the library user and a library staff member. The ISI Team has summarized the benefits and concerns about virtual reference software. A summary of virtual reference characteristics is listed below.
Benefits To Users:
Chronology of ISI Team Investigation of Virtual Reference
The ISI Team had a webcast training of LSSI Virtual Reference Toolkit on March 10, 2003. In the days following training, team members practiced with the software and learned the basic steps to use this software. During the webcast training and in the practice sessions, some team members experienced loading problems and poor response time. This was the warm-up to the trial.
Although it was the team's initial instinct to extend the trial to the Queen's community, the team decided not to for two reasons. First, problems with the software during the practice period caused concern among team members. Second, the team worried that library users would lose faith in reference services and staff, particularly at a critical time near the end of term, if these problems surfaced during the trial. The trial was modified to include library staff only.
The trial of the virtual reference software took place from Monday, March 24, 2003 to Friday, March 28, 2003. Each day, two team members monitored the program from 3:00 - 4:00 p.m. An announcement was sent out to all library staff indicating that the ISI team was experimenting with the LSSI software and asking staff to help the team by submitting questions. During five hours of trial that week, sixty questions were sent via the software.
The trial itself revealed multiple software problems. One of the first problems involved the amount of time required to answer calls. Since the software provides a timer, librarians could clock the amount of time it took to answer a call. This often took six or seven minutes; however, at times it could take as long as ten minutes to answer a call. In some cases, the call could not be retrieved at all due to problems with the software or connection.
Another problem which surfaced involved multiple calls. Librarians who tried to answer two calls at the same time often found that one of those calls would be dropped before they had a chance to retrieve it. It seemed impossible to deal with two calls simultaneously which meant that users either ended their attempt because of the massive delay in having their calls answered, or they had their calls dropped before they could be dealt with in any kind of effective way.
Once calls were answered, it was apparent that one side (often the user) would be able to send messages quickly while the other side would be unable to respond in any timely fashion. Library users often ended the call before the answer to the question could be found because of the length of time responses were taking.
It was also impossible to push pages or pull up web sites in any timely way. Often the system slowed down entirely or calls were dropped. The system also seemed to indicate that librarians had logged out when they hadn't and, as a result, had them sign in multiple times. During one session, a team member had to sign on four times.
The final result was a pilot which was a dismal failure. No one (librarians or users) reported a successful session in which everything had worked smoothly and all questions had been answered in a satisfactory manner and in a timely fashion.
Attempts to solve these technical difficulties with the vendor were unsuccessful. LSSI representatives attributed the slow response time to high levels of traffic on the internet during a peak time of service. And while there is some merit to this comment - response was better in the mornings - the speed of response was unacceptable.
Unfortunately, technical difficulties prevented the team from experimenting with virtual reference in any meaningful way.
Until our position on virtual reference has been finalized, we recommend that QUL make more effective and efficient use of our current means of serving virtual patrons. For example, it is very difficult to locate contact numbers for our various reference desks and there are no highly visible "Ask-a-Librarian" links for each of our library home pages. The following recommendations could improve our already existing electronic reference services by increasing accessibility to library reference staff and still provide efficient turnaround time.
Phone reference offers the opportunity for simultaneous reference service to those who prefer immediate communication and response. Reference staff have noticed increasingly that patrons are now able to communicate by phone while simultaneously using computers. This provides the opportunity for over-the-phone, online instruction that allows both reference staff and patrons to access licensed databases - a service not currently or consistently available with virtual reference software.
Recommendation: Provide direct telephone lines to every library's reference desk, with voice mail to ensure replies.Currently, some of our reference desk lines either share a line with their unit's circulation desk or the reference line is an internal extension. In either of these cases, the patron must explain his/her question twice, creating an unnecessary block to our service and an added layer of patron frustration that presents Queen's Library's reference service in a poor light.
Recommendation: Offer 1-800 number for distance education users and outreach services.Bracken Library has piloted this service to its outreach users, and it has proven to be popular, successful and not prohibitively expensive. Such a service could also work to the advantage of distance education students who are reluctant to use our reference services because of long-distance charges.
Bracken Library has solved the problem of potential misuse of the long distance service by simply removing the phone from the desk when the desk is not staffed. It is also possible to restrict long distance service to certain hours on a phone line, with 24/7 voice mail availability.
Post reference desk phone numbers clearly from the library homepage (e.g. Phone a Reference Desk button which would link to a page listing all Queen's reference desk phone numbers).
This increases our visibility and accessibility to patrons. Even library staff often find it difficult to quickly locate the phone numbers of reference desks in other campus libraries. Library reference desk phone numbers could be publicized widely in other ways as well - e.g. bookmarks, residences, Queen's Journal.
While it makes sense to answer quick reference questions or to provide simultaneous instruction on the phone, an advantage of e-mail reference is that it allows time for thoughtful responses to complex reference questions. A second advantage is the existence of written documentation that leaves a paper trail for the patron to refer back to if necessary. E-mail responses could also include links to relevant web sites that may help answer the reference question.
Recommendation: Post an "Ask-a-librarian" link prominently on the library's homepage.This could lead to the page that now links to the reference desk associated with each discipline (http://library.queensu.ca/librequest/ref_index.htm). Increased visibility would lead to an increase in our current electronic reference statistics.
While ISI Team members found some merit in the concept of virtual reference, the Team was disappointed with the virtual reference software that was considered. From our collective experience, virtual reference was deemed a poor substitute for more immediate and gratifying reference exchanges offered by face-to-face reference, telephone reference and even e-mail reference. For this reason, the ISI Team has offered the above recommendations about enhancements to existing telephone and e-mail service.
Furthermore the Team characterized the software as too cumbersome to be useful at this time. Additional research and development by virtual reference vendors promise to offer improved software in the not too distant future. Enhancements such as voice-over-ip and video-streaming will mitigate against the inherent communication flaws associated with chat.
The ISI Team makes the following recommendations.
Recommendation: That QUL not embark upon virtual reference in the upcoming academic year with LSSI's Virtual Reference Toolkit.
Appendix One - Selected ResourcesWeb Sites
Duke University Libraries. Live On Line Reference page.
Available at http://www.ryerson.ca/library/ask/McConnell.pdf
Last Accessed May 6, 2003
Last Accessed May 6, 2003
Available at http://www.nlc-bnc.ca/vrc-rvc/s34-155-e.html#virtual
Last Accessed May 6, 2003.
Available at http://www.lis.uiuc.edu/~b-sloan/digiref.html
Last Accessed May 6, 2003.
Books and Reports
Maxwell, Nancy Kalikow (2002). Establish and maintaining live online reference service. Library Technology Reports, 38(4), 3-78. July/August 2002
Meola, Marc, and Stormont, Sam (2002). Starting and Operating Live Virtual Reference Services: A How-To-Do-It Manual for Librarians. New York: Neal-Schuman Press, 2002.
Ronan, Jana and Turner, Carol(2002) . Spec Kit 273 Chat Reference. Washington: ARL, 2002.
Comments from ISI Team Members
1) Brenda Reed
My biggest concern with the product was that it seemed to severely limit our ability to provide timely, efficient and accurate reference service -- such that the Queen's Library's Reference service would not be viewed by patrons as a worthwhile service. (And I would not want to be in that position!)
Why do I say this? Well, because the whole reason for a patron's choice of "Chat" reference over "Email" or "Phone" reference would presumably be based on an expected immediacy of service in an online environment. (Email would not provide the immediacy that might be desired and phone would not provide -- although this arguable -- the online environment).
My experience as both a patron and as a librarian tester of the VR Toolkit was that immediacy was not a feature of the product.
As the librarian, for example, I would consistently see that a patron was logged in, and I would "open" the session. That is, I would click to indicate that I wanted to open the session. My experience was that this "open" operation would take an average of 6 minutes to occur. This is completely unacceptable, and I would assume that during that 6-minute wait most patrons would decide that the service was not worth their time. As the librarian I was equally frustrated, as I knew that I could likely have emailed back and forth to the patron 10 times by the time the Chat opened, or have answered their question in a few seconds or minutes on the phone.
The next frustration was that web pages that were intended to be viewed by the patron at the same time as I was viewing them -- pages that I had "pushed" to them -- were often not viewable by the patron for several minutes after I was already seeing them as the librarian. This is not a good thing in itself -- but what is worse is that the librarian ends up not knowing what is on the patron's screen -- and may be unconsciously continuing on with instruction/answering -- to the total confusion of the patron because they cannot see on the screen the items that the librarian is referring to.
These are technical details, to be sure, and maybe in a better online situation than the "pilot" was conducted in these problems would not occur. Still, it does seem to me that this "Chat" service may be more suited to "quick-answer" questions of the facilitative or basic-type reference question, rather than as a means of instruction or of responding to complex reference questions. Complex reference questions are called complex for a reason -- and they cannot usually be answered "off the top of one's head" without a significant amount of conversation between the patron and the librarian. Thought -- and thus time -- is required for complex questions most of the time -- and the "Chat" environment is just not suited to this type of in-depth answer.
Furthermore, the inability to search Queen's Library's licensed databases with the patron severely limits the instructional capability of VR Toolkit. Since it is the mandate of Public Service Librarians to provide information literacy instruction as we answer patron reference questions, I think that it is not acceptable to purchase a product that does not allow instruction in the very products that we use to answer complex reference questions.
I think that the "trial" experience with VR Toolkit was valuable to me as a Public Services Librarian, though, because it made me realize that we currently are NOT making optimum use of the email and phone "electronic" reference services that we have available to us. Most distance education students now are able to both be on the phone and be sitting in front of their computers. This is the best of all alternatives, in a way, as both patron and librarian can be conducting a search at the same time -- and you can both be going in to the licensed databases -- and the librarian can be providing timely, helpful instruction -- while always insuring that the patron is on the same screen as the librarian.
I think that we should investigate how we can improve our current electronic reference services, and become more efficient promoters and users of current, highly-developed technologies such as the phone and email before moving on to a technology that is not completely functional yet.
2) Judy Nesbitt
Trial Monday March 24, 2003, 3-4pm: First day of trial, paired with Melody.
Started to sign on as librarian at 2:50 pm. After about 20 minutes, when the screen had not yet fully downloaded, it turned bright red! The following message appeared: "Inbox is unable to connect to server. Please log out and log in again. If problem persists, contact administrator." After rebooting, librarian was signed on twice. Library home page ever fully downloaded during the hour. No patrons attempted contact. The time it took to sign on was problematic++.
Trial Wednesday March 26, 2003, 3-4 pm: Third day of trial, paired with Melody.
Rebooted computer before beginning. VR downloaded to the point that a patron name could be seen. (The library home page had not yet downloaded) Communication was impossible with this patron. A message to reboot appeared. After rebooting, another patron name appeared, but it was impossible to communicate (?) The first patron, who was still waiting, (unknown to the librarians) received a "canned message saying someone would be with me in 2 minutes." This patron saw no live action in 40 minutes of waiting. Numerous patrons tried to connect (12?) but most (all?) were unsuccessful.
Conclusion: both attempts disastrous. Much too slow to download.
3) Ken Pearce
Judy's web page window took forever to load (as librarian) and I clicked on QCAT on my screen and that was what ended coming up on her screen. The message box worked fine. When I took my turn as librarian it worked much more quickly. However, when I was typing an answer to Judy's question, she clicked on her webpage and the cursor on my message box moved to the qcat screen. The end of my message got typed in the qcat search box. If you are not looking at the screen when you are typing, it may be a problem.
4) Lucinda Walls
Here are my comments on the inadequacies of this type of reference service which I see from the trial of the LSSI software:
People presumably choose to use "chat" because they have a sense of being with someone to talk in real time. Without visual or auditory clues, however, there seemed to be some crossover messages between librarian and patron while the former was trying to get the information typed and sent, and the latter was in a waiting mode and wondering what was going on at the other end. Even for fast typists, it may be difficult for the librarian to get complex information or instructions out fast enough for an expectant user. A question on library hours took several minutes to perform with this software, when a telephone call or click on a library web page would have only taken a matter of seconds, and the answer would not necessarily need to have come from a reference librarian. It also leaves the question whether this is a proper form of reference service for more time-consuming, in-depth questions.
My concerns would be that: 1) A Virtual reference desk would get reduced to questions that are more circulation or policy-oriented, rather than "true" reference, and hence, not a good use of reference staff time. 2) The limitations of using licensed databases would hinder a reference librarian trying to give in-depth subject help. 3) The frustration factor with library services would increase, due to slow/dropped responses.
5) Melody Burton
My greatest frustration with this trial is that the technical glitches prevented the team from actually testing virtual reference. Whether the problems rested with LSSI, internet connectivity or configuration problems is irrelevant. The team could not experiment with features of virtual reference because basic components of the software failed.
My personal opinion about virtual reference is more optimistic than our trial. I see virtual reference as an add-on medium for reference service. As such, it holds the potential to reach a segment of our population that has not, does not, will not use our traditional services. The capacity to serve remote users at their desktop is powerful and one that I hope future ISI Teams will investigate.
6) Nancy McCormack
Of all of the various ways in which Reference Services might be delivered--in person, via telephone, or via e-mail--I found this method to be the least satisfactory. Technical problems aside (and it really is hard to put them aside given the overwhelming number of technical problems), this is a medium which requires that reference be done in short, telegraphic-style sentences, and does not permit any type of detailed questioning or complex exchange with a patron. It is almost impossible to do a proper reference interview to determine what the patron is really looking for, or to engage in anything resembling a conversation with a patron. As a result, it seems impossible to answer any but the most basic reference questions in a satisfactory way (provided the technology itself can be coaxed to work). I can't see how this could do anything except frustrate and alienate patrons, and shine an unflattering light on the library itself.
7) Patricia Oakley
The potential to use virtual reference software as an additional means of communicating with users is exciting. All librarians have had questions from patrons where virtual reference software would have provided an excellent communication tool. Unfortunately the virtual software used in the ISI Team's trial did not meet the needs of the user trying to connect to the library nor the librarian trying to provide assistance. Problems that were experienced when I "worked the virtual reference desk" included:
I had two callers during the one hour test. In the first case, it took about 11 minutes to complete the loading of the program. I was able to reply through the "chat" section and the caller was able to see the record from the QCAT. However, when I tried to send a web page to her, the system stopped responding and the caller was not able to see anything about the webpage. She finally gave up although I was trying to provide explanation about what I was trying to do.
The second caller gave up and disconnected from the program after she waited for 5 minutes while the program was still loading itself.
I feel that the software might not be very reliable and with the traffic flow from the internet, the situation got even worse. It reached to the point (speed) that no one can bear with. Therefore, I feel that the time is not ready for the VR we are talking about, at least with this LSSI system.
July 2, 2003