INFORMATION SERVICES & INSTRUCTION FUNCTIONAL TEAM:

Virtual Reference at Queen's University Library:
An Investigative Report

Submitted by the ISI Team May 2003

Purpose of Report

The provision of a live online reference service was explicitly identified as a major initiative in the 2002 Queens Library Strategic Plan. A live online reference service would provide an additional method for the provision of reference services to Queen's faculty, students and staff. The ISI Team was asked to explore live online virtual reference and report its findings to the Management Team. This report summarizes the ISI Team's investigation as of April 30, 2003.

What is Live Virtual Reference?

Current methods of reference service at Queen's University Library range from the traditional method of face-to-face services, e-mail and telephone. Virtual reference does not replace current and traditional methods of reference service rather it capitalizes on another medium of delivery and communication - the World Wide Web.

There is some confusion as to what constitutes online virtual reference. The word live is included with virtual reference to differentiate this service from other methods of internet mediated reference services such as e-mail. E-mail (asynchronous) is currently used as a method of communication and reference service at Queens University Library. Unlike e-mail, live virtual reference (synchronous) allows patrons to receive information over the internet, in real time.

Characteristics of Live Online Virtual Reference

The rise of live online virtual reference at universities, public libraries and government libraries is a direct outgrowth of the dominance of the internet and parallels the commercial world's 24/7 marketing strategy. In fact, virtual reference software is derived from call-centre software with the explicit advantage of offering service around the clock from any location.

Within academia, the growth and success of the virtual library has created an environment for virtual reference. As collections become accessible remotely, live online virtual reference is a tool to insert a human service element into interactions between user and collection. Contrary to a utopian view of electronic resources, it is still necessary for staff to mediate some proportion of user transactions, even those completed offsite.

Library literature and academic listservs are drowning in testimonials of virtual reference at individual institutions and collaborative efforts among libraries who unite for geographical, logistical, subject or opportunistic reasons. Within Ontario, individual libraries are offering virtual reference (Windsor, Waterloo) and an OCUL pilot project among three institutions (Guelph, Ryerson and York) is underway. Across Canada, partnerships have sprung up opportunistically (UNB and Alberta). The National Library of Canada, seen as a pioneer for virtual reference on the international stage, is launching a national service and has invited libraries across Canada to participate in their initiative. Outside of Canada, there are examples of virtual reference collaborations based upon subject-expertise (engineering libraries, law libraries) and collaborations among libraries who serve a common "user" across a region (state government libraries, branches of a state-wide public library). All of these models offer benefits to their user groups.

While the exact features of virtual reference vary among software products and individual library configurations, the common concept is to provide a mechanism for library users to obtain assistance at their point of need. Virtual reference software also permits library staff to "push pages" or send information to the user that responds to their query. Chat software is the dominant feature that facilitates conversation between the library user and a library staff member. The ISI Team has summarized the benefits and concerns about virtual reference software. A summary of virtual reference characteristics is listed below.

Benefits To Users:

  • Matches expansion of the "virtual library" model. Synchronizes library reference services with online research tools offered to users.
  • Offers services to new library users who may not be able or willing to come to the library. For example distance students may not have the option of ever coming to a physical library when reference services are available or users may not want to leave their workstation to consult with a librarian.
  • Augments traditional library reference service. Library clientele who are not using or satisfied with traditional methods of reference service delivery may use and prefer this mode of delivery.
  • Addresses the diversity of the Queens community. The user population now consists of distance education students, community health care professionals as well as adjunct faculty at off campus locations. This trend is expected to continue and expand.
  • Eliminates the constraints of space and distance. Co-browsing virtual software facilitates mediated searches with the client no matter where that client is located.
  • Provides an additional teaching tool and forum particularly with the co-browsing capability.
  • Offers convenience to users. Eliminates need to tie up phone lines.
  • Provides options for ESL students, students with hearing problems and different learning or communication styles who may benefit from having a virtual reference option.
Benefits to the Library System:
  • Potential for collaborative or consortial library reference system.
  • Provides possibility of expanding 24/7 reference services or extending existing hours of reference service.
  • Learning a new technology - cool factor. Chat technology is an accepted mode of communication among younger or savvy students and faculty.
Concerns
  • Software in its infancy. Virtual reference software may not be robust enough to handle sophisticated reference questions or even deliver all of the features it promises such as co-browsing. Modifications from its call-centre origins may be insufficient to make virtual reference software an irresistible option. More research and development should deliver much more user- and staff-friendly virtual reference software.
  • Staffing considerations. Virtual reference is an add-on service that does not substitute for existing services. Additional staff resources must be allocated to offer a viable, successful virtual reference service.
  • Limitations to text-based chat. Although chat has a popular following, it is a poor substitute for face-to-face communication. Visual clues, voice intonation and body language are compelling components of human communication that are lost in a chat environment.
  • Staff training. All libraries report that they severely underestimated the amount of staff training and time required to offer virtual reference. Significant additional workload is associated with virtual reference.
  • Requires coordination if adopting a collaborative model or a model that is system-wide or cross-unit.
  • Subject specialization. To be viable, it takes a team to staff the virtual reference desk. In our current model of faculty libraries, expertise is segregated by discipline with no immediate capability to bridge all disciplines.
  • Confidentiality of staff and clients. Virtual reference captures the reference interaction and sends transcripts of the interaction to the library user, library staff member and virtual reference coordinator.
  • Software cost. There are initial costs for software purchase and training, and ongoing costs associated with licensing and maintenance.
  • Copyright and licensing considerations. The ability to co-browse licensed materials is essential to demonstrating library resources. There are significant restrictions associated with our licensing agreements that would restrict virtual reference to the Queen's community and require authentication.
  • Support or staff buy in. The creation of a virtual reference desk requires staff participation. This is essential for the success of the service. Staff need to understand and support the initiative, the software and the staff who will provide the service.
  • Systems support. Systems staff need to be able to provide local technical support and liaise with the vendor for maintenance and technical help.

Chronology of ISI Team Investigation of Virtual Reference

November 2001 Virtual reference is listed as one of the Team priorities for the ISI Team.
January 2002 The ISI Team begins reading library literature about virtual reference.
May 2002 The ISI Team organizes a webcast from ACRL that highlights VR for reference librarians as a way of competing with commercial information services.
August 2002 AUL Mary Mason requests that the ISI Team begin exploring the possibility of VR as a system wide initiative in keeping with QUL strategic plan.
October 2002 The ISI Team organizes a webcast demonstration of OCLC's virtual reference product - QuestionPoint.
October 2002 Melody Burton visits York University to learn about their experiences with VR.
November 2002 Melody Burton attends the Virtual Reference Desk Conference in Chicago.
December 2002 The ISI Team organizes and interested Systems staff attend a webcast demonstration of the LSSI's product Virtual Reference Toolkit.
February 2003 Librarians from Ryerson and York give a presentation to library staff on their experiences with LSSI software and the challenges involved with setting up a collaborative virtual reference service.
March 2003 The ISI Team experiments with LSSI software by conducting a trial among library staff.

Trial of LSSI's Virtual Reference Toolkit

The ISI Team had a webcast training of LSSI Virtual Reference Toolkit on March 10, 2003. In the days following training, team members practiced with the software and learned the basic steps to use this software. During the webcast training and in the practice sessions, some team members experienced loading problems and poor response time. This was the warm-up to the trial.

Although it was the team's initial instinct to extend the trial to the Queen's community, the team decided not to for two reasons. First, problems with the software during the practice period caused concern among team members. Second, the team worried that library users would lose faith in reference services and staff, particularly at a critical time near the end of term, if these problems surfaced during the trial. The trial was modified to include library staff only.

The trial of the virtual reference software took place from Monday, March 24, 2003 to Friday, March 28, 2003. Each day, two team members monitored the program from 3:00 - 4:00 p.m. An announcement was sent out to all library staff indicating that the ISI team was experimenting with the LSSI software and asking staff to help the team by submitting questions. During five hours of trial that week, sixty questions were sent via the software.

The trial itself revealed multiple software problems. One of the first problems involved the amount of time required to answer calls. Since the software provides a timer, librarians could clock the amount of time it took to answer a call. This often took six or seven minutes; however, at times it could take as long as ten minutes to answer a call. In some cases, the call could not be retrieved at all due to problems with the software or connection.

Another problem which surfaced involved multiple calls. Librarians who tried to answer two calls at the same time often found that one of those calls would be dropped before they had a chance to retrieve it. It seemed impossible to deal with two calls simultaneously which meant that users either ended their attempt because of the massive delay in having their calls answered, or they had their calls dropped before they could be dealt with in any kind of effective way.

Once calls were answered, it was apparent that one side (often the user) would be able to send messages quickly while the other side would be unable to respond in any timely fashion. Library users often ended the call before the answer to the question could be found because of the length of time responses were taking.

It was also impossible to push pages or pull up web sites in any timely way. Often the system slowed down entirely or calls were dropped. The system also seemed to indicate that librarians had logged out when they hadn't and, as a result, had them sign in multiple times. During one session, a team member had to sign on four times.

The final result was a pilot which was a dismal failure. No one (librarians or users) reported a successful session in which everything had worked smoothly and all questions had been answered in a satisfactory manner and in a timely fashion.

Attempts to solve these technical difficulties with the vendor were unsuccessful. LSSI representatives attributed the slow response time to high levels of traffic on the internet during a peak time of service. And while there is some merit to this comment - response was better in the mornings - the speed of response was unacceptable.

Unfortunately, technical difficulties prevented the team from experimenting with virtual reference in any meaningful way.

Recommendations and Virtual Reference Alternatives

Until our position on virtual reference has been finalized, we recommend that QUL make more effective and efficient use of our current means of serving virtual patrons. For example, it is very difficult to locate contact numbers for our various reference desks and there are no highly visible "Ask-a-Librarian" links for each of our library home pages. The following recommendations could improve our already existing electronic reference services by increasing accessibility to library reference staff and still provide efficient turnaround time.

Phone Reference

Phone reference offers the opportunity for simultaneous reference service to those who prefer immediate communication and response. Reference staff have noticed increasingly that patrons are now able to communicate by phone while simultaneously using computers. This provides the opportunity for over-the-phone, online instruction that allows both reference staff and patrons to access licensed databases - a service not currently or consistently available with virtual reference software.

Recommendation: Provide direct telephone lines to every library's reference desk, with voice mail to ensure replies.
Currently, some of our reference desk lines either share a line with their unit's circulation desk or the reference line is an internal extension. In either of these cases, the patron must explain his/her question twice, creating an unnecessary block to our service and an added layer of patron frustration that presents Queen's Library's reference service in a poor light.
Recommendation: Offer 1-800 number for distance education users and outreach services.
Bracken Library has piloted this service to its outreach users, and it has proven to be popular, successful and not prohibitively expensive. Such a service could also work to the advantage of distance education students who are reluctant to use our reference services because of long-distance charges.

Bracken Library has solved the problem of potential misuse of the long distance service by simply removing the phone from the desk when the desk is not staffed. It is also possible to restrict long distance service to certain hours on a phone line, with 24/7 voice mail availability.

Post reference desk phone numbers clearly from the library homepage (e.g. Phone a Reference Desk button which would link to a page listing all Queen's reference desk phone numbers).

This increases our visibility and accessibility to patrons. Even library staff often find it difficult to quickly locate the phone numbers of reference desks in other campus libraries. Library reference desk phone numbers could be publicized widely in other ways as well - e.g. bookmarks, residences, Queen's Journal.

E-Mail Reference

While it makes sense to answer quick reference questions or to provide simultaneous instruction on the phone, an advantage of e-mail reference is that it allows time for thoughtful responses to complex reference questions. A second advantage is the existence of written documentation that leaves a paper trail for the patron to refer back to if necessary. E-mail responses could also include links to relevant web sites that may help answer the reference question.

Recommendation: Post an "Ask-a-librarian" link prominently on the library's homepage.
This could lead to the page that now links to the reference desk associated with each discipline (http://library.queensu.ca/librequest/ref_index.htm). Increased visibility would lead to an increase in our current electronic reference statistics.

Conclusion and Recommendations

While ISI Team members found some merit in the concept of virtual reference, the Team was disappointed with the virtual reference software that was considered. From our collective experience, virtual reference was deemed a poor substitute for more immediate and gratifying reference exchanges offered by face-to-face reference, telephone reference and even e-mail reference. For this reason, the ISI Team has offered the above recommendations about enhancements to existing telephone and e-mail service.

Furthermore the Team characterized the software as too cumbersome to be useful at this time. Additional research and development by virtual reference vendors promise to offer improved software in the not too distant future. Enhancements such as voice-over-ip and video-streaming will mitigate against the inherent communication flaws associated with chat.

The ISI Team makes the following recommendations.

Recommendation: That QUL not embark upon virtual reference in the upcoming academic year with LSSI's Virtual Reference Toolkit.

Recommendation: That the succeeding ISI Team continue to investigate virtual reference software products in 2003 - 2004.

Recommendation: That the developments at the OCUL and National Library of Canada be closely monitored as they may provide insight into a preferred action for virtual reference at QUL.

Appendix One - Selected Resources

Web Sites

Duke University Libraries. Live On Line Reference page.
Available at http://www.lib.duke.edu/reference/liveonlineref.htm
Last Accessed May 6, 2003.

  • Includes virtual reference software comparison table.
Granfield Diane McConnell Library (2002). Funding Report Ryerson Report A Digital Reference Service for a Digital Library: Chat Technology in a Remote Reference Service.
Available at http://www.ryerson.ca/library/ask/McConnell.pdf
Last Accessed May 6, 2003
  • Report of a virtual library pilot project undertaken by McConnell Library (Ryerson University)
ISI Team, Queens University - Virtual Reference page. Available at http://library.queensu.ca/webisi/vr.htm
Last Accessed May 6, 2003 National Library of Canada. National Library of Canada Virtual Reference page.
Available at http://www.nlc-bnc.ca/vrc-rvc/s34-155-e.html#virtual
Last Accessed May 6, 2003.
  • Included on Canadian discussion lists, Canadian virtual reference services, as well as a general bibliography.
Sloan, B. (2003). DIGITAL REFERENCE SERVICES BIBLIOGRAPHY.
Available at http://www.lis.uiuc.edu/~b-sloan/digiref.html
Last Accessed May 6, 2003.
  • A 500 item bibliography of items related to the topic of online or virtual or digital reference services. Approximately 45% of the items listed in this bibliography are available via the Web. Links are included.

Books and Reports

Maxwell, Nancy Kalikow (2002). Establish and maintaining live online reference service. Library Technology Reports, 38(4), 3-78. July/August 2002

Meola, Marc, and Stormont, Sam (2002). Starting and Operating Live Virtual Reference Services: A How-To-Do-It Manual for Librarians. New York: Neal-Schuman Press, 2002.

Ronan, Jana and Turner, Carol(2002) . Spec Kit 273 Chat Reference. Washington: ARL, 2002.

Appendix Two

Comments from ISI Team Members

1) Brenda Reed

My biggest concern with the product was that it seemed to severely limit our ability to provide timely, efficient and accurate reference service -- such that the Queen's Library's Reference service would not be viewed by patrons as a worthwhile service. (And I would not want to be in that position!)

Why do I say this? Well, because the whole reason for a patron's choice of "Chat" reference over "Email" or "Phone" reference would presumably be based on an expected immediacy of service in an online environment. (Email would not provide the immediacy that might be desired and phone would not provide -- although this arguable -- the online environment).

My experience as both a patron and as a librarian tester of the VR Toolkit was that immediacy was not a feature of the product.

As the librarian, for example, I would consistently see that a patron was logged in, and I would "open" the session. That is, I would click to indicate that I wanted to open the session. My experience was that this "open" operation would take an average of 6 minutes to occur. This is completely unacceptable, and I would assume that during that 6-minute wait most patrons would decide that the service was not worth their time. As the librarian I was equally frustrated, as I knew that I could likely have emailed back and forth to the patron 10 times by the time the Chat opened, or have answered their question in a few seconds or minutes on the phone.

The next frustration was that web pages that were intended to be viewed by the patron at the same time as I was viewing them -- pages that I had "pushed" to them -- were often not viewable by the patron for several minutes after I was already seeing them as the librarian. This is not a good thing in itself -- but what is worse is that the librarian ends up not knowing what is on the patron's screen -- and may be unconsciously continuing on with instruction/answering -- to the total confusion of the patron because they cannot see on the screen the items that the librarian is referring to.

These are technical details, to be sure, and maybe in a better online situation than the "pilot" was conducted in these problems would not occur. Still, it does seem to me that this "Chat" service may be more suited to "quick-answer" questions of the facilitative or basic-type reference question, rather than as a means of instruction or of responding to complex reference questions. Complex reference questions are called complex for a reason -- and they cannot usually be answered "off the top of one's head" without a significant amount of conversation between the patron and the librarian. Thought -- and thus time -- is required for complex questions most of the time -- and the "Chat" environment is just not suited to this type of in-depth answer.

Furthermore, the inability to search Queen's Library's licensed databases with the patron severely limits the instructional capability of VR Toolkit. Since it is the mandate of Public Service Librarians to provide information literacy instruction as we answer patron reference questions, I think that it is not acceptable to purchase a product that does not allow instruction in the very products that we use to answer complex reference questions.

I think that the "trial" experience with VR Toolkit was valuable to me as a Public Services Librarian, though, because it made me realize that we currently are NOT making optimum use of the email and phone "electronic" reference services that we have available to us. Most distance education students now are able to both be on the phone and be sitting in front of their computers. This is the best of all alternatives, in a way, as both patron and librarian can be conducting a search at the same time -- and you can both be going in to the licensed databases -- and the librarian can be providing timely, helpful instruction -- while always insuring that the patron is on the same screen as the librarian.

I think that we should investigate how we can improve our current electronic reference services, and become more efficient promoters and users of current, highly-developed technologies such as the phone and email before moving on to a technology that is not completely functional yet.

2) Judy Nesbitt

Trial Monday March 24, 2003, 3-4pm: First day of trial, paired with Melody.

Started to sign on as librarian at 2:50 pm. After about 20 minutes, when the screen had not yet fully downloaded, it turned bright red! The following message appeared: "Inbox is unable to connect to server. Please log out and log in again. If problem persists, contact administrator." After rebooting, librarian was signed on twice. Library home page ever fully downloaded during the hour. No patrons attempted contact. The time it took to sign on was problematic++.

Trial Wednesday March 26, 2003, 3-4 pm: Third day of trial, paired with Melody.

Rebooted computer before beginning. VR downloaded to the point that a patron name could be seen. (The library home page had not yet downloaded) Communication was impossible with this patron. A message to reboot appeared. After rebooting, another patron name appeared, but it was impossible to communicate (?) The first patron, who was still waiting, (unknown to the librarians) received a "canned message saying someone would be with me in 2 minutes." This patron saw no live action in 40 minutes of waiting. Numerous patrons tried to connect (12?) but most (all?) were unsuccessful.

Conclusion: both attempts disastrous. Much too slow to download.

3) Ken Pearce

Judy's web page window took forever to load (as librarian) and I clicked on QCAT on my screen and that was what ended coming up on her screen. The message box worked fine. When I took my turn as librarian it worked much more quickly. However, when I was typing an answer to Judy's question, she clicked on her webpage and the cursor on my message box moved to the qcat screen. The end of my message got typed in the qcat search box. If you are not looking at the screen when you are typing, it may be a problem.

4) Lucinda Walls

Here are my comments on the inadequacies of this type of reference service which I see from the trial of the LSSI software:

People presumably choose to use "chat" because they have a sense of being with someone to talk in real time. Without visual or auditory clues, however, there seemed to be some crossover messages between librarian and patron while the former was trying to get the information typed and sent, and the latter was in a waiting mode and wondering what was going on at the other end. Even for fast typists, it may be difficult for the librarian to get complex information or instructions out fast enough for an expectant user. A question on library hours took several minutes to perform with this software, when a telephone call or click on a library web page would have only taken a matter of seconds, and the answer would not necessarily need to have come from a reference librarian. It also leaves the question whether this is a proper form of reference service for more time-consuming, in-depth questions.

My concerns would be that: 1) A Virtual reference desk would get reduced to questions that are more circulation or policy-oriented, rather than "true" reference, and hence, not a good use of reference staff time. 2) The limitations of using licensed databases would hinder a reference librarian trying to give in-depth subject help. 3) The frustration factor with library services would increase, due to slow/dropped responses.

5) Melody Burton

My greatest frustration with this trial is that the technical glitches prevented the team from actually testing virtual reference. Whether the problems rested with LSSI, internet connectivity or configuration problems is irrelevant. The team could not experiment with features of virtual reference because basic components of the software failed.

My personal opinion about virtual reference is more optimistic than our trial. I see virtual reference as an add-on medium for reference service. As such, it holds the potential to reach a segment of our population that has not, does not, will not use our traditional services. The capacity to serve remote users at their desktop is powerful and one that I hope future ISI Teams will investigate.

6) Nancy McCormack

Of all of the various ways in which Reference Services might be delivered--in person, via telephone, or via e-mail--I found this method to be the least satisfactory. Technical problems aside (and it really is hard to put them aside given the overwhelming number of technical problems), this is a medium which requires that reference be done in short, telegraphic-style sentences, and does not permit any type of detailed questioning or complex exchange with a patron. It is almost impossible to do a proper reference interview to determine what the patron is really looking for, or to engage in anything resembling a conversation with a patron. As a result, it seems impossible to answer any but the most basic reference questions in a satisfactory way (provided the technology itself can be coaxed to work). I can't see how this could do anything except frustrate and alienate patrons, and shine an unflattering light on the library itself.

7) Patricia Oakley

The potential to use virtual reference software as an additional means of communicating with users is exciting. All librarians have had questions from patrons where virtual reference software would have provided an excellent communication tool. Unfortunately the virtual software used in the ISI Team's trial did not meet the needs of the user trying to connect to the library nor the librarian trying to provide assistance. Problems that were experienced when I "worked the virtual reference desk" included:

  • Length of time to pick up a caller- Multiple callers slowed down the connect time considerably. Sometimes it was quick and easy to pick up a caller(s) at other times the initial contact was quickly established but then communication between the parties slowed to a crawl.
  • Losing callers in the middle of a query - which happened more often than expected. Software problems, connect problems or termination by the librarian or caller because of frustration with the software resulted in some queries being lost in virtual space.
  • Never being quite sure if the caller was seeing what you thought they were seeing. If the caller happened to be in the next room there was a strong desire to go and look at their computer screen or simply to communicate across the walls.
  • Length of time required to answer the question. Using a keyboard to answer a question can take time depending on keyboarding skills. The librarian not only had to track down the answer but also communicate this answer in such a way to avoid ambiguity at the caller's end. I was often thinking, as I watched the timer continue to count, that the question could be dealt with in much less time using more conventional means, such as the telephone.
  • I was never quite sure if the software was going to behave as expected. When it did there was a huge sigh of relief, when it didn't it was extremely frustrating.
8) Wenyan Wu

I had two callers during the one hour test. In the first case, it took about 11 minutes to complete the loading of the program. I was able to reply through the "chat" section and the caller was able to see the record from the QCAT. However, when I tried to send a web page to her, the system stopped responding and the caller was not able to see anything about the webpage. She finally gave up although I was trying to provide explanation about what I was trying to do.

The second caller gave up and disconnected from the program after she waited for 5 minutes while the program was still loading itself.

I feel that the software might not be very reliable and with the traffic flow from the internet, the situation got even worse. It reached to the point (speed) that no one can bear with. Therefore, I feel that the time is not ready for the VR we are talking about, at least with this LSSI system. July 2, 2003
 


ISI Index -- Last revised: 3 July, 2003 -- StaffWeb