[ONLINE]
technomonitor

What is Credible Information?

Donald T. Hawkins
InfoResources

ONLINE, September 1999
Copyright © 1999 Information Today, Inc.

The WWW-VL is an outstanding comprehensive resource on Web site quality ...
Validation of information obtained from the Internet seems to be a topic of considerable interest at the moment. Greg Notess [1] and Hal Kirkwood [2] wrote about it in ONLINE last year, and in the most recent issue, Rettig and LaGuardia [3] examined a variety of evaluation criteria that had been proposed by several authors and developed a "review canon" for Web sites. Coincidentally with their efforts, I became interested in the same subject. It was interesting (and comforting!) to find that the criteria I developed and those propounded by Rettig and LaGuardia have many similarities and drew on many of the same Web sites on the subject. However, there were also some significant differences, so I am devoting this TECHNOMONITOR column to them. The preceding articles and this column, taken together, should offer a comprehensive picture on the subject of judging Web sites and evaluating their information.

I approached the subject from the viewpoint that information users want credible information. The dictionary definition of "credible" is "believable," "plausible," or "worthy of confidence." The root word is from the Latin credere, meaning "to believe." "Plausible," a synonym, means "seemingly or apparently valid," and comes from the Latin plausibilis, and means "deserving applause."

"TRADITIONAL" VERSUS WEB PUBLISHING

Why should we be concerned with the quality of information obtained from the Web? To answer this question, it is instructive to consider how an authoritative or scholarly article typically makes its way from the author's word processor to the printed page.

Figure 1 shows the major steps in the process of producing an article for publication in a scholarly journal. The steps with a small hurdle near them represent opportunities for review, feedback to the author, and revision. The hurdle indicates that those steps are important milestones that must be passed before the process can continue. It is important to note that feedback to the author frequently results in significant revisions or changes to the document. The author might even decide not to proceed with further writing if, for example, the document has major flaws, the research upon which it is based is incomplete or faulty, or if it duplicates research done elsewhere.

The process illustrated here is rigorous, and it usually ensures that only high-quality articles are published. It is rare that obvious errors survive the extensive reviews that take place before an article appears in print. Major drawbacks to this process are the amount of time between completion of the research by the author and an article appearing in print (which is why preprint circulation has come into favor) and problems with the peer review process.

Now, contrast that process to publishing on the Web (Figure 2). It is immediately obvious that the Web publishing process is much shorter than the traditional publishing process, and there are no review hurdles to be overcome. As soon as the first version of the report is written, it can be posted on a Web site. Once there, it can remain indefinitely or be removed at the author's discretion. There is no quality control in the Web publishing process. (Of course, many organizations, for proprietary reasons, prohibit their employees from posting material on a public Web site, and others have restrictions and internal procedures that must be followed before posting.)

Because of the lack of a review process and the ease of posting, the question of credible information is much more critical when dealing with information taken from the Internet. Some people think the Internet is a wonderful repository of information on almost any subject. Others view it as a cultural garbage dump, containing everything found in society--from the best to the worst. Still others, because of the impressive technological capabilities of the Web (such as Java, etc.), view it as bad information in impressive packaging, leading to the widespread view that all information available on the Web is shallow, misleading, and not credible. (These opinions are not mutually exclusive. All of them may be true--one assumes at different times--depending the purpose for which one is using the Web.)

DECIDING WHAT INFORMATION IS CREDIBLE

With such a wide variety of information available, how does one decide if the information retrieved from the Web is credible? Rettig and LaGuardia incorporate several excellent criteria into their "canon," and I will not repeat them here. I suggest that the following criteria might be added to the "canon:"

CREDIBILITY CRITERIA RANKING

I decided to rank-order the credibility criteria listed by 14 sites on information quality (taken from the Evaluation of Information Resources page of the WWW-VL) to see if some could be considered more important than others. I tabulated the criteria that their authors considered important in evaluating Web-based information and then grouped similar criteria together. The results are shown in Figure 4.

It is striking to note that the criteria deemed important by the Web site authors fall into four distinct groups. The three criteria occurring most often were mentioned by ten or more of the authors, the next group were mentioned by five or six authors, the next by two or three authors, and the last group were mentioned by only one author. Although these data are based on only 14 Web sites, they can be used to construct a rank-ordered list of criteria for deciding whether a Web site contains credible information. The Table at left presents such a list, along with some questions that one might ask to decide whether the information obtained from a Web site is credible.

CONCLUSIONS

This brief study has outlined some criteria for deciding whether information obtained from Web sites is credible. It is important to recognize that information does not gain or lose credibility simply by virtue of its format (print or electronic). Therefore, one should use the same criteria to evaluate electronic information that would be used for print information. However, because of the ephemeral, dynamic, and fluid nature of the Web, and the lack of a review process, one must be much more cautious when evaluating information obtained from it than when evaluating information obtained from a peer-reviewed or scholarly journal.

Additional or more stringent evaluation criteria should be used for Web-based sources than for print sources. Methods of evaluating print sources have evolved over many years and have stood the test of time. The Web is at most five or six years old, and it is constantly evolving. Information taken from it may not be of the same quality as information from a traditional print source. A popular misconception today is "If it is on the Internet, it must be (choose one) true, reliable, valid, correct." Those who hold this view may be in for a rude shock. The well-known adage, caveat emptor (Let the buyer beware) can be well revised for Web-based information as caveat lector (Let the reader beware)!


Web Site Evaluation Criteria and Testing Questions

Group 1
Currency/Updating
  • How current is the site?
  • How often is it updated?
  • Purpose/Audience
  • What is the purpose of the site (Why is it there)?
  • Who is the audience?
  • Is the purpose stated?
  • Bias/Objectivity
  • Is the information objective?
  • Are there obvious biases?
  • Is the author on a crusade?
  • Author/Publisher
  • Does the author's name appear on the site?
  • Who is the author?
  • Is the author well known?
  • Do you personally know the author or his/her work?
  • How well qualified is the author?
  • Is there an email address on the site for contacting him/her?
  • Who is the publisher?
  • Is the publisher a reputable organization?
  • Group 2
    Scope
  • What is the scope of the site (breadth, depth, etc.)?
  • Accuracy/Relevance
  • Is it possible to determine the accuracy or relevance of the information?
  • Design/Format
  • What is the design and format of the site?
  • Is it just a lot of fancy graphics with little information behind them?
  • Authority
  • Is the information authoritative?
  • Who maintains the site?
  • Are they an authority in their field?
  • Group 3
    Uniqueness/Stability
  • How long has the site existed?
  • Has it remained stable during that time?
  • Is the information it presents unique?
  • Structure/Indexing
  • What is the structure of the site?
  • Does it have a search engine associated with it?
  • Review/Ratings
  • Has the site been reviewed?
  • Has it received any ratings?
  • Group 4
    Writing & Data Quality
  • What is the quality of the writing and data on the site?
  • Selection Criteria
  • How is the information on the site selected?
  • Links
  • Are there links to/from other sites?
  • What is the quality of the sites linked to/from the site?

  • Reading List

    Articles

    Anagnostelis, Betsy; Cooke, Alison; and McNab, Alison, "Thinking Critically About Information on the Web", Vine 104, p21-28 (1997). Available at http://www.omni.ac.uk/agec/vine.html

    Ballard, Terry, "Internet Reference: Just the Good Stuff", Information Today, p51 (December 1997)

    Cyberskeptics Guide to Internet Research, Bibliodata, Needham Heights, MA. Published monthly. For further information, see http://www.bibliodata.com

    Garvin, Andrew P., with Robert Berkman and Hubert Bermont, The Art of Being Well Informed: What You Need to Know to Gain the Winning Edge in Business, Avery Publishing Group, Inc., Garden City Park, NY, (1993). (Contains chapter on evaluating information resources)

    Smith, Alastair G., "Testing the Surf: Criteria for Evaluating Internet Information Resources", Public-Access Computer Systems Review 8(3) (1997). Available at .

    Web Sites

    Alves, Susanna, "World Wide Web Site Evaluation for Information Professionals"
    http://www.bf.rmit.edu.au/Dimals/rguides/website.htm

    American Library Association, "How to Tell if You are Looking at a Great Web Site"
    http://www.ala.org/parentspage/greatsites/criteria.html

    Auer, Nicole, "Bibliography on Evaluating Internet Resources"
    http://refserver.lib.vt.edu/libinst/critTHINK.HTM

    Beck, Susan E., "The Good, The Bad & The Ugly: or, Why it's a Good Idea to Evaluate Web Sources. Evaluation Criteria"
    http://lib.nmsu.edu/staff/susabeck/evalcrit.html

    "Bookmark Tutorial: Evaluating Internet Resources"
    http://library.berkeley.edu:8000/bkmk/select.html

    Brandt, D. Scott, "Evaluating Information on the Internet"
    http://thorplus.lib.purdue.edu/~techman/evaluate.htm

    Clement, Gail P., "Pharmaceutical Resources on the Internet: Criteria for Assessing Quality and Value"
    http://www.fiu.edu/~clementg/medical/slaqv.html

    Edwards, Jana, "Tips for Evaluating a World Wide Web Search"

    Grassian, Esther, "Thinking Critically About World Wide Web Resources"
    http://www.library.ucla.edu/libraries/college/instruct/web/critical.htm

    Kirk, Elizabeth E., "Evaluating Information Found on the Internet"
    http://milton.mse.jhu.edu:8001/research/education/net.html

    Kwan, Julie, "Criteria for Evaluating Information Resources"
    http://www-lib.usc.edu/Info/Sci/pubs/criteval.html

    Patterson, Shawn, "Evaluating and Citing Internet Resources"
    http://www.udmercy.edu/htmls/Academics/library/WEBPAGE

    Scholz-Crane, Ann, "Evaluating World Wide Web Information"
    http://crab.rutgers.edu/~scholzcr/eval.html

    Smith, Alastair, "Criteria for Evaluation of Internet Information Resources"
    http://www.vuw.ac.nz/~agsmith/evaln/evaln.htm

    Smith, Alastair G., "Evaluation of Information Resources"
    http://www.vuw.ac.nz/~agsmith/evaln/index.htm

    Terrass, Richard, "Evaluating Internet Resources"
    http://web.wn.net/~usr/ricter/web/valid.html

    Tillman, Hope N., "Evaluating Quality on the Net"
    http://www.tiac.net/users/hope/findqual.html

    Tyburski, Genie, "Information Quality: A Checklist for Determining the Value of Information Retrieved from the Internet"
    http://www.virtualchase.com/quality.shtml

    Tyburski, Genie, "Publishers Wanted, No Experience Necessary: Information Quality on the Web"
    http://www.llrx.com/columns/quality.htm

    Wittman, Sandra, "Evaluating Web Sites" (Evaluation Criteria) http://www.oakton.edu/~wittman/find/eval.htm

    World Wide Virtual Library: Information Quality
    http://www.ciolek.com/WWWVL-InfoQuality.html


    REFERENCES

    [1] Notess, Greg R. "Tips for Evaluating Web Databases." DATABASE 21, No. 2 (April-May 1998): pp. 69-72.

    [2] Kirkwood, Hal. "Beyond Evaluation: A Model for Cooperative Evaluation of Internet Resources." ONLINE 22, No. 4 (July/August 1998): pp. 66-72.

    [3] Rettig, James, and LaGuardia, Cheryl. "Beyond 'Beyond Cool': Reviewing Web Resources." ONLINE 23, No. 4 (July/August 1999): pp. 51-55.


    Donald T. Hawkins (D.T.Hawkins@worldnet.att.net) is the Editor-in-Chief of Information Science Abstracts and Fulltext Sources Online, and the Editor of the ASIDIC Newsletter.

    Comments? Email letters to the Editor at editor@infotoday.com.

    [infotoday.com] [ONLINE] [Current Issue] [Subscriptions] [Top]

    Copyright © 1999, Information Today, Inc. All rights reserved.
    Comments