Friday, May 16, 2008

MARC Online

More news from LOC.
The Network Development and MARC Standards Office is pleased to announce that the Full versions of the all five MARC 21 formats are now available online, along with the Online Concise.
The "full" version of a format contains detailed descriptions of every data element, along with examples, input conventions, and history sections - all of the information from the printed formats. There are no textual differences between the Online Full and the printed documentation. The Concise still contains all of the elements and enough description to serve many lookup needs. Changes from the most recent update of the formats are indicated in the text of both the Online Concise and the Online Full.

Links in LC Records

News about 856 links from LOC.
I've received a couple of questions recently about the 856 links in LC records for the TOCs, descriptions, bios, sample texts, etc. and wanted to spread the word about what we did.

Every month, around the first of the month, folks run their link checkers to validate the links in their copies of LC records. The volume of traffic against our web server was tremendous. A couple of times it nearly brought the server down. We tried several things to minimize the impact if it looked like a link checker was running against the web server, but this didn't seem to help the problem. In the end, we moved all of the files that are in the 856 fields to a different, larger, more robust server. Apparently this is causing link checkers to report that there is a redirect and people are asking if they need to change the URL for the links. I would say that there is no need to change the 856 links from to In fact, I am still adding the URLs as

LC is committed to maintaining these URLs, you should not be experiencing access problems with them except when running link checkers or maybe harvesters. I appreciate any reports of wrong connections or other serious problems with the files. By my count, we have over 710,000 links in the LC catalog now, so you can see this is a major commitment for LC.

Wednesday, May 14, 2008

Manifestations and Near-Equivalents

Martha M. Yee continues to make her work readily available.
The two articles about 'manifestation' (the word everyone used to mean 'expression' until FRBR came along) that I published in 1994 are now available at the University of California eScholarship Repository, as follows:

Manifestations and Near-Equivalents: Theory, with Special Attention to Moving-Image Materials. Library Resources & Technical Services 1994; 38:227-256.

Manifestations and Near-Equivalents of Moving Image Works: a Research Project. Library Resources & Technical Services 1994; 38:355-372.

Re: Recommendation and Ranganathan

I hope everybody here is also reading Lorcan Dempsey's weblog. However, just in case there are some who don't, begin with the excellent post Recommendation and Ranganathan. I thought the description of the four types of metadata a very good place to start thinking and discussion.

Tuesday, May 13, 2008

eXtensible Text Framework (XTF)

The California Digital Library (CDL) is pleased to announce a new release of its search and display technology, the eXtensible Text Framework (XTF) version 2.1. XTF is an open source, highly flexible software application that supports the search, browse and display of heterogeneous digital content. XTF offers efficient and practical methods for creating customized end-user interfaces for distinct digital content collections.

Highlights from the 2.1 release include:
  • Extensive interface improvements, including new search forms, built-in faceted browsing, and a new look and feel.
  • Increased support for document and information exchange formats.
    • XHTML and OAI-PMH output
    • NLM article format indexing and output
    • Microsoft Word indexing
  • Streamlined XSLT stylesheets for simpler deployment and
  • Updated documentation that has been moved to the XTF project wiki, allowing XTF implementers to share solutions with entire user community.
  • "Freeform" Boolean query language, offered as an experimental feature.
  • Backward compatibility with existing XTF implementations.
A complete list of changes is available on the XTF Project page on SourceForge, where the distribution (including documentation) can also be downloaded.

Since the first deployment of XTF in 2005, the development strategy has been to build and maintain an indexing and display technology that is not only customizable, but also draws upon tested components already in use by the digital library and search communities - in particular the Lucene text search engine, Java, XML, and XSLT. By coordinating these pieces in a single platform that can be used to create multiple unique applications, CDL has succeeded in dramatically reducing the investment in infrastructure, staff training and development for new digital content projects.

XTF offers a suite of customizable features that support diverse intellectual access to content. Interfaces can be designed to support the distinct tools and presentations that are useful and meaningful to specific audiences. In addition, XTF offers the following core features:
  • Easy to deploy: Drops directly in to a Java application server such as Tomcat or Resin; has been tested on Solaris, Mac, Linux, and Windows operating systems.
  • Easy to configure: Can create indexes on any XML element or attribute; entire presentation layer is customizable via XSLT.
  • Robust: Optimized to perform well on large documents (e.g., a single text that exceeds 10MB of encoded text); scales to perform well on collections of millions of documents; provides full Unicode support.
  • Extensible:
    • Works well with a variety of authentication systems (e.g., IP address lists, LDAP, Shibboleth).
    • Provides an interface for external data lookups to support thesaurus-based term expansion, recommender systems, etc.
    • Can power other digital library services (e.g., XTF contains an OAI-PMH data provider that allows others to harvest metadata, and an SRU interface that exposes searches to federated search engines).
    • Can be deployed as separate, modular pieces of a third-party system (e.g., the module that displays snippets of matching text).
  • Powerful for the end user:
    • Spell checking of queries
    • Faceted displays for browsing
    • Dynamically updated browse lists
    • Session-based bookbags
These basic features can be tuned and modified. For instance, the same bookbag feature that allows users to store links to entire books, can also store links to citable elements of an object, such as a note or other reference.

XTF was actually used as an experimental OPAC technology at the CDL for an experiment with ranking and recommendation features with our catalog data.

Posted to many e-mail distribution lists.

Non-Latin Data in Name Authority Records

From LC:
As previously announced, MDS- Name Authority records will be enhanced with non-Latin script data in 4XX fields and selected notes beginning June 1, 2008, (see earlier announcements at and for additional information.) An additional FAQ related to the project will be posted at shortly.

An effort to automatically pre-populate existing authority records with non-Latin references by OCLC, Inc. will also begin in early June 2008. The initial rate of pre-population will be limited to several hundred records per week, and will grow to a rate of approximately 25,000 records per week. Note that other clean-up projects that have recently increased the volume of name authority records ( ) will be suspended during this pre-population effort. It is estimated that approximately 400,000 pre-population records will be distributed over a number of months.

CDS is making available a file of name authority test records containing non-Latin script data. The file of 110 test records can be found on the Library of Congress rs7 server under the /emds/test subdirectory with file names of names.nonlatintest.records for the MARC 8 version and names.nonlatintest.records.utf8 for the UTF8 version.


I've been blasted with comment spam. So I've had to turn on the comment moderation function.

It is a shame how these few folks can ruin things for all. A few years back a e-card was a fun thing to receive and send. now so many are spam, I've stopped sending and opening them. Open comments seem ready to go the same way.