1 <?xml version="1.0" standalone="no"?>
2 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1//EN"
3 "http://www.oasis-open.org/docbook/xml/4.1/docbookx.dtd"
5 <!ENTITY % local SYSTEM "local.ent">
7 <!ENTITY % entities SYSTEM "entities.ent">
9 <!ENTITY % common SYSTEM "common/common.ent">
12 <!-- $Id: book.xml,v 1.5 2007-01-19 18:28:08 quinn Exp $ -->
15 <title>Pazpar2 - User's Guide and Reference</title>
17 <firstname>Sebastian</firstname><surname>Hammer</surname>
20 <year>©right-year;</year>
21 <holder>Index Data</holder>
25 Pazpar2 is a high-performance, user interface-independent, data
26 model-independent metasearching
27 middleware featuring merging, relevance ranking, record sorting,
31 This document is a guide and reference to Pazpar version &version;.
36 <imagedata fileref="common/id.png" format="PNG"/>
39 <imagedata fileref="common/id.eps" format="EPS"/>
46 <chapter id="introduction">
47 <title>Introduction</title>
49 Pazpar2 is a stand-alone metasearch client with a webservice API, designed
50 to be used either from a browser-based client (JavaScript, Flash, Java,
51 etc.), from from server-side code, or any combination of the two.
52 Pazpar2 is a highly optimized client designed to
53 search many resources in parallel. It implements record merging,
54 relevance-ranking and sorting by arbitrary data content, and facet
55 analysis for browsing purposes. It is designed to be data model
56 independent, and is capable of working with MARC, DublinCore, or any
57 other XML-structured response format -- XSLT is used to normalize and extract
58 data from retrieval records for display and analysis. It can be used
59 against any server which supports the Z39.50 protocol. Proprietary
60 backend modules can be used to support a large number of other protocols
61 (please contact Index Data for further information about this).
64 Additional functionality such as
65 user management, attractive displays are expected to be implemented by
66 applications that use pazpar2. Pazpar2 is user interface independent.
67 Its functionality is exposed through a simple REST-style webservice API,
68 designed to be simple to use from an Ajax-enbled browser, Flash
69 animation, Java applet, etc., or from a higher-level server-side language
70 like PHP or Java. Because session information can be shared between
71 browser-based logic and your server-side scripting, there is tremendous
72 flexibility in how you implement your business logic on top of pazpar2.
75 Once you launch a search in pazpar2, the operation continues behind the
76 scenes. Pazpar2 connects to servers, carries out searches, and
77 retrieves, deduplicates, and stores results internally. Your application
78 code may periodically inquire about the status of an ongoing operation,
79 and ask to see records or other result set facets. Result become
80 available immediately, and it is easy to build end-user interfaces which
81 feel extremely responsive, even when searching more than 100 servers
85 Pazpar2 is designed to be highly configurable. Incoming records are
86 normalized to XML/UTF-8, and then further normalized using XSLT to a
87 simple internal representation that is suitable for analysis. By
88 providing XSLT stylesheets for different kinds of result records, you
89 can tune pazpar2 to work against different kinds of information
90 retrieval servers. Finally, metadata is extracted, in a configurable
91 way, from this internal record, to support display, merging, ranking,
92 result set facets, and sorting. Pazpar2 is not bound to a specific model
93 of metadata, such as DublinCore or MARC -- by providing the right
94 configuration, it can work with a number of different kinds of data in
95 support of many different applications.
98 Pazpar2 is designed to be efficient and scalable. You can set it up to
99 search several hundred targets in parallel, or you can use it to support
100 hundreds of concurrent users. It is implemented with the same attention
101 to performance and economy that we use in our indexing engines, so that
102 you can focus on building your application, without worrying about the
103 details of metasearch logic. You can devote all of your attention to
104 usability and let pazpar2 do what it does best -- metasearch.
107 If you wish to connect to commercial or other databases which do not
108 support open standards, please contact Index Data. We have a licensing
109 agreement with a third party vendor which will enable pazpar2 to access
110 thousands of online databases, in addition the vast number of catalogs
111 and online services that support the Z39.50 protocol.
114 Pazpar2 is our attempt to re-think the traditional paradigms for
115 implementing and deploying metasearch logic, with an uncompromising
116 approach to performance, and attempting to make maximum use of the
117 capabilities of modern browsers. The demo user interface that
118 accompanies the distribution is but one example. If you think of new
119 ways of using pazpar2, we hope you'll share them with us, and if we
120 can provide assistance with regards to training, design, programming,
121 integration with different backends, hosting, or support, please don't
122 hesitate to contact us. If you'd like to see functionality in pazpar2
123 that is not there today, please don't hesitate to contact us. It may
124 already be in our development pipeline, or there might be a
125 possibility for you to help out by sponsoring development time or
126 code. Either way, get in touch and we will give you straight answers.
134 <chapter id="license">
135 <title>Pazpar2 License</title>
136 <para>To be decided and written.</para>
139 <chapter id="installation">
140 <title>Installation</title>
142 Pazpar2 depends on the following tools/libraries:
144 <varlistentry><term><ulink url="&url.yaz;">YAZ</ulink></term>
147 The popular Z39.50 toolkit for the C language. YAZ must be
148 compiled with Libxml2/Libxslt support.
155 In order to compile Pazpar2 an ANSI C compiler is
156 required. The requirements should be the same as for YAZ.
159 <section id="installation.unix">
160 <title>Installation on Unix (from Source)</title>
162 Here is a quick step-by-step guide on how to compile the
163 tools that Pazpar2 uses. Only few systems have none of the required
164 tools binary packages. If, for example, Libxml2/libxslt are already
165 installed as development packages use these.
169 Ensure that the development libraries + header files are
170 available on your system before compiling Pazpar2. For installation
171 of YAZ, refer to the YAZ installation chapter.
174 gunzip -c pazpar2-version.tar.gz|tar xf -
183 <section id="installation.debian">
184 <title>Installation on Debian GNU/Linux</title>
186 All dependencies for Pazpar2 are available as
187 <ulink url="&url.debian;">Debian</ulink>
188 packages for the sarge (stable in 2005) and etch (testing in 2005)
192 The procedures for Debian based systems, such as
193 <ulink url="&url.ubuntu;">Ubuntu</ulink> is probably similar
196 apt-get install libyaz-dev
199 With these packages installed, the usual configure + make
200 procedure can be used for Pazpar2 as outlined in
201 <xref linkend="installation.unix"/>.
207 <title>Using pazpar2</title>
209 This chapter provides a general introduction to the use and deployment of pazpar2.
212 <section id="architecture">
213 <title>Pazpar2 and your systems architecture</title>
215 Pazpar2 is designed to provide asynchronous, behind-the-scenes
216 metasearching functionality to your application, exposing this
217 functionality using a simple webservice API that can be accessed
218 from any number of development environments. In particular, it is
219 possible to combine pazpar2 either with your server-side dynamic
220 website scripting, with scripting or code running in the browser, or
221 with any combination of the two. Pazpar2 is an excellent tool for
222 building advanced, Ajax-based user interfaces for metasearch
223 functionality, but it isn't a requirement -- you can choose to use
224 pazpar2 entirely as a backend to your regular server-side scripting.
225 When you do use pazpar2 in conjunction
226 with browser scripting (JavaScript/Ajax, Flash, applets, etc.), there are
227 special considerations.
231 Pazpar2 implements a simple but efficient HTTP server, and it is
232 designed to interact directly with scripting running in the browser
233 for the best possible performance, and to limit overhead when
234 several browser clients generate numerous webservice requests.
235 However, it is still desirable to use a conventional webserver,
236 such as Apache, to serve up graphics, HTML documents, and
237 server-side scripting. Because the security sandbox environment of
238 most browser-side programming environments only allows communication
239 with the server from which the enclosing HTML page or object
240 originated, pazpar2 is designed so that it can act as a transparent
241 proxy in front of an existing webserver (see <xref
242 linkend="pazpar2_conf"/> for details). In this mode, all regular
243 HTTP requests are transparently passed through to your webserver,
244 while pazpar2 only intercepts search-related webservice requests.
248 If you want to expose your combined service on port 80, you can
249 either run your regular webserver on a different port, a different
250 server, or a different IP address associated with the same server.
254 Sometimes, it may be necessary to implement functionality on your
255 regular webserver that makes use of search results, for example to
256 implement data import functionality, emailing results, history
257 lists, personal citation lists, interlibrary loan functionality
258 ,etc. Fortunately, it is simple to exchange information between
259 pazpar2, your browser scripting, and backend server-side scripting.
260 You can send a session ID and possibly a record ID from your browser
261 code to your server code, and from there use pazpar2s webservice API
262 to access result sets or individual records. You could even 'hide'
263 all of pazpar2s functionality between your own API implemented on
264 the server-side, and access that from the browser or elsewhere. The
265 possibilities are just about endless.
269 <section id="data_model">
270 <title>Your data model</title>
272 Pazpar2 does not have a preconceived model of what makes up a data
273 model. There are no assumption that records have specific fields or
274 that they are organized in any particular way. The only assumption
275 is that data comes packaged in a form that the software can work
276 with (presently, that means XML or MARC), and that you can provide
277 the necessary information to massage it into pazpar2's internal
282 Handling retrieval records in pazpar2 is a two-step process. First,
283 you decide which data elements of the source record you are
284 interested in, and you specify any desired massaging or combining of
285 elements using an XSLT stylesheet (MARC records are automatically
286 normalized to MARCXML before this step). If desired, you can run
287 multiple XSLT stylesheets in series to accomplish this, but the
288 output of the last one should be a representation of the record in a
289 schema that pazpar2 understands.
293 The intermediate, internal representation of the record looks like
296 <record xmlns="http://www.indexdata.com/pazpar2/1.0"
297 mergekey="title The Shining author King, Stephen">
299 <metadata type="title">The Shining</metadata>
301 <metadata type="author">King, Stephen</metadata>
303 <metadata type="kind">ebook</metadata>
305 <!-- ... and so on -->
309 As you can see, there isn't much to it. There are really only a few
310 important elements to this file.
314 Elements should belong to the namespace
315 http://www.indexdata.com/pazpar2/1.0. If the root node contains the
316 attribute 'mergekey', then every record that generates the same
317 merge key (normalized for case differences, white space, and
318 truncation) will be joined into a cluster. In other words, you
319 decide how records are merged. If you don't include a merge key,
320 records are never merged. The 'metadata' elements provide the meat
321 of the elements -- the content. the 'type' attribute is used to
322 match each element against processing rules that determine what
323 happens to the data element next.
327 The next processing step is the extraction of metadata from the
328 intermediate representation of the record. This is governed by the
329 'metadata' elements in the 'service' section of the configuration
330 file. See <xref linkend="config-server"/> for details. The metadata
331 in the retrieval record ultimately drives merging, sorting, ranking,
332 the extraction of browse facets, and display, all configurable.
336 <section id="client">
337 <title>Client development</title>
339 You can use pazpar2 from any environment that allows you to use
340 webservices. The initial goal of the software was to support
341 Ajax-based applications, but there literally are no limits to what
342 you can do. You can use pazpar2 from Javascript, Flash, Java, etc.,
343 on the browser side, and from any development environment on the
344 server side, and you can pass session tokens and record IDs freely
345 around between these environments to build sophisticated applications.
346 Use your imagination.
350 The webservice API of pazpar2 is described in detail in <xref
351 linkend="pazpar2_protocol"/>.
355 In brief, you use the 'init' command to create a session, a
356 temporary workspace which carries information about the current
357 search. You start a new search using the 'search' command. Once the
358 search has been started, you can follow its progress using the
359 'stat', 'bytarget', 'termlist', or 'show' commands. Detailed records
360 can be fetched using the 'record' command.
363 </chapter> <!-- Using pazpar2 -->
365 <reference id="reference">
366 <title>Reference</title>
369 The material in this chapter is drawn directly from the individual
377 <!-- Keep this comment at the end of the file
382 sgml-minimize-attributes:nil
383 sgml-always-quote-attributes:t
386 sgml-parent-document: nil
387 sgml-local-catalogs: nil
388 sgml-namecase-general:t