-<!-- $Id: book.xml,v 1.9 2006-04-20 11:11:41 mike Exp $ -->
+<!-- $Id: book.xml,v 1.16 2006-04-24 12:50:24 marc Exp $ -->
<bookinfo>
<title>Metaproxy - User's Guide and Reference</title>
<author>
</author>
<copyright>
<year>2006</year>
- <holder>Index Data</holder>
+ <holder>Index Data ApS</holder>
</copyright>
<abstract>
<simpara>
Metaproxy is a universal router, proxy and encapsulated
metasearcher for information retrieval protocols. It accepts,
processes, interprets and redirects requests from IR clients using
- standard protocols such as ANSI/NISO Z39.50 (and in the future SRU
- and SRW), as well as functioning as a limited
- HTTP server. Metaproxy is configured by an XML file which
+ standard protocols such as
+ <ulink url="&url.z39.50;">ANSI/NISO Z39.50</ulink>
+ (and in the future <ulink url="&url.sru;">SRU</ulink>
+ and <ulink url="&url.srw;">SRW</ulink>), as
+ well as functioning as a limited
+ <ulink url="&url.http;">HTTP</ulink> server.
+ Metaproxy is configured by an XML file which
specifies how the software should function in terms of routes that
the request packets can take through the proxy, each step on a
route being an instantiation of a filter. Filters come in many
should not at this stage redistribute the code without explicit
written permission from the copyright holders, Index Data ApS.
</simpara>
+ <simpara>
+ <inlinemediaobject>
+ <imageobject>
+ <imagedata fileref="common/id.png" format="PNG"/>
+ </imageobject>
+ <imageobject>
+ <imagedata fileref="common/id.eps" format="EPS"/>
+ </imageobject>
+ </inlinemediaobject>
+ </simpara>
</abstract>
</bookinfo>
<title>Introduction</title>
- <para>
- <ulink url="http://www.indexdata.com/metaproxy/">Metaproxy</ulink>
- is a standalone program that acts as a universal router, proxy and
- encapsulated metasearcher for information retrieval protocols such
- as Z39.50, and in the future SRU and SRW. To clients, it acts as a
- server of these
- protocols: it can be searched, records can be retrieved from it,
- etc. To servers, it acts as a client: it searches in them,
- retrieves records from them, etc. it satisfies its clients'
- requests by transforming them, multiplexing them, forwarding them
- on to zero or more servers, merging the results, transforming
- them, and delivering them back to the client. In addition, it
- acts as a simple HTTP server; support for further protocols can be
- added in a modular fashion, through the creation of new filters.
- </para>
- <screen>
- Anything goes in!
- Anything goes out!
- Cold bananas, fish, pyjamas,
- Mutton, beef and trout!
+ <para>
+ <ulink url="&url.metaproxy;">Metaproxy</ulink>
+ is a standalone program that acts as a universal router, proxy and
+ encapsulated metasearcher for information retrieval protocols such
+ as <ulink url="&url.z39.50;">Z39.50</ulink>, and in the future
+ <ulink url="&url.sru;">SRU</ulink> and <ulink url="&url.srw;">SRW</ulink>.
+ To clients, it acts as a server of these protocols: it can be searched,
+ records can be retrieved from it, etc.
+ To servers, it acts as a client: it searches in them,
+ retrieves records from them, etc. it satisfies its clients'
+ requests by transforming them, multiplexing them, forwarding them
+ on to zero or more servers, merging the results, transforming
+ them, and delivering them back to the client. In addition, it
+ acts as a simple <ulink url="&url.http;">HTTP</ulink> server; support
+ for further protocols can be added in a modular fashion, through the
+ creation of new filters.
+ </para>
+ <screen>
+ Anything goes in!
+ Anything goes out!
+ Cold bananas, fish, pyjamas,
+ Mutton, beef and trout!
- attributed to Cole Porter.
- </screen>
- <para>
- Metaproxy is a more capable alternative to
- <ulink url="http://www.indexdata.com/yazproxy/">YAZ Proxy</ulink>,
- being more powerful, flexible, configurable and extensible. Among
- its many advantages over the older, more pedestrian work are
- support for multiplexing (encapsulated metasearching), routing by
- database name, authentication and authorisation and serving local
- files via HTTP. Equally significant, its modular architecture
- facilitites the creation of pluggable modules implementing further
- functionality.
- </para>
+ </screen>
+ <para>
+ Metaproxy is a more capable alternative to
+ <ulink url="&url.yazproxy;">YAZ Proxy</ulink>,
+ being more powerful, flexible, configurable and extensible. Among
+ its many advantages over the older, more pedestrian work are
+ support for multiplexing (encapsulated metasearching), routing by
+ database name, authentication and authorisation and serving local
+ files via HTTP. Equally significant, its modular architecture
+ facilitites the creation of pluggable modules implementing further
+ functionality.
+ </para>
+ <para>
+ This manual will briefly describe Metaproxy's licensing situation
+ before giving an overview of its architecture, then discussing the
+ key concept of a filter in some depth and giving an overview of
+ the various filter types, then discussing the configuration file
+ format. After this come several optional chapters which may be
+ freely skipped: a detailed discussion of virtual databases and
+ multi-database searching, some notes on writing extensions
+ (additional filter types) and a high-level description of the
+ source code. Finally comes the reference guide, which contains
+ instructions for invoking the <command>metaproxy</command>
+ program, and detailed information on each type of filter,
+ including examples.
+ </para>
</chapter>
<title>The Metaproxy Licence</title>
<para>
<emphasis role="strong">
- No decision has yet been made on the terms under which
- Metaproxy will be distributed.
+ No decision has yet been made on the terms under which
+ Metaproxy will be distributed.
</emphasis>
It is possible that, unlike
other Index Data products, metaproxy may not be released under a
</para>
</chapter>
+ <chapter id="installation">
+ <title>Installation</title>
+ <para>
+ Metaproxy depends on the folloing tools/libraries:
+ <variablelist>
+ <varlistentry><term><ulink url="&url.yazplusplus;">YAZ++</ulink></term>
+ <listitem>
+ <para>
+ This is a C++ library based on <ulink url="&url.yaz;">YAZ</ulink>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term><ulink url="&url.libxslt;">Libxslt</ulink></term>
+ <listitem>
+ <para>This is an XSLT processor - based on
+ <ulink url="&url.libxml2;">Libxml2</ulink>. Both Libxml2 and
+ Libxslt must be installed with the development components.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term><ulink url="&url.boost;">Boost</ulink></term>
+ <listitem>
+ <para>
+ The popular C++ library.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ In order to compile Metaproxy a modern C++ compiler is
+ required. Boost, in particular, requires the C++ compiler
+ to facilitate the newest features. Refer to Boost
+ <ulink url="&url.boost.compilers.status;">Compiler Status</ulink>
+ for more information.
+ </para>
+ <para>
+ We have succesfully used Metaproxy with Boost using the compilers
+ <ulink url="&url.gcc;">GCC</ulink> version 4.0 and
+ <ulink url="&url.vstudio;">Microsoft Visual Studio</ulink> 2003/2005.
+ </para>
+ <section id="installation.unix">
+ <title>Installation on Unix (from Source)</title>
+ <para>
+ Here is a quick step-by-step guide on how to compile all the
+ tools that Metaproxy uses. Only few systems have none of the required
+ tools binary packages. If, for example, Libxml2/libxslt are already
+ installed as development packages use those (and omit compilation).
+ </para>
+
+ <para>
+ Libxml2/libxslt:
+ </para>
+ <screen>
+ gunzip -c libxml2-version.tar.gz|tar xf -
+ cd libxml2-version
+ ./configure
+ make
+ su
+ make install
+ </screen>
+ <screen>
+ gunzip -c libxslt-version.tar.gz|tar xf -
+ cd libxslt-version
+ ./configure
+ make
+ su
+ make install
+ </screen>
+ <para>
+ YAZ/YAZ++:
+ </para>
+ <screen>
+ gunzip -c yaz-version.tar.gz|tar xf -
+ cd yaz-version
+ ./configure
+ make
+ su
+ make install
+ </screen>
+ <screen>
+ gunzip -c yazpp-version.tar.gz|tar xf -
+ cd yazpp-version
+ ./configure
+ make
+ su
+ make install
+ </screen>
+ <para>
+ Boost:
+ </para>
+ <screen>
+ gunzip -c boost-version.tar.gz|tar xf -
+ cd boost-version
+ ./configure
+ make
+ su
+ make install
+ </screen>
+ <para>
+ Metaproxy:
+ </para>
+ <screen>
+ gunzip -c metaproxy-version.tar.gz|tar xf -
+ cd metaproxy-version
+ ./configure
+ make
+ su
+ make install
+ </screen>
+ </section>
+ <section id="installation.debian">
+ <title>Installation on Debian</title>
+ <para>
+ ### To be written
+ </para>
+ </section>
+
+ <section id="installation.windows">
+ <title>Installation on Windows</title>
+ <para>
+ ### To be written
+ </para>
+ </section>
+ </chapter>
+
<chapter id="architecture">
<title>The Metaproxy Architecture</title>
<para>
</section>
- <section>
+ <section id="overview.filter.types">
<title>Overview of filter types</title>
<para>
We now briefly consider each of the types of filter supported by
</section>
- <section>
+ <section id="future.directions">
<title>Future directions</title>
<para>
Some other filters that do not yet exist, but which would be
</para>
</section>
- <section>
+ <section id="overview.xml.structure">
<title>Overview of XML structure</title>
<para>
All elements and attributes are in the namespace
</section>
- <section>
+ <section id="example.configuration">
<title>An example configuration</title>
<para>
The following is a small, but complete, Metaproxy configuration
<section>
<title>Introductory notes</title>
+ <warning>
+ <title>Lark's vomit</title>
+ <para>
+ This chapter goes into a level of technical detail that is
+ probably not necessary in order to configure and use Metaproxy.
+ It is provided only for those who like to know how things work.
+ You should feel free to skip on to the next section if this one
+ doesn't seem like fun.
+ </para>
+ </warning>
<para>
Two of Metaproxy's filters are concerned with multiple-database
operations. Of these, <literal>virt_db</literal> can work alone
while <literal>multi</literal> can work with the output of
<literal>virt_db</literal> to perform multicast searching, merging
the results into a unified result-set. The interaction between
- these two filters is necessarily complex, reflecting the real
- complexity of multicast searching in a protocol such as Z39.50
- that separates initialisation from searching, with the database to
- search known only during the latter operation.
+ these two filters is necessarily complex: it reflecting the real,
+ irreducible complexity of multicast searching in a protocol such
+ as Z39.50 that separates initialisation from searching, and in
+ which the database to be searched is not known at initialisation
+ time.
+ </para>
+ <para>
+ Hold on tight - this may get a little hairy.
+ </para>
+ <para>
+ In the general course of things, a Z39.50 Init request may carry
+ with it an otherInfo packet of type <literal>VAL_PROXY</literal>,
+ whose value indicates the address of a Z39.50 server to which the
+ ultimate connection is to be made. (This otherInfo packet is
+ supported by YAZ-based Z39.50 clients and servers, but has not yet
+ been ratified by the Maintenance Agency and so is not widely used
+ in non-Index Data software. We're working on it.)
+ The <literal>VAL_PROXY</literal> packet functions
+ analogously to the absoluteURI-style Request-URI used with the GET
+ method when a web browser asks a proxy to forward its request: see
+ the
+ <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2"
+ >Request-URI</ulink>
+ section of
+ <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616.html"
+ >the HTTP 1.1 specification</ulink>.
+ </para>
+ <para>
+ The role of the <literal>virt_db</literal> filter is to rewrite
+ this otherInfo packet dependent on the virtual database that the
+ client wants to search. For example, a <literal>virt_db</literal>
+ filter could be set up so that searches in the virtual database
+ ``lc'' are forwarded to the Library of Congress server, and
+ searches in the virtual database ``id'' are forwarded to the toy
+ GILS database that Index Data hosts for testing purposes. A
+ <literal>virt_db</literal> configuration to make this switch would
+ look like this:
+ </para>
+ <screen><![CDATA[
+ <filter type="virt_db">
+ <virtual>
+ <database>lc</database>
+ <target>z3950.loc.gov:7090/Voyager</target>
+ </virtual>
+ <virtual>
+ <database>id</database>
+ <target>indexdata.dk/gils</target>
+ </virtual>
+ </filter>]]></screen>
+ <para>
+ When Metaproxy receives a Z39.50 Init request from a client, it
+ doesn't immediately forward that request to the back-end server.
+ Why not? Because it doesn't know <emphasis>which</emphasis>
+ back-end server to forward it to until the client sends a search
+ request that specifies the database that it wants to search in.
+ Instead, it just treasures the Init request up in its heart; and,
+ later, the first time the client does a search on one of the
+ specified virtual databases, a connection is forged to the
+ appropriate server and the Init request is forwarded to it. If,
+ later in the session, the same client searches in a different
+ virtual database, then a connection is forged to the server that
+ hosts it, and the same cached Init request is forwarded there,
+ too.
</para>
<para>
- ### Much, much more to say!
+ All of this clever Init-delaying is done by the
+ <literal>frontend_net</literal> filter. The
+ <literal>virt_db</literal> filter knows nothing about it; in
+ fact, because the Init request that is received from the client
+ doesn't get forwarded until a Search reqeust is received, the
+ <literal>virt_db</literal> filter (and the
+ <literal>z3950_client</literal> filter behind it) doesn't even get
+ invoked at Init time. The <emphasis>only</emphasis> thing that a
+ <literal>virt_db</literal> filter ever does is rewrite the
+ <literal>VAL_PROXY</literal> otherInfo in the requests that pass
+ through it.
</para>
</section>
</chapter>
</para>
</section>
- <section>
+ <section id="individual.classes">
<title>Individual classes</title>
<para>
The classes making up the Metaproxy application are here listed by
</section>
- <section>
+ <section id="other.source.files">
<title>Other Source Files</title>
<para>
In addition to the Metaproxy source files that define the classes
&manref;
</section>
</chapter>
-
-
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "main.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- nxml-child-indent: 1
- End:
- -->
+<!-- Keep this Emacs mode comment at the end of the file
+Local variables:
+mode: nxml
+End:
+-->
\ No newline at end of file
projects / metaproxy-moved-to-github.git / blobdiff