idzebra-moved-to-github.git
17 years agoAdded support for specification of staticrank for grs-class of filters.
Adam Dickmeiss [Fri, 22 Dec 2006 13:57:25 +0000 (13:57 +0000)]
Added support for specification of staticrank for grs-class of filters.
This is enabled by using 'staticrank indexname' in .abs. Contents of
indexname (elm ... indexname:w) is then used as value for staticrank.

17 years agoUsing atoizint for staticrank
Adam Dickmeiss [Fri, 22 Dec 2006 13:57:03 +0000 (13:57 +0000)]
Using atoizint for staticrank

17 years agoAdded function atoizint with a atoi/atoll wrapper.
Adam Dickmeiss [Fri, 22 Dec 2006 13:48:20 +0000 (13:48 +0000)]
Added function atoizint with a atoi/atoll wrapper.

17 years agoAdd timing for zebra_start/zebra_stop.
Adam Dickmeiss [Fri, 22 Dec 2006 12:14:25 +0000 (12:14 +0000)]
Add timing for zebra_start/zebra_stop.

17 years agoUse atoll/atoi for ascii to zint conversion in alvis filter.
Adam Dickmeiss [Fri, 22 Dec 2006 12:13:39 +0000 (12:13 +0000)]
Use atoll/atoi for ascii to zint conversion in alvis filter.

17 years agoDo not warn for truncmax limit
Adam Dickmeiss [Wed, 20 Dec 2006 14:29:04 +0000 (14:29 +0000)]
Do not warn for truncmax limit

17 years agoDo not truncate more than at most 'truncmax' terms in one single
Adam Dickmeiss [Wed, 20 Dec 2006 14:19:21 +0000 (14:19 +0000)]
Do not truncate more than at most 'truncmax' terms in one single
term with right-truncation/regular attribute. By default, 'truncmax',
is 10000.

17 years agoFix reference to uninitialized var (bug introduced in previous commit).
Adam Dickmeiss [Wed, 20 Dec 2006 08:37:49 +0000 (08:37 +0000)]
Fix reference to uninitialized var (bug introduced in previous commit).

17 years agoUsing zebra::index::field:s rather than zebra::sort::field for retrieval
Adam Dickmeiss [Tue, 19 Dec 2006 16:57:38 +0000 (16:57 +0000)]
Using zebra::index::field:s rather than zebra::sort::field for retrieval
of sort keys.

17 years agoFix bad ptr in yaz_log debug msg
Adam Dickmeiss [Tue, 19 Dec 2006 13:55:05 +0000 (13:55 +0000)]
Fix bad ptr in yaz_log debug msg

17 years agoAvoid file sync for read-only files.
Adam Dickmeiss [Tue, 19 Dec 2006 13:17:34 +0000 (13:17 +0000)]
Avoid file sync for read-only files.

17 years agoChanged assert in isamb, since we have larger keys
Adam Dickmeiss [Tue, 19 Dec 2006 00:25:41 +0000 (00:25 +0000)]
Changed assert in isamb, since we have larger keys

17 years agoImplemented sorting via the ISAMB system. To enable, use sortindex:b
Adam Dickmeiss [Mon, 18 Dec 2006 23:40:06 +0000 (23:40 +0000)]
Implemented sorting via the ISAMB system. To enable, use sortindex:b
in zebra.cfg. The sort keys can also be fetched using zebra::sort:field .

17 years agoDisable check: bug #730
Adam Dickmeiss [Sat, 16 Dec 2006 09:04:28 +0000 (09:04 +0000)]
Disable check: bug #730

17 years agoDefine YAZ_HAVE_XML2
Adam Dickmeiss [Fri, 15 Dec 2006 21:11:37 +0000 (21:11 +0000)]
Define YAZ_HAVE_XML2

17 years agoFixed bug #767: Zebra indexing stylesheet fails to work. The problem
Adam Dickmeiss [Wed, 13 Dec 2006 13:05:45 +0000 (13:05 +0000)]
Fixed bug #767: Zebra indexing stylesheet fails to work. The problem
was that the root node was not of type element and split_level = 0 (deal with
whole record / no split). We make split level 0 a special case (again) and
use extract_full for this.

17 years agoMore digits in output for total docs
Adam Dickmeiss [Tue, 12 Dec 2006 17:33:35 +0000 (17:33 +0000)]
More digits in output for total docs

17 years agoRename isb to isb_postings.
Adam Dickmeiss [Tue, 12 Dec 2006 15:23:45 +0000 (15:23 +0000)]
Rename isb to isb_postings.

17 years agoPossible fix of two bugs (1) bug #759: Crash during retrieval of some
Adam Dickmeiss [Tue, 12 Dec 2006 15:22:26 +0000 (15:22 +0000)]
Possible fix of two bugs (1) bug #759: Crash during retrieval of some
records (2) bug #765.

17 years agoAdded gnuplot scripts for dictisam timings, filesystems comparisons.
Adam Dickmeiss [Tue, 12 Dec 2006 13:54:33 +0000 (13:54 +0000)]
Added gnuplot scripts for dictisam timings, filesystems comparisons.

17 years agoAll output to stdout for easy feet into gnuplot. The following is now
Adam Dickmeiss [Tue, 12 Dec 2006 13:51:23 +0000 (13:51 +0000)]
All output to stdout for easy feet into gnuplot. The following is now
logged: user,real,sys for dict+isam, postings, documents, distinct
words, new words, dict splits, isamb leaf splits, isamb interior splits.

17 years agoAdded functions isamb_set_int_count, isamb-set_cache_size,
Adam Dickmeiss [Tue, 12 Dec 2006 13:46:41 +0000 (13:46 +0000)]
Added functions isamb_set_int_count, isamb-set_cache_size,
isamb_get_int_splits, isamb_get_leaf_splits, iasmb_set_int_count.

17 years agoAdded statistics functions dict_get_no_{lookup,insert,split}.
Adam Dickmeiss [Tue, 12 Dec 2006 13:42:23 +0000 (13:42 +0000)]
Added statistics functions dict_get_no_{lookup,insert,split}.

17 years agoRemove unused variable (dst_buf)
Adam Dickmeiss [Mon, 11 Dec 2006 17:08:03 +0000 (17:08 +0000)]
Remove unused variable (dst_buf)

17 years agoAllow MARC line format to be read.
Adam Dickmeiss [Mon, 11 Dec 2006 15:08:55 +0000 (15:08 +0000)]
Allow MARC line format to be read.

17 years agoUsing timing utility
Adam Dickmeiss [Mon, 11 Dec 2006 10:02:14 +0000 (10:02 +0000)]
Using timing utility

17 years agoAdded a utility for doing timings.
Adam Dickmeiss [Mon, 11 Dec 2006 09:50:36 +0000 (09:50 +0000)]
Added a utility for doing timings.

17 years agoMore appropriate usage
Adam Dickmeiss [Sun, 10 Dec 2006 21:02:28 +0000 (21:02 +0000)]
More appropriate usage

17 years agoMake MARC indexer with different ISAM strategy.
Adam Dickmeiss [Sun, 10 Dec 2006 21:00:56 +0000 (21:00 +0000)]
Make MARC indexer with different ISAM strategy.

17 years agoUse rand() twice on platforms where RAND_MAX < 2^16.
Adam Dickmeiss [Sun, 10 Dec 2006 20:59:52 +0000 (20:59 +0000)]
Use rand() twice on platforms where RAND_MAX < 2^16.

17 years agoAdded new test which makes variations to ISAM entry size.
Adam Dickmeiss [Sun, 10 Dec 2006 11:54:29 +0000 (11:54 +0000)]
Added new test which makes variations to ISAM entry size.

17 years agoProduce .dat files with bench1. prefix
Adam Dickmeiss [Sun, 10 Dec 2006 11:54:11 +0000 (11:54 +0000)]
Produce .dat files with bench1. prefix

17 years agoAllow size of ISAM entry to be specified in bench test
Adam Dickmeiss [Sun, 10 Dec 2006 11:49:16 +0000 (11:49 +0000)]
Allow size of ISAM entry to be specified in bench test

17 years agoAdded some performance test utilities.
Adam Dickmeiss [Sat, 9 Dec 2006 08:03:57 +0000 (08:03 +0000)]
Added some performance test utilities.

17 years agoAdded some ISAMB benchmark code
Adam Dickmeiss [Thu, 7 Dec 2006 21:13:56 +0000 (21:13 +0000)]
Added some ISAMB benchmark code

17 years agoAllow different values for cache parameter for isamb_open: 0=normal,
Adam Dickmeiss [Thu, 7 Dec 2006 19:23:56 +0000 (19:23 +0000)]
Allow different values for cache parameter for isamb_open:  0=normal,
1=caching of pages, -1 testing (NULL ISAM).

17 years agoRemove @XSLT_LIBS@ from idzebra-config --libs / --lalibs output. XSLT
Adam Dickmeiss [Thu, 7 Dec 2006 10:01:32 +0000 (10:01 +0000)]
Remove @XSLT_LIBS@ from idzebra-config --libs / --lalibs output. XSLT
libs is part of @YAZLIB@ already.

17 years agoUse zint for numbers in update statistics.
Adam Dickmeiss [Wed, 6 Dec 2006 22:52:38 +0000 (22:52 +0000)]
Use zint for numbers in update statistics.

17 years agoAdded new object check_res.
Adam Dickmeiss [Wed, 6 Dec 2006 13:00:12 +0000 (13:00 +0000)]
Added new object check_res.

17 years agoAdded more to list of options.
Adam Dickmeiss [Wed, 6 Dec 2006 10:26:54 +0000 (10:26 +0000)]
Added more to list of options.

17 years agoDeal with a picky ambiguity with configs like a.b where both suffix and
Adam Dickmeiss [Wed, 6 Dec 2006 10:26:40 +0000 (10:26 +0000)]
Deal with a picky ambiguity with configs like a.b where both suffix and
prefix are allowed. Both a and b must be checked.

17 years agoFixed bug #445: zebra.cfg does not check for valid directives.
Adam Dickmeiss [Tue, 5 Dec 2006 14:06:29 +0000 (14:06 +0000)]
Fixed bug #445: zebra.cfg does not check for valid directives.

17 years agoProvide defaults for profilePath and modulePath for the zebra{srv,idx}.
Adam Dickmeiss [Tue, 5 Dec 2006 09:26:37 +0000 (09:26 +0000)]
Provide defaults for profilePath and modulePath for the zebra{srv,idx}.

17 years agoReformat a bit.
Adam Dickmeiss [Tue, 5 Dec 2006 09:26:04 +0000 (09:26 +0000)]
Reformat a bit.

17 years agoDeal with NULL profile_path.
Adam Dickmeiss [Tue, 5 Dec 2006 09:24:47 +0000 (09:24 +0000)]
Deal with NULL profile_path.

17 years agoFor Res(ources), maintain reference count (poor man's smart ptr).
Adam Dickmeiss [Tue, 5 Dec 2006 09:24:31 +0000 (09:24 +0000)]
For Res(ources), maintain reference count (poor man's smart ptr).

17 years agoExperiment with alternative reckeys store.
Adam Dickmeiss [Tue, 5 Dec 2006 08:14:59 +0000 (08:14 +0000)]
Experiment with alternative reckeys store.

17 years agoOnly add first in field marker if there field has term(s) in it
Adam Dickmeiss [Tue, 5 Dec 2006 08:14:47 +0000 (08:14 +0000)]
Only add first in field marker if there field has term(s) in it

17 years agoFixed bug #739: Extended service update fails with Alvis filter.
Adam Dickmeiss [Sun, 3 Dec 2006 16:05:13 +0000 (16:05 +0000)]
Fixed bug #739: Extended service update fails with Alvis filter.
This change makes it possible to have register and shadow as same
directory. The size limits is only counted for each file type (so if
shadow is 10M and register is 20M the total size of directory is 30M).

17 years agoRemoved unused var
Adam Dickmeiss [Sun, 3 Dec 2006 15:55:02 +0000 (15:55 +0000)]
Removed unused var

17 years agoRemoved a few heavily used yaz_log calls
Adam Dickmeiss [Thu, 30 Nov 2006 11:03:57 +0000 (11:03 +0000)]
Removed a few heavily used yaz_log calls

17 years agoAdded support for specification of approximative limits for whole query.
Adam Dickmeiss [Thu, 30 Nov 2006 10:33:18 +0000 (10:33 +0000)]
Added support for specification of approximative limits for whole query.
This is specified as attribute type 12. Semantics is the same as
estimatehits in zebra.cfg.

17 years agoChange IDs for the special retrieval stuff.
Adam Dickmeiss [Thu, 30 Nov 2006 10:29:23 +0000 (10:29 +0000)]
Change IDs for the special retrieval stuff.

17 years agoDisable xelm path ! case code in case of index_name (att in 1.3) being NULL.
Adam Dickmeiss [Wed, 29 Nov 2006 18:06:57 +0000 (18:06 +0000)]
Disable xelm path ! case code in case of index_name (att in 1.3) being NULL.
It's not documented and not in use for Zebra 2. The .abs reader can
never generate it so it's dead code which just slows indexing down.

17 years agoCut memory for sorting in half when threaded sort is in effect.
Adam Dickmeiss [Wed, 29 Nov 2006 10:31:29 +0000 (10:31 +0000)]
Cut memory for sorting in half when threaded sort is in effect.

17 years agoVersion 2.0.7.
Adam Dickmeiss [Wed, 29 Nov 2006 10:24:01 +0000 (10:24 +0000)]
Version 2.0.7.

17 years agoshortened zebra::meta:.sysno lement XML format
Marc Cromme [Wed, 29 Nov 2006 09:01:53 +0000 (09:01 +0000)]
shortened zebra::meta:.sysno lement XML format

17 years agoadded example of config file
Marc Cromme [Tue, 28 Nov 2006 14:37:45 +0000 (14:37 +0000)]
added example of config file

17 years agoadded entry on encoding directive
Marc Cromme [Tue, 28 Nov 2006 14:18:26 +0000 (14:18 +0000)]
added entry on encoding directive

17 years agoadded more information on character mapping
Marc Cromme [Tue, 28 Nov 2006 13:05:57 +0000 (13:05 +0000)]
added more information on character mapping
added entry on equivalent directive
added short examples

17 years agoTowards 2.0.6. ZEBRA.2.0.6
Adam Dickmeiss [Tue, 28 Nov 2006 11:42:13 +0000 (11:42 +0000)]
Towards 2.0.6.

17 years agoIssue warning if staticrank is < 0. And set to 0
Adam Dickmeiss [Tue, 28 Nov 2006 08:44:10 +0000 (08:44 +0000)]
Issue warning if staticrank is < 0. And set to 0

17 years agoAdded assert
Adam Dickmeiss [Tue, 28 Nov 2006 08:43:53 +0000 (08:43 +0000)]
Added assert

17 years agoZebra 2 RPM with the following packages: idzebra-2.0, libidzebra-2.0,
Adam Dickmeiss [Mon, 27 Nov 2006 21:50:15 +0000 (21:50 +0000)]
Zebra 2 RPM with the following packages: idzebra-2.0, libidzebra-2.0,
libidzebra-2.0-modules, libidzebra-2.0-devel.

17 years agoRequire YAZ 2.1.40 or later
Adam Dickmeiss [Mon, 27 Nov 2006 14:27:12 +0000 (14:27 +0000)]
Require YAZ 2.1.40 or later

17 years agoAvoid indexing of zebraInfo stuff. Problem is that explain.abs has lines
Adam Dickmeiss [Mon, 27 Nov 2006 10:40:33 +0000 (10:40 +0000)]
Avoid indexing of zebraInfo stuff. Problem is that explain.abs has lines
elm (4,2)  targetInfo  ExplainCategory
and it will zebraInfo child as ExplainCategory. This is now prevented
by making a zebraInfo node with no children.

17 years agoBetter debugging for indexing.. Introduction of zebra_log_dict_entry.
Adam Dickmeiss [Mon, 27 Nov 2006 10:10:14 +0000 (10:10 +0000)]
Better debugging for indexing.. Introduction of zebra_log_dict_entry.

17 years agoFixed bug #736: Updates gets slower. The problem was that duplicate
Adam Dickmeiss [Mon, 27 Nov 2006 10:09:48 +0000 (10:09 +0000)]
Fixed bug #736: Updates gets slower. The problem was that duplicate
name tags (with database name in it) was generated for attributeDetails.
This caused indexes to grow and grow for each update iteration.

17 years agoDisable hit snippets for now. It does not work well and it slows retrieval
Adam Dickmeiss [Mon, 27 Nov 2006 09:17:01 +0000 (09:17 +0000)]
Disable hit snippets for now. It does not work well and it slows retrieval
down a lot.

17 years agoFixed bug #735: Delete fails to process rest of file after a record that
Adam Dickmeiss [Sat, 25 Nov 2006 09:15:19 +0000 (09:15 +0000)]
Fixed bug #735: Delete fails to process rest of file after a record that
does not exist,

17 years agoAdded new object files to list
Adam Dickmeiss [Fri, 24 Nov 2006 19:48:23 +0000 (19:48 +0000)]
Added new object files to list

17 years agoRephase a bit.
Adam Dickmeiss [Fri, 24 Nov 2006 13:25:42 +0000 (13:25 +0000)]
Rephase a bit.

17 years agoMention --disable-shared option as a way to prevent modules to be
Adam Dickmeiss [Fri, 24 Nov 2006 13:25:02 +0000 (13:25 +0000)]
Mention --disable-shared option as a way to prevent modules to be
built.

17 years agoMoved section about special retrieval zebra:: to Architecture chapter.
Adam Dickmeiss [Fri, 24 Nov 2006 13:05:11 +0000 (13:05 +0000)]
Moved section about special retrieval zebra:: to Architecture chapter.
Added table with supported zebra:: elements.

17 years agoupdated test since non-used attributes are not longer printed out
Marc Cromme [Fri, 24 Nov 2006 12:21:31 +0000 (12:21 +0000)]
updated test since non-used attributes are not longer printed out

17 years agoUsing wrbuf rather than sprintf. Allow all syntaxes for ::data fetch.
Adam Dickmeiss [Fri, 24 Nov 2006 11:35:23 +0000 (11:35 +0000)]
Using wrbuf rather than sprintf. Allow all syntaxes for ::data fetch.

17 years agoAdded check for sys/utsname.h.
Adam Dickmeiss [Thu, 23 Nov 2006 21:38:26 +0000 (21:38 +0000)]
Added check for sys/utsname.h.

17 years agoAdded Include of string.h
Adam Dickmeiss [Thu, 23 Nov 2006 21:35:02 +0000 (21:35 +0000)]
Added Include of string.h

17 years agoOmit test with multiple processes and update on FreeBSD. Bug #733.
Adam Dickmeiss [Thu, 23 Nov 2006 21:34:22 +0000 (21:34 +0000)]
Omit test with multiple processes and update on FreeBSD. Bug #733.

17 years agoMake buildconf.sh work for FreeBSD
Adam Dickmeiss [Thu, 23 Nov 2006 18:42:48 +0000 (18:42 +0000)]
Make buildconf.sh work for FreeBSD

18 years agoUse size of rec->info stuff in meta record fetch. Failing that will
Adam Dickmeiss [Thu, 23 Nov 2006 14:01:21 +0000 (14:01 +0000)]
Use size of rec->info stuff in meta record fetch. Failing that will
result in a SEGV or similar.

18 years agoimplemented full metadata display of fast element set names
Marc Cromme [Thu, 23 Nov 2006 09:03:50 +0000 (09:03 +0000)]
implemented full metadata display of fast element set names
implemented more test of fast element set names, including zebra::meta and zebra::data element set names
updated documentation on fast element set names

18 years agoAdded include of yaz/xmalloc.h.
Adam Dickmeiss [Wed, 22 Nov 2006 14:12:49 +0000 (14:12 +0000)]
Added include of yaz/xmalloc.h.

18 years agoRemove function tl_start_log.
Adam Dickmeiss [Wed, 22 Nov 2006 14:06:53 +0000 (14:06 +0000)]
Remove function tl_start_log.

18 years agoAdded a few memset for structures that are later written to disk.
Adam Dickmeiss [Wed, 22 Nov 2006 11:13:17 +0000 (11:13 +0000)]
Added a few memset for structures that are later written to disk.
This makes valgind happy on platforms which aligns members in structs.

18 years agoAdded initializer value for two locals
Adam Dickmeiss [Wed, 22 Nov 2006 10:26:12 +0000 (10:26 +0000)]
Added initializer value for two locals

18 years agoFor build script: -p enables profiling.
Adam Dickmeiss [Wed, 22 Nov 2006 09:42:27 +0000 (09:42 +0000)]
For build script: -p enables profiling.

18 years agoGet rid of SYSNO which is zint anyway. Removed various prototypes
Adam Dickmeiss [Tue, 21 Nov 2006 22:17:49 +0000 (22:17 +0000)]
Get rid of SYSNO which is zint anyway. Removed various prototypes
of unused/undefined functions.

18 years agoAdded
Adam Dickmeiss [Tue, 21 Nov 2006 17:55:47 +0000 (17:55 +0000)]
Added

18 years agoMake threading for index sort configurable.
Adam Dickmeiss [Tue, 21 Nov 2006 17:48:08 +0000 (17:48 +0000)]
Make threading for index sort configurable.

18 years agoRemove log msg
Adam Dickmeiss [Tue, 21 Nov 2006 14:54:12 +0000 (14:54 +0000)]
Remove log msg

18 years agoSplit it_key stuff into it_key.c. For indexing, sort key chunks in a separate
Adam Dickmeiss [Tue, 21 Nov 2006 14:32:38 +0000 (14:32 +0000)]
Split it_key stuff into it_key.c. For indexing, sort key chunks in a separate
thread (POSIX threads only).

18 years agoIncrease reckeys hash size from 1023 to 32767.
Adam Dickmeiss [Mon, 20 Nov 2006 13:59:35 +0000 (13:59 +0000)]
Increase reckeys hash size from 1023 to 32767.

18 years agoRemove obsolete SORT_EXTRA code (no longer in use).
Adam Dickmeiss [Mon, 20 Nov 2006 13:59:13 +0000 (13:59 +0000)]
Remove obsolete SORT_EXTRA code (no longer in use).

18 years agoContact address for Zebra is zebra-help@indexdata.dk
Adam Dickmeiss [Fri, 17 Nov 2006 17:32:45 +0000 (17:32 +0000)]
Contact address for Zebra is zebra-help@indexdata.dk

18 years agoupdated docs to reflect change to 'zebra::index::' special elem set name
Marc Cromme [Fri, 17 Nov 2006 14:54:00 +0000 (14:54 +0000)]
updated docs to reflect change to 'zebra::index::' special elem set name

18 years agoadded tests fror wrong element set names which are not recognized
Marc Cromme [Fri, 17 Nov 2006 14:24:56 +0000 (14:24 +0000)]
added tests fror wrong element set names which are not recognized

18 years agopreventing later memory acess violation when one leaves retrieve.c:zebra_special_fetc...
Marc Cromme [Fri, 17 Nov 2006 13:47:22 +0000 (13:47 +0000)]
preventing later memory acess violation when one leaves retrieve.c:zebra_special_fetch by error condition before setting *rec_lenp by setting int len = 0; upon initialization.

18 years agoremoving unnecessary comments
Marc Cromme [Fri, 17 Nov 2006 08:20:17 +0000 (08:20 +0000)]
removing unnecessary comments
not setting the  *rec_lenp = 0; at retrieve.c:127 to provoke test failure with segfault in the last unit test of test/api/t16.c , which is commented out. It seems like  *rec_lenp = wrbuf_len(wrbuf); at retrieve.c:247 leaves rec_lenp in an uninitialized state, which causes a segfault later. More investigation needed.

18 years agoprotected wrong buffer-length write by initializing to 0, but there is more debugging...
Marc Cromme [Thu, 16 Nov 2006 21:58:54 +0000 (21:58 +0000)]
protected wrong buffer-length write by initializing to 0, but there is more debugging needed: there is still something fishy with the last out-commented test of test/api/t16.c.