kasykhistory - history of Kasyk
Although Kasyk has been released as version 1.0.0, it is a proven system. Approximately 10 man years have already been put into the development of Kasyk. And it has been in use as a search engine for more than 7 years already. Then how come you've never heard of Kasyk before?
Kasyk started out life as NexTrieve in 1995. It was originally developed by Kim Hendrikse for his company Nexial Systems and marketed as a closed source product in the period 1995-2000. Well, marketed? The few articles that have been written about it, called it one of the best kept secrets of the Internet.
The old versions of NexTrieve were only capable of indexing ASCII characters a-z and 0-9. They only could do fuzzy searching, which at the time was quite unique in the world. However, these versions of NexTrieve also had scalability problems and the fuzzy searching, without an exact searching component, had some weird borderline cases that couldn't be fixed without a major architectural overhaul. During the period 1997-1999 Hendrikse did some development work at introducing exact searching to NexTrieve, but this never made it to the production stage.
My company xxLINK Internet Services was one of the first users of NexTrieve. xxLINK was the first commercial web-presence provider in the Netherlands (since 1994). I developed a Perl module Nextrieve.pm, that was used internally at xxLINK to interface with NexTrieve. With this Perl module, xxLINK was able to offer high quality search functionality for the websites of its clients. And thereby provided a nice cash flow for xxLINK.
From 1996, Search.NL was started by Hendrikse and me as a search engine service for the Dutch webspace. This web search engine served as a showcase for the NexTrieve software and the hosting services of xxLINK Internet Services. It was co-maintained by Nexial Systems and xxLINK Internet Services. Some efforts were made to run this service commercially, but they all failed.
After xxLINK was sold at the end of 1999, I made it clear to Hendrikse that I was interested in taking NexTrieve to a new level by investing some of the money I had made. Together with my partner Wendy G.A. van Dijk, I started a new company "Dijkmat", which was to do these types of investments in startup companies. A not uncommon phenomenon in those crazy days of the Internet bubble.
Hendrikse was interested. This resulted in the founding of a joint-venture Search Holding, co-owned by Dijkmat and Kiwi (a holding company of Kim Hendrikse and his partner Willemijn Kivits), which opened its offices in Echt (in the south of the Netherlands) at the end of the summer of 2000. This join-venture also tried to further commercialize the Search.NL search engine service.
Gordon Clare, an old-time university buddy of Kim Hendrikse, and Hendrikse himself were hired by Search Holding to further develop the NexTrieve software. After a 3-day brainstorming session by Hendrikse, Clare and me in December 2000, development work on the new NexTrieve software was started.
Instead of only being able to index a-z and 0-9, the new version of NexTrieve was to be Unicode-aware and therefore able to index any language in the world. Other enhancements included an XML-interface for addressing each of the functionalities of NexTrieve, such as indexing and searching. And exact searching, text types with different weights and extended attribute and constraint capabilities. And a bunch of other goodies such as basically no RAM limitation on the size of the document being indexed, or the number of documents being indexed.
The new version of NexTrieve was basically started by Clare from scratch. In the end, the only things that remained of the old NexTrieve was the state machine that was used to quickly evaluate a constraint on a document, and the concept of using trigrams (character triplets) for fuzzy searching. But even the state machine was significantly enhanced to allow for string matching operations (previously, only numeric and flag checking were possible) and allowing sets of numbers to be specified with the in() function.
During 2001, I developed a newsgroup search engine service of Dutch newsgroups for Search.NL, based on the new NexTrieve architecture. This service opened its doors at the end of the summer of 2001 for the Dutch public. Although it was a technological success, because it tested the new capabilities of NexTrieve thoroughly (shaking out many bugs), it never became a commercial success and was closed down in early 2002.
At the end of 2001, I developed a completely new set of Perl modules for the new version of NexTrieve and released these to the world as NexTrieve.pm (available from the Comprehensive Perl Archive Network, or CPAN). This supported filtering operations for email, HTML, relational databases, PDF-files and Microsoft Word documents, which in turn made it possible to search the contents of all of these types of information.
In January 2002, all the hard work that was completely funded by Dijkmat (about 4 man years worth) of Clare, Hendrikse and me paid off: the new version of NexTrieve was ready for the general public. A new website was created and made on-line and NexTrieve was offered to the public at a 500 US$ license fee per server. Even an on-line payment system was developed. NexTrieve was ready to be unleashed on the world. But it didn't happen.
By March 2002, funding by Dijkmat had run out. Kiwi notified Dijkmat that the joint-venture agreement was null and void because Dijkmat "had not invested enough" in Search Holding. Hendrikse told me that he didn't want my technical involvement in NexTrieve anymore either. Nexial assumed property of the newly developed NexTrieve software but did not make any effort to make NexTrieve more known in the world or make any money with it.
Several proposals were made by me to solve the problems, with the major ingredient always being releasing the source code of NexTrieve under the GNU General Public License.
A period of stalemate started which involved many meetings, official letters, lawyers, judges and court hearings and court judgements.
On the evening of Sunday, January 12, 2003, Hendrikse made the complete source code of NexTrieve available on a new version of the NexTrieve website. Under the completely unrestrictive MIT-license.
Dijkmat decided, in an effort to reduce the damage done by Hendrikse, to take the source code of NexTrieve and create a source code split named Kasyk: Knowhow About Searching Your Knowledge. The name is inspired by my mother's maiden name "Kasik".
In an effort to create a valid business using a business model similar to the one MySQL AB uses, the source code was released under the GNU General Public License. Binary versions of Kasyk will also be made available to the general public at a license fee. Support services for Kasyk are also being offered by Dijkmat and me in particular.
So, here it is, available to you, after it having been gathering dust for a year and after my dust removal. For all of you creating or using GPL'ed software out there. For all of you with searching needs (and who doesn't need to find something somewhere sometime). Dijkmat will continue the support of Kasyk in the future by only putting in a lot of effort (instead of a lot of money also). That is what my dust removal was: I went over all of the source, reorganized source files, started uniforming the XML, solved some problems, fixed a (heisen)bug in the threaded Kasyk server, further documented the source, rewrote most of the documentation from scratch, removed redundant code, added some features and finished the first stage of a huge test suite.
In the unlikely event that you should find a bug, please report it to bugs@Kasyk.org. If you are able to supply a patch for the bug, it will be deeply appreciated by all people that are or will be using Kasyk in the future.
There is a huge list of proposed enhancements which you will find in the TODO file. If you have ideas of enhancements of your own, please let them be known to ideas@Kasyk.org.
April 2003, Elizabeth Mattijsen, Dijkmat BV, Bosstraat 21, 6101 NV Echt, The Netherlands
Kasyk home, Kasyk introduction, vision behind Kasyk, future of Kasyk.
See http://www.kasyk.nl/history.html for the most up-to-date version of this information.
Copyright © 2003 Dijkmat BV
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Kasyk Information: Kasyk version 1.0.0, generated on Tue Nov 25 12:09:47 2003.