Index new meta field and plans for future

In this post we want to introduce new field “updated_time”.
“updated_time” is a string with the time when index was modified formatted according to the ISO 8601 format.

To see metadata you can use methods “GET /v1/indexes” or “GET /v1/index/name” which will return metadata for an index.
This metadata will be in the form of a JSON object and will include the following fields:

    "started": true,
    "code": "y4ary7ubus",
    "creation_time": "2012-08-16T11:31:44+01:00",
    "size": "507",
    "updated_time": "2013-04-14T16:26:39+01:00",
    "public_search": false,
    "status": "STARTED"

This was a tiny addition to current meta data, but we are in a process of development very cool statistics dashboard.
We are hope that it will help you better understand an effect of search requests and
give you good insights about usage patterns of you indexes.

Big thanks to Easy store hosting for insights!

Fell free to try IndexDen – we are open for everyone!

IndexDen E-commerce cheat sheet

In this tutorial I will collect best practices and most useful information about creating e-commerce application using IndexDen. I will talk about such entities like Products, Categories and Custom fields which are the basic data of any e-commerce application.

Getting started

Download IndexDen API client library:
IndexDen API’s client libraries overview

Documentation and tutorials for most common libraries:
PHP, Python, Ruby, Java, .NET and Rails

Must read before start programming:

This is only first “Getting started!” part of the tutorial. In next part I will describe how to organize data like Products, Categories and Custom fields in indexes to gain maximum of IndexDen abilities.



IndexDen for E-commerce

Lately we added several new features to IndexDen (IndexTank API).
The key goal was to integrate e-commerce solutions and IndexDen.

What we have done:

  1. New search options
  2. Improved internal storage for Categories and Variables
  3. Improved clients API (right now only for PHP)

New search options

Category Rollup

First search options is “category_rollup” – comma-separated list of categories. Use category_rollup to generate facets without respect of current filters.

For example.
You have Brands category for cell phones with values: Palm, Samsung and Motorolla.
When user will choose one of the brand i.e. Samsung then facets list will look like :

It is good, but id doesn’t say how many entries in the other Categories: Palm and Motorolla.

Now with category_rollup it is possible to see it. Use category_rollup=brand in the search request and you will got following results:
Motorolla (30)

So, even when filter is applied for the Brand you can see facets for other Brands too.

Match any field

Second option “match_any_field” allow to search inside all fields in the index.

For example we have two fields: text and title. Using match_any_field=true IndexDen will search inside both fields.
So, it is more not necessary to specify each field is search query like: title:some query OR text:some query.


Improved internal storage for Categories and Variables

We optimized internal storage-search system to support hundreds of Categories and Variables.
It means that you could add hundreds of product “custom fields” and product categories to your product index.

Improved clients API

To support new functions we also improved existing PHP API. We added support for new parameters to the search function.
You can check it out: IndexTank PHP API

How to find related or duplicate items with IndexDen

Many applications and websites faced with the challenge of finding “related items” like:

  • Related articles in a blog
  • Related products in a shop
  • And so on

How to determine related item?
In the comparison of text object – related items determined based on the percentage of text similarity. For instance if text A similar more than on 80% to text B then both text object are related to each other.

How IndexDen can help?

Recently IndexDen added new feature to the API – quorum operator. With quorum operator you could match those documents that pass a given threshold of given words.
For example: search query like “the world is a wonderful place”/3 will match all documents that have at least 3 of the 6 specified words.

Ok, lets try real example with IndexDen.

Continue reading

New Fuzzy search feature – quorum operator in IndexDen API

To continue this article on “Fuzzy Search – find what I meant, not what I said!” we’re at IndexDen developed a new fuzzy searching query operator.

First I want to summarize IndexDen Fuzzy searching tools:

  • Autocomplete service that allows you to see words from you corpus as you start typing them;
  • Stemming (english and russian) allows you to find documents with linguistically similar words. For instance when you’re searching for “dogs” you will find documents which contain word “dog” or if you’re searching for word “running”  you will find documents which contain “run”, “runner” and other similar words;

This is all good tools which helps a lot, but what if you’re searching for very specific phrase which are not exists in any document. For instance you’re search for “pear pie by grandma recipe”, there is a great chance that no documents will be matched by such specific query.

To solve this problem in the original IndexTank API, you need to perform two or more queries with less search terms each time. This is very expensive, because each query take time and with latency of 200ms – three queries will take about 600ms of overall response time.

That’s why we’re now releasing a third tool for this need: quorum operator which will solve a problem described above. This feature allow match those documents that pass a given threshold of given words. Example:

$index->search(' "pear pie by grandma recipe"/3 ');

Will match all documents that have at least 3 of the 5 specified words.

In conclusion I want to say that this is great addition to the IndexDen API, because it allow using only one query to get a bunch of relevant results.

If you think this is what your search solution needs or just want to try it out, join us!