Indexing

Indices and languages

Create an index, pick its language, read its status, and change the language later with a reindex.

What an index is

An index holds one catalog: its documents, its mapping (the fields and what each is used for), its search profiles, synonyms and ranking rules. A project can hold several indices, for example one per shop language or one per brand. Searches never cross indices.

Each index has one language. The language decides how text is split into words, which word forms count as the same word and which short words are ignored.

Create an index

Give the index a name and a language. The name is unique in its project and 1 to 100 characters long. The language defaults to english.

The New index dialog with a name field and the language list open, English (US) selectedCreate an index from the Indices page

Leave out the fields and the index starts with only id. The first document upload then maps every field it can recognise. To control the mapping from the start, send fields with the create request. See inferred mapping.

Open Indices and click New index. Enter a Name, pick a Language and click Create index. Only organization owners and admins see the button.

POST/v1/projects/:projectId/indicesProject admin or a full-access secret keyGET/v1/projects/:projectId/indicesProject member or any secret keyGET/v1/projects/:projectId/indices/:indexIdProject member or any secret keyDELETE/v1/projects/:projectId/indices/:indexIdProject admin or a full-access secret key

The list and the single index return id, name, language, status, docCount, activeVersion and buildingVersion. The single index also returns activeReindexTaskId and openReplaceSessionId, so a client can pick up a running reindex or an open replace session again.

DELETE answers 204 and removes the index with its documents, profiles, synonyms and rules at once. Tasks that have not finished are cancelled. There is no undo.

A key limited to some indices cannot create an index: the API answers 403 INDEX_SCOPED_KEY. Such a key sees only its own indices in the list. Any other index id answers 404.

Index status

status tells you whether the index has what its saved mapping asks for. Search keeps working in every status except CREATING.

An index overview with the READY badge, the document count, active version v1, the language, search stats and a four-step setup checklist, all doneThe index overview shows its status and what is left to set up

StatusWhat it meansWhat to do
CREATINGThe index is being set up. It lasts a moment. If setup fails, the index is removed again.Wait.
READYThe live version matches the saved mapping.Nothing.
NEEDS_REINDEXYou saved a mapping with a breaking change. Search and writes use the live version with the previous mapping.Reindex.
REINDEXINGA new version is being built, by a reindex or a replace session. Writes reach both versions.Wait, or cancel.
FAILEDThe last build failed and was dropped. Search uses the live version as before.Read the task's errors, then reindex again.

The cabinet shows the status on the Indices list and on the index overview. The overview also shows a Reindex needed warning while the status is NEEDS_REINDEX.

Languages

Every index language and the analysis it gets:

Index languages
LanguagelanguageAnalysis
English (US)englishStemming and stop words
BulgarianbulgarianStemming and stop words
CroatiancroatianBasic matching: no word-form reduction
DanishdanishStemming and stop words
DutchdutchStemming and stop words
EstonianestonianStemming and stop words
FinnishfinnishStemming and stop words
FrenchfrenchStemming and stop words
GermangermanStemming and stop words
GreekgreekStemming and stop words
HungarianhungarianStemming and stop words
ItalianitalianStemming and stop words
LatvianlatvianStemming and stop words
NorwegiannorwegianStemming and stop words
PolishpolishStemming and stop words
PortugueseportugueseStemming and stop words
RomanianromanianStemming and stop words
RussianrussianStemming and stop words
SerbianserbianStemming and stop words
SlovenianslovenianBasic matching: no word-form reduction
SpanishspanishStemming and stop words
SwedishswedishStemming and stop words
Language-neutralstandardBasic matching: no word-form reduction

Language tiers

Languages come in two tiers.

  • Stemming and stop words. Stemming reduces word forms, so "boxes" finds "box". Stop words such as "the" and "and" are ignored.
  • Basic matching: no word-form reduction. Text is lowercased and accents are folded, so "Café" finds "cafe". "boxes" does not find "box". Croatian and Slovenian are in this tier. standard (Language-neutral) is too: use it for a catalog in several languages or one made mostly of codes.

The cabinet groups the language picker by tier. The index overview shows the tier under the language when the language has basic matching only.

A single searchable field can use a different language from its index, for example an English title in a German index. See the field's Language option.

Change the language

The language is part of the mapping. A change re-analyses every text field, so it is a breaking change and needs a reindex. The cabinet has no control for it. Send the whole mapping with the new language:

PUT/v1/projects/:projectId/indices/:indexId/mappingProject admin or a full-access secret key
curl https://api.zaproo-search.com/v1/projects/$PROJECT_ID/indices/$INDEX_ID/mapping \
  -H "Authorization: Bearer $ZS_KEY" > mapping.json
# edit "language" in mapping.json, then drop the "unmapped" list
curl -X PUT https://api.zaproo-search.com/v1/projects/$PROJECT_ID/indices/$INDEX_ID/mapping \
  -H "Authorization: Bearer $ZS_KEY" -H "Content-Type: application/json" -d @mapping.json

The answer's diff lists the language change as breaking and the index moves to NEEDS_REINDEX. The index's language shows the new value at once, but search keeps the old analysis until you reindex.

Errors

CodeWhen
409 INDEX_NAME_TAKENThe project already has an index with this name.
403 INDEX_SCOPED_KEYA key limited to some indices tried to create one.
400 LANGUAGE_UNAVAILABLEThe search cluster lacks the OpenSearch plugin the language needs. Polish needs analysis-stempel. The hosted service has it.
404 INDEX_NOT_FOUNDThe index does not exist, belongs to another project or is outside the key's scope.