api-search-elasticsearch

Elasticsearch Patterns

Quick Guide: Use @elastic/elasticsearch (v8.x/v9.x) as the TypeScript client. Elasticsearch is near real-time -- documents are NOT searchable immediately after indexing; they become visible after a refresh (default: every 1 second on active indices). You MUST define explicit mappings before indexing -- dynamic mapping infers types from the first document, and mismatched types in later documents cause hard failures you cannot fix without reindexing. Use search_after + Point in Time (PIT) for deep pagination -- NOT from/size beyond 10,000 hits and NOT the scroll API (deprecated for search). Use the bulk API or client.helpers.bulk() for any batch operation -- never loop individual index calls.

<critical_requirements>

CRITICAL: Before Using This Skill

All code must follow project conventions in CLAUDE.md (kebab-case, named exports, import ordering, import type, named constants)

(You MUST define explicit index mappings BEFORE indexing documents -- dynamic mapping infers types from the first document, and if a later document sends a different type for the same field, indexing fails with a mapper_parsing_exception that CANNOT be fixed without reindexing into a new index)

(You MUST use the bulk API for batch operations -- looping individual client.index() calls is orders of magnitude slower and can overwhelm the cluster with HTTP connections)

(You MUST NOT use from/size pagination beyond 10,000 results -- Elasticsearch throws Result window is too large by default; use search_after + PIT instead)

(You MUST NOT use refresh: true or refresh: "wait_for" in production request handlers -- forcing a refresh on every write degrades cluster performance; let the default 1-second refresh interval handle it)

</critical_requirements>

Examples

• [Core Patterns](examples/core.md) -- Client setup, index management, document CRUD, search basics, TypeScript integration
• [Aggregations](examples/aggregations.md) -- Terms, range, date_histogram, nested, pipeline aggregations
• [Vector Search](examples/vector-search.md) -- Dense vector fields, kNN queries, hybrid search, similarity metrics
• [Pagination](examples/pagination.md) -- from/size, search_after, Point in Time, scroll helpers
• [Bulk Operations](examples/bulk-operations.md) -- Bulk API, bulk helper, reindexing patterns

Additional resources:

• [reference.md](reference.md) -- Search DSL cheat sheet, mapping types, aggregation reference, decision frameworks

Auto-detection: Elasticsearch, elasticsearch, @elastic/elasticsearch, client.search, client.index, client.bulk, client.indices.create, client.indices.putMapping, dense_vector, knn, search_after, point in time, openPIT, aggregations, aggs, bool query, match query, term query, multi_match, nested query, range query, client.helpers.bulk, client.helpers.scrollSearch, BulkResponse, SearchResponse, MappingProperty

When to use:

• Full-text search with advanced relevance tuning (BM25, custom analyzers, boosting)
• Aggregations and analytics (terms, histograms, pipeline aggregations)
• Vector/semantic search with kNN on dense_vector fields
• Log and event data search with time-based queries
• Complex structured queries combining bool, nested, range, and geo filters
• Search across large datasets requiring deep pagination (search_after + PIT)

Key patterns covered:

• Client initialization and connection management
• Index management with explicit mappings and settings
• Document CRUD (index, get, update, delete)
• Search DSL (match, term, bool, range, nested, multi_match)
• Aggregations (terms, range, date_histogram, nested, pipeline)
• Full-text analysis (custom analyzers, tokenizers, filters)
• Vector search (dense_vector, kNN, hybrid text+vector)
• Bulk operations and reindexing
• Deep pagination (search_after + PIT)

When NOT to use:

• Simple keyword search on small datasets (client-side filtering or database LIKE queries are simpler)
• Primary data store (Elasticsearch is a search engine, not a database -- always have a source of truth elsewhere)
• Strong consistency requirements (Elasticsearch is eventually consistent by design)
• Simple autocomplete on a small list (a prefix trie or client-side filter is simpler)

<philosophy>

Philosophy

Elasticsearch is a distributed search and analytics engine built on Apache Lucene. It excels at full-text search, structured queries, aggregations, and vector search at scale. Core principles:

1. Near real-time, not real-time -- Documents are indexed into segments. A refresh (default: every 1 second on active indices) makes new segments searchable. Do not expect immediate consistency after writes.
2. Mappings are immutable -- Once a field type is set (text, keyword, integer, etc.), it cannot be changed. Wrong types require reindexing into a new index. Always define mappings explicitly before first document.
3. Search engine, not database -- Elasticsearch should not be your source of truth. Always have a primary database and sync to Elasticsearch for search.
4. Bulk everything -- The bulk API amortizes HTTP overhead across thousands of operations. Never loop individual index/update/delete calls.
5. Pagination has limits -- from/size is capped at 10,000 hits by default (index.max_result_window). Deep pagination requires search_after + Point in Time (PIT). The scroll API is deprecated for search use cases.
6. Text vs keyword matters -- text fields are analyzed (tokenized, lowercased) for full-text search. keyword fields are exact-match only. Getting this wrong means either broken search or broken aggregations/filters.

</philosophy>

<patterns>

Core Patterns

Pattern 1: Client Setup

Initialize the client with node URL and authentication. The client supports Elastic Cloud, API keys, basic auth, and bearer tokens.

// Good Example -- Typed client setup with environment validation
import { Client } from "@elastic/elasticsearch";

function createElasticsearchClient(): Client {
  const node = process.env.ELASTICSEARCH_URL;
  if (!node) {
    throw new Error("ELASTICSEARCH_URL environment variable is required");
  }

  return new Client({
    node,
    auth: {
      apiKey: process.env.ELASTICSEARCH_API_KEY ?? "",
    },
  });
}

export { createElasticsearchClient };

Why good: Environment variable validation, named export, API key auth (preferred over basic auth in production)

// Bad Example -- Hardcoded credentials
import { Client } from "@elastic/elasticsearch";
const client = new Client({
  node: "http://localhost:9200",
  auth: { username: "elastic", password: "changeme" },
});

Why bad: Hardcoded node URL and credentials leak in version control, basic auth with default password

See [examples/core.md](examples/core.md) for Elastic Cloud setup, health checks, and child clients.

Pattern 2: Index with Explicit Mappings

Always define mappings before indexing. Dynamic mapping infers types from the first document -- if wrong, you must reindex.

// Good Example -- Explicit mappings with text + keyword multi-field
const INDEX_NAME = "products";

await client.indices.create({
  index: INDEX_NAME,
  settings: {
    number_of_replicas: 1,
    refresh_interval: "1s",
  },
  mappings: {
    properties: {
      name: {
        type: "text",
        fields: { keyword: { type: "keyword" } },
      },
      description: { type: "text", analyzer: "standard" },
      price: { type: "float" },
      categories: { type: "keyword" },
      inStock: { type: "boolean" },
      createdAt: { type: "date" },
    },
  },
});

Why good: Explicit types prevent mapping conflicts, text + keyword multi-field allows both full-text search and exact-match filtering/aggregation on name

// Bad Example -- No mappings, relying on dynamic mapping
await client.indices.create(