Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 40 additions & 1 deletion docs/how-to/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

This page contains how-to documentation for installing, configuring, and running CipherStash Proxy.

> [!IMPORTANT]
> This guide is for Proxy 3.x and EQL v3. Proxy 2.x uses EQL v2's separate
> encrypted-column and search-index configuration; use the
> [Proxy 2.2 documentation](https://github.com/cipherstash/proxy/tree/v2.2.4/docs)
> when operating a 2.x deployment. Do not mix configuration instructions from
> the two versions.

Comment thread
coderabbitai[bot] marked this conversation as resolved.
## Table of contents

- [Installing Proxy](#installing-proxy)
Expand All @@ -11,6 +18,7 @@ This page contains how-to documentation for installing, configuring, and running
- [Running Proxy locally](#running-proxy-locally)
- [Setting up the database schema](#setting-up-the-database-schema)
- [Creating columns with the right types](#creating-columns-with-the-right-types)
- [Bloom-filter text matching](#bloom-filter-text-matching)
- [Encrypting data in an existing database](#encrypting-data-in-an-existing-database)

## Installing Proxy
Expand All @@ -27,7 +35,10 @@ services:
db:
# Your Postgres container config
proxy:
image: cipherstash/proxy:latest
# Pin a Proxy 3.0 release — see https://hub.docker.com/r/cipherstash/proxy/tags
# for available releases. `latest` may point at a newer major or minor version
# whose configuration differs from this guide.
image: cipherstash/proxy:3.0
container_name: proxy
ports:
- 6432:6432
Expand Down Expand Up @@ -227,6 +238,34 @@ When deploying CipherStash Proxy into production environments with real data, we

To see more examples of how to modify your database schema, check out [the example schema](../sql/schema-example.sql) from [Getting started](#getting-started).

### Bloom-filter text matching

The `eql_v3_text_match`, `eql_v3_text_search`, and
`eql_v3_text_search_ore` domains carry a Bloom-filter (`bf`) term for fuzzy
text matching. Proxy derives the match-index configuration from the domain;
there is no separate search-config row or per-column SQL configuration in EQL
v3.

Proxy currently uses these fixed parameters:

| Parameter | Value | Effect |
|-----------|-------|--------|
| n-gram length | `3` | Text is tokenized into overlapping three-character tokens. Inputs shorter than three characters do not produce a match token. |
| Bloom filter size (`m`) | `2048` bits | Sets the size of the probabilistic match term. |
| Hash count (`k`) | `6` | Sets how many Bloom-filter positions each token occupies. |

Before tokenization, Proxy strips a leading or trailing `%` and then a leading
or trailing `_`. It applies this preprocessing to stored values as well as
query operands, even when those characters are literal data. Proxy then
generates overlapping three-character tokens and downcases each token.
Wildcard characters within the input are not interpreted specially.

Bloom matching is probabilistic: matching rows contain every bit set by the
query term, but unrelated values can occasionally be false positives. Changing
the token length, `m`, or `k` would make existing stored and query terms
incompatible, so EQL v3 does not expose those values as per-column tuning
options.

## Encrypting data in an existing database

CipherStash Proxy includes an `encrypt` tool – a CLI application to encrypt existing data, or to apply index changes after changes to the encryption configuration of a protected database.
Expand Down
11 changes: 8 additions & 3 deletions docs/reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

This page contains reference documentation for configuring CipherStash Proxy and its features.

> [!IMPORTANT]
> This reference describes Proxy 3.x and EQL v3. For a Proxy 2.x deployment,
> use the [Proxy 2.2 reference](https://github.com/cipherstash/proxy/tree/v2.2.4/docs/reference);
> EQL v2 configuration is not compatible with EQL v3 domain types.

## Table of contents

- [Proxy config options](#proxy-config-options)
Expand Down Expand Up @@ -118,9 +123,9 @@ connection_timeout = "300000"
# Env: CS_DATABASE__WITH_TLS_VERIFICATION
with_tls_verification = "false"

# Encrypt configuration reload interval in sec
# Sets how frequently Encrypted index configuration should be reloaded
# The configuration specifies the encrypted columns in the database
# EQL domain/schema reload interval in sec
# Sets how frequently Proxy refreshes the encryption configuration it derives
# from EQL v3 column domain types in the database schema
# Optional
# Default: `60`
# Env: CS_DATABASE__CONFIG_RELOAD_INTERVAL
Expand Down
80 changes: 12 additions & 68 deletions docs/reference/searchable-json.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

This document outlines the supported JSONB functions and operators in CipherStash Proxy for encrypted data.

> [!IMPORTANT]
> This page is for Proxy 3.x and EQL v3. Proxy 2.x requires a separate EQL v2
> `ste_vec` search configuration; follow the
> [Proxy 2.2 searchable JSON documentation](https://github.com/cipherstash/proxy/blob/v2.2.4/docs/reference/searchable-json.md)
> for a 2.x deployment.


## Table of Contents

Expand Down Expand Up @@ -36,76 +42,14 @@ This document outlines the supported JSONB functions and operators in CipherStas

EQL v3 encrypted-JSON columns are self-configuring: the `eql_v3_json_search`
domain type is the SteVec (searchable encrypted JSON) configuration, so the
column type alone enables JSON search. There is no separate
`add_search_config` call as in EQL v2.
column type alone enables JSON search. Do not create a separate search
configuration for this column.

> **Note:** JSONB literals in INSERT and UPDATE statements work directly without explicit `::jsonb` type casts. The proxy infers the JSONB type from the target column and handles encryption transparently.

#### Configuration options

> **EQL v2 legacy:** In EQL v2 the `ste_vec` index was configured explicitly via
> `add_search_config`, and the options below (and the `add_search_config` examples
> in this section) describe that mechanism. In EQL v3 the `eql_v3_json_search`
> domain type carries a fixed default configuration, so these options are not
> set per-column via SQL. The descriptions are retained to explain the indexing
> behaviour.

The `ste_vec` index configuration accepts the following options:

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `prefix` | string | (required) | Unique prefix for the index, typically `table/column` |
| `term_filters` | array | `[]` | Filters applied to indexed terms (e.g., `[{"kind": "downcase"}]`) |
| `array_index_mode` | string or object | `"all"` | Controls which array selectors are generated during indexing |

#### Array index mode

The `array_index_mode` option controls how arrays within JSONB documents are indexed. This affects which JSONPath selectors can be used to query array data.

**Preset values:**

- `"all"` (default) - Generates all selector types. This is backwards compatible with existing configurations.
- `"none"` - Disables array indexing entirely.

**Object form for fine-grained control:**

```json
{
"item": true,
"wildcard": true,
"position": false
}
```

| Selector | JSONPath | Description |
|----------|----------|-------------|
| `item` | `[@]` | EQL array element selector for functions like `jsonb_array_length` |
| `wildcard` | `[*]` | Standard JSONPath wildcard for iterating array elements |
| `position` | `[0]`, `[1]`, etc. | Positional access to specific array indices |

**Example with array_index_mode:**

```sql
SELECT eql_v2.add_search_config(
'cipherstash',
'encrypted_jsonb',
'ste_vec',
'jsonb',
'{"prefix": "cipherstash/encrypted_jsonb", "array_index_mode": "all"}'
);
```

**Example disabling positional indexing:**

```sql
SELECT eql_v2.add_search_config(
'events',
'payload',
'ste_vec',
'jsonb',
'{"prefix": "events/payload", "array_index_mode": {"item": true, "wildcard": true, "position": false}}'
);
```
Proxy derives a unique `table/column` selector prefix, applies no term filters,
and indexes array item, wildcard, and positional selectors. These settings are
fixed in EQL v3 and are not configured per column.

### JSON document structure

Expand Down Expand Up @@ -734,4 +678,4 @@ The following JSON data types are fully supported:
- Non-existent fields return `NULL`
- Invalid JSONPath expressions may cause query errors
- Type mismatches in comparisons follow PostgreSQL JSONB semantics
- Array functions on non-arrays return empty results
- Array functions on non-arrays return empty results
Loading